Merge branch 'raxod502:master' into master

CONTRIBUTING.md

@@ -1,6 +1,7 @@
-# Contributing guide
+# Contributing guide / Documentation
 
 * [Criteria for language inclusion](doc/what-languages.md)
+* [Running Riju locally](doc/local.md)
 * [How to add your own language to Riju](doc/tutorial.md)
 * [Deep dive on Riju build system](doc/build.md)
 * [Riju infrastructure layout](doc/infra.md)

README.md

@@ -47,7 +47,7 @@ being nice include:
 * *Mining cryptocurrency.* Since hosting Riju comes out of my
   paycheck, this is exactly equivalent to stealing, which isn't nice.
 
-## Can I help?
+## Can I help? / Documentation
 
 Absolutely, please see [Contributing guide](CONTRIBUTING.md).
 

doc/local.md

@@ -0,0 +1,110 @@
+# Running Riju locally
+
+This document has the steps necessary to build and run Riju locally,
+without modifications. See the other documentation pages for
+information about customizing and rebuilding Riju, or extending it by
+adding or modifying the supported languages.
+
+If you run into any trouble following the guide, please do not
+hesitate to open an issue!
+
+## Project setup
+
+Clone locally:
+
+```
+$ git clone https://github.com/raxod502/riju.git
+$ cd riju
+```
+
+Install [Docker](https://www.docker.com/). Then you can build and
+start the admin shell:
+
+```
+$ make image shell I=admin
+```
+
+All future operations can be done inside the admin shell, where Riju's
+dependencies are already installed.
+
+## Start tmux
+
+Start a tmux session:
+
+```
+$ make tmux
+```
+
+If you don't know how to use tmux, see [a
+cheatsheet](https://danielmiessler.com/study/tmux/). The useful
+keybindings are:
+
+* `control-b c`: open new tab
+* `control-b p/n`: previous/next tab
+* `control-b "`: split tab into top and bottom panes
+* `control-b %`: split tab into left and right panes
+* `control-b <arrows>`: move between panes
+* `control-b control-b <something>`: if you have two tmuxes nested,
+  use `control-b` twice to do a command on the inner one instead of
+  the outer one
+
+## Fetch base Ubuntu image
+
+Make sure you're using the same version of Ubuntu as the mainline
+Riju:
+
+```
+$ make sync-ubuntu
+```
+
+## Start Riju server
+
+Use `dep`, the Riju build tool, to compile the Docker image that the
+Riju server will run inside:
+
+```
+$ dep image:runtime
+```
+
+Start Riju in development mode:
+
+```
+$ make shell I=runtime E=1 CMD="make dev"
+```
+
+You should now be able to navigate to <http://localhost:6119> and see
+that Riju is running, although it does not have any languages
+installed.
+
+## Build languages
+
+Building all languages supported by Riju is a lengthy process which
+often requires some debugging as something upstream has broken since
+the last time I built everything. You may be more interested in
+getting your favorite language(s) up and running. To do that, find the
+relevant language (say `foo.yaml`) in the `langs` subdirectory, and
+run:
+
+```
+$ dep image:lang-foo
+```
+
+After this completes successfully, you'll automatically be able to use
+that language in the web interface without needing to restart.
+
+You can also run:
+
+```
+$ dep test:lang-foo
+```
+
+This will, in addition to building the language image, run the defined
+integration tests to make sure it is functioning properly.
+
+If you *do* want to build and test all supported languages, run:
+
+```
+$ dep deploy:ready
+```
+
+Expect this to take many hours.

doc/selfhosting.md

@@ -209,8 +209,21 @@ This is the URL for your public ECR repository.
 
 Obtain the DNS record for Riju's ALB from `terraform output` and
 install it as a CNAME record in your DNS panel. After DNS propagates,
-you should now be able to receive a 502 from Riju with no body
-content.
+you should now be able to receive a 502 from the load balancer.
+
+## Launch instance
+
+Navigate to your EC2 dashboard instances. Select "Launch instance from
+template" and select `riju-server` for the launch template.
+
+## Attach to target group
+
+Once your instance is running you can attach it to the target group.
+Navigate to Load Balancing > Target Groups. Select `riju-server-http`
+and register the instance that you just launched. Within a minute or
+two, you should still be getting 502s, but now with an empty response
+body (these are now coming from the Riju server itself rather than
+from the load balancer).
 
 ## Build and deploy
 

docker/runtime/install.bash

@@ -38,6 +38,7 @@ clang
 docker-ce-cli
 make
 nodejs
+python3-pip
 yarn
 
 # packaging tools

packer/cloudwatch.json

@@ -5,7 +5,6 @@
   },
   "metrics": {
     "append_dimensions": {
-      "RijuInstanceGroup": "Webserver",
       "ImageId": "${aws:ImageId}",
       "InstanceId": "${aws:InstanceId}",
       "InstanceType": "${aws:InstanceType}"
@@ -16,15 +15,24 @@
     ],
     "metrics_collected": {
       "cpu": {
+        "append_dimensions": {
+          "RijuInstanceGroup": "Webserver"
+        },
         "measurement": ["usage_active"],
         "metrics_collection_interval": 60
       },
       "disk": {
+        "append_dimensions": {
+          "RijuInstanceGroup": "Webserver"
+        },
         "measurement": ["used_percent"],
         "metrics_collection_interval": 60,
         "resources": ["/", "/mnt/riju"]
       },
       "mem": {
+        "append_dimensions": {
+          "RijuInstanceGroup": "Webserver"
+        },
         "measurement": ["mem_used_percent"],
         "metrics_collection_interval": 60
       }

packer/promtail.yaml

@@ -22,13 +22,18 @@ scrape_configs:
     relabel_configs:
       - source_labels:
           - __journal__systemd_unit
-        regex: "(docker|riju)\\.service"
+        regex: "(docker|promtail|riju)\\.service"
         action: keep
       - source_labels:
           - __journal__systemd_unit
         regex: "docker\\.service"
         target_label: source
         replacement: "dockerd"
+      - source_labels:
+          - __journal__systemd_unit
+        regex: "promtail\\.service"
+        target_label: source
+        replacement: "promtail"
       - source_labels:
           - __journal__systemd_unit
         regex: "riju\\.service"