Start writing tutorial
This commit is contained in:
Radon Rosborough
parent
82c7f6d4af
commit
a2d889860d
132
doc/tutorial.md
132
doc/tutorial.md
|
|
@ -1 +1,133 @@
|
|||
# Adding your own language to Riju
|
||||
|
||||
Hello and welcome! This tutorial guides you through the basics of
|
||||
adding a new language to Riju, or modifying an existing language. The
|
||||
other documentation in this repo has reference material that may be
|
||||
helpful for advanced use cases, but this page should get you started.
|
||||
|
||||
If you run into any trouble following the guide, please do not
|
||||
hesitate to open an issue!
|
||||
|
||||
## Project setup
|
||||
|
||||
Fork this repository to your account on GitHub, and clone it locally:
|
||||
|
||||
```
|
||||
$ git clone https://github.com/yourname/riju.git
|
||||
$ cd riju
|
||||
```
|
||||
|
||||
Install [Docker](https://www.docker.com/). Then you can build and
|
||||
start the admin shell:
|
||||
|
||||
```
|
||||
$ make image shell I=admin
|
||||
```
|
||||
|
||||
All future operations can be done inside the admin shell, where all
|
||||
dependencies are installed automatically.
|
||||
|
||||
## Start tmux
|
||||
|
||||
Start a tmux session:
|
||||
|
||||
```
|
||||
$ make tmux
|
||||
```
|
||||
|
||||
If you don't know how to use tmux, see [a
|
||||
cheatsheet](https://danielmiessler.com/study/tmux/). The useful
|
||||
keybindings are:
|
||||
|
||||
* `control-b c`: open new tab
|
||||
* `control-b p/n`: previous/next tab
|
||||
* `control-b "`: split tab into top and bottom panes
|
||||
* `control-b %`: split tab into left and right panes
|
||||
* `control-b <arrows>`: move between panes
|
||||
* `control-b control-b <something>`: if you have two tmuxes nested,
|
||||
use `control-b` twice to do a command on the inner one instead of
|
||||
the outer one
|
||||
|
||||
## Configure local project
|
||||
|
||||
Using your regular text editor (the Riju repository is synchronized
|
||||
inside and outside of the container, so you can use whatever editor
|
||||
you would like, it doesn't have to be something in the terminal),
|
||||
create a file `.env` in the Riju repository with the following
|
||||
contents:
|
||||
|
||||
```
|
||||
DOCKER_REPO=raxod502/riju
|
||||
S3_BUCKET=riju
|
||||
```
|
||||
|
||||
This tells Riju to pull assets from the official registries that I
|
||||
maintain, so that you don't have to build them yourself.
|
||||
|
||||
## Set up Docker images
|
||||
|
||||
Download the two Docker images needed for testing a new language:
|
||||
|
||||
```
|
||||
$ make pull I=packaging
|
||||
$ make pull I=runtime
|
||||
```
|
||||
|
||||
Create a new tab in tmux (`control-b c`) and start the runtime image
|
||||
with ports exposed:
|
||||
|
||||
```
|
||||
$ make shell I=runtime E=1
|
||||
```
|
||||
|
||||
Inside that shell, start another instance of tmux:
|
||||
|
||||
```
|
||||
$ make tmux
|
||||
```
|
||||
|
||||
Now within that tmux, start Riju in development mode:
|
||||
|
||||
```
|
||||
$ make dev
|
||||
```
|
||||
|
||||
You should now be able to navigate to <http://localhost:6119> and see
|
||||
that Riju is running, although it does not have any languages
|
||||
installed.
|
||||
|
||||
Finally, switch back to the admin shell (`control-b p`). We are ready
|
||||
to start creating your new language.
|
||||
|
||||
## Create a language configuration
|
||||
|
||||
Create a file `langs/mylanguage.yaml` with the following contents:
|
||||
|
||||
```yaml
|
||||
id: "mylanguage"
|
||||
name: "My Language"
|
||||
|
||||
main: "TODO"
|
||||
template: |
|
||||
# Fill this in later
|
||||
|
||||
run: |
|
||||
echo "Hello, world!"
|
||||
```
|
||||
|
||||
Now from the admin shell, run `make repkgs L=mylanguage`. Once that
|
||||
completes, you should see your language at <http://localhost:6119>.
|
||||
Furthermore, you can switch to the runtime image (`control-b n`) and
|
||||
run `make sandbox L=mylanguage` to test your language at the command
|
||||
line (e.g. type `run` to print `Hello, world!`). Each time you modify
|
||||
the language configuration, run `make repkgs` to reinstall the
|
||||
language.
|
||||
|
||||
Follow these steps to augment your language configuration:
|
||||
|
||||
* [Install your language](tutorial/install.md)
|
||||
* [Provide run commands](tutorial/run.md)
|
||||
* [Configure tests](tutorial/tests.md)
|
||||
* [Provide metadata](tutorial/metadata.md)
|
||||
* [Add code formatter](tutorial/formatter.md)
|
||||
* [Add language server](tutorial/lsp.md)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,232 @@
|
|||
# Tutorial: install your language
|
||||
|
||||
Presumably, your language isn't installed by default in Ubuntu. If
|
||||
not, you'll need to add an `install:` block to your language
|
||||
configuration describing how to install it.
|
||||
|
||||
The easiest case is system packages, which are supported out of the
|
||||
box, for example:
|
||||
|
||||
```yaml
|
||||
install:
|
||||
apt:
|
||||
- ruby
|
||||
npm:
|
||||
- pug-cli
|
||||
pip:
|
||||
- bython
|
||||
gem:
|
||||
- solargraph
|
||||
cpan:
|
||||
- Acme::Chef
|
||||
opam:
|
||||
- ocamlformat
|
||||
```
|
||||
|
||||
For more advanced configuration you need to understand how Debian
|
||||
packaging works. To build a package, we place files into a `${pkg}`
|
||||
directory mirroring how they should be installed on the system, so for
|
||||
example a binary intended for `/usr/local/bin/prettier` would be
|
||||
placed at `${pkg}/usr/local/bin/prettier`. Then the `${pkg}` directory
|
||||
can be turned into a `.deb`, which can be installed on any system.
|
||||
|
||||
*If you have trouble, see the tutorial on [debugging package
|
||||
installation](install-debugging.md).*
|
||||
|
||||
## Download a binary or script
|
||||
|
||||
We prefer to put all binaries in `/usr/local/bin` rather than
|
||||
`/usr/bin` where they might conflict with default Ubuntu packages.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
manual: |
|
||||
install -d "${pkg}/usr/local/bin"
|
||||
|
||||
wget http://www.blue.sky.or.jp/grass/grass.rb
|
||||
chmod +x grass.rb
|
||||
cp -T grass.rb "${pkg}/usr/local/bin/grass"
|
||||
```
|
||||
|
||||
## Get the latest version from GitHub
|
||||
|
||||
There is a predefined `latest_release` function that can be used to
|
||||
get the name of the latest tag in a GitHub repository.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
manual: |
|
||||
install -d "${pkg}/usr/local/bin"
|
||||
|
||||
ver="$(latest_release snoe/clojure-lsp)"
|
||||
wget "https://github.com/snoe/clojure-lsp/releases/download/${ver}/clojure-lsp"
|
||||
chmod +x clojure-lsp
|
||||
cp clojure-lsp "${pkg}/usr/local/bin/"
|
||||
```
|
||||
|
||||
## Get the latest version from some other website
|
||||
|
||||
In the case that there's no "download the latest version" URL, we
|
||||
typically use disgusting `grep` pipelines to extract it in a
|
||||
semi-reliable way from a stable-seeming webpage.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
manual: |
|
||||
install -d "${pkg}/usr/local/bin"
|
||||
|
||||
path="$(curl -fsSL http://static.red-lang.org/download.html | grep -Eo '/dl/linux/[^"]+' | head -n1)"
|
||||
wget "http://static.red-lang.org${path}" -O red
|
||||
chmod +x red
|
||||
cp red "${pkg}/usr/local/bin/"
|
||||
```
|
||||
|
||||
## Unpack a tarball
|
||||
|
||||
Sometimes tarballs can be extracted directly to `/usr/local`. However,
|
||||
unless there's an obviously correct way to do that, we typically put
|
||||
language distributions in `/opt/<lang>` and then put symlinks in
|
||||
`/usr/local/bin` for the binaries. The `-C` and `--strip-components`
|
||||
flags are very helpful for controlling the extraction.
|
||||
|
||||
Also, one thing that might be unintuitive is that we use system-global
|
||||
paths for the targets of symlinks (`/opt/swift/bin/swiftc`) but
|
||||
`${pkg}` paths for the link names (`${pkg}/usr/local/bin/swiftc`).
|
||||
This is because while we are putting all files into `${pkg}` during
|
||||
build, the eventual place they will be installed by the package is
|
||||
into the root filesystem, so any references to paths *within* files
|
||||
(including symlink targets) must not mention `${pkg}`.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
manual: |
|
||||
install -d "${pkg}/opt/swift"
|
||||
install -d "${pkg}/usr/local/bin"
|
||||
|
||||
ver="$(latest_release apple/swift | grep -Eo '[0-9.]+')"
|
||||
wget "https://swift.org/builds/swift-${ver}-release/ubuntu2004/swift-${ver}-RELEASE/swift-${ver}-RELEASE-ubuntu20.04.tar.gz" -O swift.tar.gz
|
||||
tar -xf swift.tar.gz -C "${pkg}/opt/swift" --strip-components=2
|
||||
ln -s /opt/swift/bin/swiftc /opt/swift/bin/sourcekit-lsp "${pkg}/usr/local/bin/"
|
||||
```
|
||||
|
||||
## Compile from source
|
||||
|
||||
Sometimes there is no binary distribution. In this case you want to
|
||||
clone/download the source code and run the appropriate compilation
|
||||
command. GCC and LLVM plus a number of scripting languages are already
|
||||
installed into the `packaging` image by default, but you might have to
|
||||
include a `prepare` block listing additional things to install to be
|
||||
able to compile the software.
|
||||
|
||||
For example, in this case the `build.sh` script provided by Zot
|
||||
happens to require some Qt development dependencies at compile time,
|
||||
and a non-development dependency at runtime.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
prepare:
|
||||
apt:
|
||||
- qt5-qmake
|
||||
- qtscript5-dev
|
||||
apt:
|
||||
- libqt5script5
|
||||
manual: |
|
||||
install -d "${pkg}/usr/local/bin"
|
||||
|
||||
git clone https://github.com/manyoso/zot.git
|
||||
pushd zot
|
||||
./build.sh
|
||||
cp build/bin/zot "${pkg}/usr/local/bin/"
|
||||
popd
|
||||
```
|
||||
|
||||
## Use a shared dependency
|
||||
|
||||
Your language may need to use software such as Prettier which is also
|
||||
used by other languages but does not have an Ubuntu package. In that
|
||||
case we package it as a Riju shared dependency (check the `shared`
|
||||
directory for a list), and you declare the dependency using the `riju`
|
||||
key.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
riju:
|
||||
- prettier
|
||||
```
|
||||
|
||||
## Install custom scripts or config files
|
||||
|
||||
Sometimes you may need to add a wrapper script or config file
|
||||
somewhere on the filesystem. Riju has the `scripts` and `files` keys
|
||||
for this. The keys of these maps are the filesystem paths (`scripts`
|
||||
defaults to putting things in `/usr/local/bin` if relative paths are
|
||||
given).
|
||||
|
||||
```yaml
|
||||
install:
|
||||
scripts:
|
||||
teco-encode: |
|
||||
#!/usr/bin/env -S python3 -u
|
||||
|
||||
import re
|
||||
import sys
|
||||
|
||||
for line in sys.stdin:
|
||||
line = re.sub(r"\^(.)", lambda m: chr(ord(m.group(1)) ^ 0b1000000), line)
|
||||
line = line.replace("$", chr(27))
|
||||
print(line, end="")
|
||||
files:
|
||||
"/opt/sqlite/sqls.yml": |
|
||||
connections:
|
||||
- driver: sqlite3
|
||||
dataSourceName: db.sqlite3
|
||||
```
|
||||
|
||||
## Setting up skeleton files
|
||||
|
||||
Sometimes your language may require some configuration files in
|
||||
addition to the actual file to be compiled. Or alternatively, your
|
||||
language may do a bunch of caching work the first time it starts up,
|
||||
which you don't want to happen every time someone launches it on Riju.
|
||||
Both of these problems can be solved by preparing files to be copied
|
||||
into the user's home directory at the start of a new Riju session
|
||||
(this is done using the `setup` key, which is explained later).
|
||||
|
||||
In the following example, we ask .NET to create a project template for
|
||||
us, and then compile the project. Then we take the files that were
|
||||
created for the project, plus cache directories in `$HOME`, and copy
|
||||
them into `/opt` so that they can be copied back later using `setup`.
|
||||
By convention, we name such directories `skel`.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
prepare:
|
||||
apt:
|
||||
- $(grep-aptavail -wF Package "dotnet-sdk-3\.[0-9.]+" -s Package -n | sort -Vr | head -n1)
|
||||
manual: |
|
||||
install -d "${pkg}/opt/qsharp/skel"
|
||||
|
||||
dotnet new -i Microsoft.Quantum.ProjectTemplates
|
||||
dotnet new console -lang Q# -o main
|
||||
dotnet run --project main
|
||||
|
||||
shopt -s dotglob
|
||||
cp -R * "${HOME}/.dotnet" "${HOME}/.nuget" "${pkg}/opt/qsharp/skel/"
|
||||
rm "${pkg}/opt/qsharp/skel/main/Program.qs"
|
||||
chmod -R a=u,go-w "${pkg}/opt/qsharp/skel"
|
||||
```
|
||||
|
||||
## Dealing with versioned APT packages
|
||||
|
||||
For some reason, some APT packages have version numbers in their
|
||||
names. For example, you can't `sudo apt install lua`; you have to
|
||||
`sudo apt install lua5.4`. The best way to deal with these situations
|
||||
is to use various `grep-aptavail` hacks to identify the latest
|
||||
available version programmatically. Check the man page as well as uses
|
||||
of `grep-aptavail` in Riju to understand the options.
|
||||
|
||||
```yaml
|
||||
install:
|
||||
apt:
|
||||
- $(grep-aptavail -XF Provides lua -a -XF Version "$(grep-aptavail -XF Provides lua -s Version -n | sort -Vr | head -n1)" -s Package -n | head -n1)
|
||||
```
|
||||
|
|
@ -0,0 +1 @@
|
|||
# Tutorial: provide run commands
|
||||
Loading…
Reference in New Issue