riju/doc/tutorial/install.md

Tutorial: install your language

Most likely, your language isn't installed by default in Ubuntu. If not, you'll need to add an install: block to your language configuration describing how to install it.

The easiest case is system packages, which are supported out of the box, for example:

install:
  apt:
    - ruby
  npm:
    - pug-cli
  pip:
    - bython
  gem:
    - solargraph
  cpan:
    - Acme::Chef
  opam:
    - ocamlformat

For more advanced configuration you need to understand how Debian packaging works. To build a package, we place files into a ${pkg} directory mirroring how they should be installed on the system, so for example a binary intended for /usr/local/bin/prettier would be placed at ${pkg}/usr/local/bin/prettier. Then the ${pkg} directory can be turned into a .deb, which can be installed on any system.

If you have trouble, see the tutorial on debugging package installation.

Download a binary or script

We prefer to put all binaries in /usr/local/bin rather than /usr/bin where they might conflict with default Ubuntu packages.

install:
  manual: |
    install -d "${pkg}/usr/local/bin"

    wget http://www.blue.sky.or.jp/grass/grass.rb
    chmod +x grass.rb
    cp -T grass.rb "${pkg}/usr/local/bin/grass"    

Get the latest version from GitHub

There is a predefined latest_release function that can be used to get the name of the latest tag in a GitHub repository.

install:
  manual: |
    install -d "${pkg}/usr/local/bin"

    ver="$(latest_release snoe/clojure-lsp)"
    wget "https://github.com/snoe/clojure-lsp/releases/download/${ver}/clojure-lsp"
    chmod +x clojure-lsp
    cp clojure-lsp "${pkg}/usr/local/bin/"    

Get the latest version from some other website

In the case that there's no "download the latest version" URL, we typically use disgusting grep pipelines to extract it in a semi-reliable way from a stable-seeming webpage.

install:
  manual: |
    install -d "${pkg}/usr/local/bin"

    path="$(curl -fsSL https://static.red-lang.org/download.html | grep -Eo '/dl/linux/[^"]+' | head -n1)"
    wget "https://static.red-lang.org${path}" -O red
    chmod +x red
    cp red "${pkg}/usr/local/bin/"    

Unpack a tarball

Sometimes tarballs can be extracted directly to /usr/local. However, unless there's an obviously correct way to do that, we typically put language distributions in /opt/<lang> and then put symlinks in /usr/local/bin for the binaries. The -C and --strip-components flags are very helpful for controlling the extraction.

Also, one thing that might be unintuitive is that we use system-global paths for the targets of symlinks (/opt/swift/bin/swiftc) but ${pkg} paths for the link names (${pkg}/usr/local/bin/swiftc). This is because while we are putting all files into ${pkg} during build, the eventual place they will be installed by the package is into the root filesystem, so any references to paths within files (including symlink targets) must not mention ${pkg}. This is a standard feature of all Linux packaging tools.

install:
  manual: |
    install -d "${pkg}/opt/swift"
    install -d "${pkg}/usr/local/bin"

    ver="$(latest_release apple/swift | grep -Eo '[0-9.]+')"
    wget "https://swift.org/builds/swift-${ver}-release/ubuntu2004/swift-${ver}-RELEASE/swift-${ver}-RELEASE-ubuntu20.04.tar.gz" -O swift.tar.gz
    tar -xf swift.tar.gz -C "${pkg}/opt/swift" --strip-components=2
    ln -s /opt/swift/bin/swiftc /opt/swift/bin/sourcekit-lsp "${pkg}/usr/local/bin/"    

Compile from source

Sometimes there is no binary distribution. In this case you want to clone/download the source code and run the appropriate compilation command. GCC and LLVM plus a number of scripting languages are already installed into the packaging image by default, but you might have to include a prepare block listing additional things to install to be able to compile the software.

For example, in this case the build.sh script provided by Zot happens to require some Qt development dependencies at compile time, and a non-development dependency at runtime.

install:
  prepare:
    apt:
      - qt5-qmake
      - qtscript5-dev
  apt:
    - libqt5script5
  manual: |
    install -d "${pkg}/usr/local/bin"

    git clone https://github.com/manyoso/zot.git
    pushd zot
    ./build.sh
    cp build/bin/zot "${pkg}/usr/local/bin/"
    popd    

Use a shared dependency

Your language may need to use software such as Prettier which is also used by other languages but does not have an Ubuntu package. In that case we package it as a Riju shared dependency (check the shared directory for a list), and you declare the dependency using the riju key.

install:
  riju:
    - prettier

Install custom scripts or config files

Sometimes you may need to add a wrapper script or config file somewhere on the filesystem. Riju has the scripts and files keys for this. The keys of these maps are the filesystem paths (scripts defaults to putting things in /usr/local/bin if relative paths are given).

install:
  scripts:
    teco-encode: |
      #!/usr/bin/env -S python3 -u

      import re
      import sys

      for line in sys.stdin:
          line = re.sub(r"\^(.)", lambda m: chr(ord(m.group(1)) ^ 0b1000000), line)
          line = line.replace("$", chr(27))
          print(line, end="")      
  files:
    "/opt/sqlite/sqls.yml": |
      connections:
        - driver: sqlite3
          dataSourceName: db.sqlite3      

Dealing with versioned APT packages

For some reason, some APT packages have version numbers in their names. For example, you can't sudo apt install lua; you have to sudo apt install lua5.4. The best way to deal with these situations is to use various grep-aptavail hacks to identify the latest available version programmatically. Check the man page as well as uses of grep-aptavail in Riju to understand the options.

install:
  apt:
    - $(grep-aptavail -XF Provides lua -a -XF Version "$(grep-aptavail -XF Provides lua -s Version -n | sort -Vr | head -n1)" -s Package -n | head -n1)

Custom APT repos

Some languages are distributed in third-party APT repositories. You can identify this by looking for installation directions that say to run add-apt-repository or add a file to /etc/apt/sources.list.d. Frequently you're also asked to import a custom signing key using apt-key.

Riju provides shorthands for these operations. The aptKey option can be either a URL or a hexadecimal string (both forms appear in commands you're asked to run). The aptRepo option is a line that can be added to /etc/apt/sources.list.d, which can be copied out of the suggested installation command.

Assuming these keys are specified, then all the packages in the custom APT repo(s) will be available for selection under the apt option, in addition to standard Ubuntu packages.

install:
  aptKey:
    - "https://keybase.io/crystal/pgp_keys.asc"
  aptRepo:
    - "deb [arch=amd64] https://dist.crystal-lang.org/apt crystal main"
  apt:
    - crystal

Getting the name of the current Ubuntu release

Sometimes you need to download from a URL or add an APT repository whose name depends on the version (e.g. 21.04) or name (e.g. hirsute) of the current Ubuntu release. These parameters are automatically available in the ${ubuntu_ver} and ${ubuntu_name} variables, which can be used in most option values:

install:
  aptKey:
    - "B4112585D386EB94"
  aptRepo:
    - "deb [arch=amd64] https://dl.hhvm.com/ubuntu ${ubuntu_name} main"
  apt:
    - hhvm

Dealing with broken certificate chains

Consider the following errors from curl and wget respectively:

curl: (60) SSL certificate problem: unable to get local issuer certificate
ERROR: cannot verify ioke.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
  Unable to locally verify the issuer's authority.

These are often because of a server misconfiguration. Servers are supposed to send all intermediate certificates in their certificate chain, but sometimes they fail to do this. In that case, you need to manually download the missing intermediate certificate from its issuer, and install it.

You can get the information about which certificate is required from SSL Labs. Then, by Googling the name of the missing certificate that SSL Labs has found, you can typically find it as a download from the certificate issuer.

Riju has a cert option which takes URLs of certificates to download and install (note the use of prepare here because we need the certificate when building the package, not just when installing it):

install:
  prepare:
    cert:
      - "https://letsencrypt.org/certs/lets-encrypt-r3.pem"
  apt:
    - default-jdk
  manual: |
    install -d "${pkg}/opt/ioke"
    install -d "${pkg}/usr/local/bin"

    wget https://ioke.org/dist/ioke-ikj-latest.tar.gz -O ioke.tar.gz
    tar -xf ioke.tar.gz -C "${pkg}/opt/ioke" --strip-components=1
    ln -s /opt/ioke/bin/ioke "${pkg}/usr/local/bin/ioke"    

Setting up skeleton files

Sometimes your language may require some configuration files in addition to the actual file to be compiled. Or alternatively, your language may do a bunch of caching work the first time it starts up, which you don't want to happen every time someone launches it on Riju. Both of these problems can be solved by preparing files to be copied into the user's home directory at the start of a new Riju session (this is done using the setup key, which is explained later).

In the following example, we ask .NET to create a project template for us, and then compile the project. Then we take the files that were created for the project, plus cache directories in $HOME, and copy them into /opt so that they can be copied back later using setup. By convention, we name such directories skel.

install:
  prepare: &install-dotnet
    preface: |
      wget "https://packages.microsoft.com/config/ubuntu/${ubuntu_ver}/packages-microsoft-prod.deb"
      sudo --preserve-env=DEBIAN_FRONTEND apt-get install ./packages-microsoft-prod.deb
      sudo --preserve-env=DEBIAN_FRONTEND apt-get update      
    apt:
      - $(grep-aptavail -wF Package "dotnet-sdk-3\.[0-9.]+" -s Package -n | sort -Vr | head -n1)
  <<: *install-dotnet
  manual: |
    install -d "${pkg}/opt/qsharp/skel-home"
    install -d "${pkg}/opt/qsharp/skel-src"

    dotnet new -i Microsoft.Quantum.ProjectTemplates
    dotnet new console -lang Q# -o main
    dotnet run --project main

    shopt -s dotglob
    cp -R main "${pkg}/opt/qsharp/skel-src/"
    cp -R "${HOME}/.dotnet" "${HOME}/.nuget" "${pkg}/opt/qsharp/skel-home/"
    rm "${pkg}/opt/qsharp/skel-src/main/Program.qs"
    chmod -R a=u,go-w "${pkg}/opt/qsharp"    

setup: |
  shopt -s dotglob
  cp -R /opt/qsharp/skel-src/* ./
  cp -R /opt/qsharp/skel-home/* "${HOME}/"  

What about...?

Refer to jsonschema.yaml for the complete reference about what keys are accepted in install. See generate-build-script.js for how these are interpreted programmatically, and see the generated build.bash and install.bash scripts (in build/lang/<name>/) for examples of the outputs. And of course, check some of the other languages, since probably what you want to do has come up before at some point.