Commit Graph

512 Commits

Author SHA1 Message Date
Radon Rosborough 5752918068 Make users.js robust to concurrency 2021-01-07 21:09:33 -08:00
Radon Rosborough 9a99429f48 Miscellaneous work starting to support langs 2021-01-06 21:44:27 -08:00
Radon Rosborough 6450df047f Configure more package managers 2021-01-02 15:41:47 -08:00
Radon Rosborough d782cc1fb5 Import APT package installation 2021-01-02 11:08:14 -08:00
Radon Rosborough ec94a44030 Re-add install clause for Python 2021-01-02 10:18:15 -08:00
Radon Rosborough b849045d12 Fix Markdown test
It previously relied on multiple trailing newlines and that is not
allowed by our new YAML format.
2021-01-02 10:14:56 -08:00
Radon Rosborough a6adfea206 Rename Node.js => JavaScript 2021-01-02 10:14:56 -08:00
Radon Rosborough 8cd93a744f Remove some spurious trailing newlines 2021-01-02 10:14:56 -08:00
Radon Rosborough 2daa11c7aa Import all language configs into YAML
This was an absolutely disgusting operation. Here's the commands I
used to do it (see <https://docs.gomplate.ca/> for the tool):

$ node
> require('fs').writeFile('langs.json', JSON.stringify(require('./langs').langs), console.log)
$ ./gomplate -c .=langs.json -f template.yaml > langs.yaml
$ cat langs.json | jq 'keys[]' -r L | while read lang; do cat langs.yaml | sed -n "/^ #START $lang\$/,/^ #END $lang\$/p" | tail -n+2 | head -n-1 | sed 's/^  //' > langs/$lang.yaml; done

I had some difficulty getting valid YAML to be generated, so I found
the tool at <http://www.yamllint.com/> to be helpful in identifying
errors.

For validation that nothing important had changed in the
transformation, I used <http://www.jsondiff.com/> to compare
langs.json against the output of this command:

$ ./gomplate -c .=langs.json -f template.yaml | node yaml2json.js | jq '(.. | select(type == "string")) |= sub("\n+$";"") | (.. | select(.template?) | .template?) |= (.+"\n") | (.. | select(.format?.input?) | .format?.input?) |= (.+"\n")'

Here's yaml2json.js:

import fs from "fs";
import YAML from "yaml";

console.log(JSON.stringify(YAML.parse(fs.readFileSync(0, "utf-8")), null, 2));

And here's the template.yaml, which is approximately the worst thing
ever:

{{ range . }}
"{{ .id }}":
 #START {{ .id }}
  id: "{{ .id }}"
{{ if has . "aliases" }}  aliases:
{{ range .aliases }}    - "{{ . }}"
{{ end }}{{ end }}  name: "{{ .name }}"
{{ if has . "monacoLang" }}  monacoLang: {{ .monacoLang }}
{{ end }}{{ if has . "info" }}
  info:
{{ if has .info "impl" }}    impl: "{{ .info.impl }}"
{{ end }}{{ if has .info "version" }}    version: "{{ .info.version }}"
{{ end }}    year: {{ .info.year }}
    desc: "{{ .info.desc }}"
{{ if eq "string" (printf "%T" .info.ext) }}    ext: {{ .info.ext }}
{{ else if eq 0 (len .info.ext) }}    ext: []
{{ else }}    ext:
{{ range .info.ext }}      - {{ . }}
{{ end }}{{ end }}    web:
{{ if has .info.web "wiki" }}      wiki: "{{ .info.web.wiki }}"
{{ end }}{{ if has .info.web "esolang" }}      esolang: "{{ .info.web.esolang }}"
{{ end }}{{ if has .info.web "home" }}      home: "{{ .info.web.home }}"
{{ end }}{{ if has .info.web "impl" }}      impl: "{{ .info.web.impl }}"
{{ end }}      source: {{ if eq "string" (printf "%T" .info.web.source) }}"{{ .info.web.source }}"
{{ else }}null
{{ end }}{{ if eq "string" (printf "%T" .info.category) }}    category: {{ .info.category }}
{{ else if eq 0 (len .info.category) }}    category: []
{{ else }}    category:
{{ range .info.category }}      - {{ . }}
{{ end }}{{ end }}{{ if eq "string" (printf "%T" .info.mode) }}    mode: {{ .info.mode }}
{{ else if eq 0 (len .info.mode) }}    mode: []
{{ else }}    mode:
{{ range .info.mode }}      - {{ . }}
{{ end }}{{ end }}{{ if eq "string" (printf "%T" .info.platform) }}    platform: {{ .info.platform }}
{{ else if eq 0 (len .info.platform) }}    platform: []
{{ else }}    platform:
{{ range .info.platform }}      - {{ . }}
{{ end }}{{ end }}{{ if eq "string" (printf "%T" .info.syntax) }}    syntax: {{ .info.syntax }}
{{ else if eq 0 (len .info.syntax) }}    syntax: []
{{ else }}    syntax:
{{ range .info.syntax }}      - {{ . }}
{{ end }}{{ end }}{{ if eq "string" (printf "%T" .info.typing) }}    typing: {{ .info.typing }}
{{ else if eq 0 (len .info.typing) }}    typing: []
{{ else }}    typing:
{{ range .info.typing }}      - {{ . }}
{{ end }}{{ end }}{{ if eq "string" (printf "%T" .info.paradigm) }}    paradigm: {{ .info.paradigm }}
{{ else if eq 0 (len .info.paradigm) }}    paradigm: []
{{ else }}    paradigm:
{{ range .info.paradigm }}      - {{ . }}
{{ end }}{{ end }}{{ if eq "string" (printf "%T" .info.usage) }}    usage: {{ .info.usage }}
{{ else if eq 0 (len .info.usage) }}    usage: []
{{ else }}    usage:
{{ range .info.usage }}      - {{ . }}
{{ end }}{{ end }}{{ end }}{{ if or (has . "daemon") (has . "setup") }}
{{ end }}{{ if has . "daemon" }}  daemon: |
{{ .daemon | indent 4 }}
{{ end }}{{ if has . "setup" }}  setup: |
{{ .setup | indent 4 }}
{{ end }}{{ if has . "repl" }}
  repl: |
{{ .repl | indent 4 }}
{{ end }}{{ if has . "input" }}{{ if .input | strings.Contains "\x1b" }}  input: "{{ .input | strings.ReplaceAll "\x1b" "\\x1b" }}"
{{ else }}  input: |{{ if .input | regexp.Match "^\\s" }}2{{ end }}
{{ .input | indent 4 }}
{{ end }}{{ end }}{{ if has . "output" }}  output: |
{{ .output | indent 4 }}
{{ end }}
  main: "{{ .main }}"
  template: {{ if eq .id "whitespace" }}{{ .template | strings.Quote }}
{{ else }}|{{ if .template | regexp.Match "^\\s" }}2{{ end }}
{{ .template | indent 4 }}{{ end }}{{ if has . "prefix" }}  prefix: |
{{ .prefix | indent 4 }}
{{ end }}{{ if has . "suffix" }}  suffix: |
{{ .suffix | indent 4 }}
{{ end }}{{ if has . "createEmpty" }}  createEmpty: "{{ .createEmpty }}"
{{ end }}
{{ if has . "compile" }}  compile: |
{{ .compile | indent 4 }}
{{ end }}  run: |
{{ .run | indent 4 }}
{{ if has . "helloInput" }}  helloInput: |
{{ .helloInput | indent 4 }}
{{ end }}{{ if has . "hello" }}  hello: |
{{ .hello | indent 4 }}
{{ end }}{{ if has . "helloMaxLength" }}  helloMaxLength: {{ .helloMaxLength }}
{{ end }}{{ if has . "runReplInput" }}  runReplInput: |
{{ .runReplInput | indent 4 }}
{{ end }}{{ if has . "runReplOutput" }}  runReplOutput: |
{{ .runReplOutput | indent 4 }}
{{ end }}{{ if has . "scope" }}
  scope:
    code: |
{{ .scope.code | indent 6 }}
{{ if has .scope "after" }}    after: "{{ .scope.after }}"
{{ end }}{{ if has .scope "input" }}{{ if .scope.input | strings.Contains "\x1b" }}    input: "{{ .scope.input | strings.ReplaceAll "\x1b" "\\x1b" }}"
{{ else }}    input: |
{{ .scope.input | indent 6 }}
{{ end }}{{ end }}{{ if has .scope "output" }}    output: |
{{ .scope.output | indent 6 }}
{{ end }}{{ end }}{{ if has . "format" }}
  format:
    run: |
{{ .format.run | indent 6 }}
{{ if has .format "input" }}    input: |
{{ .format.input | indent 6 }}{{ end }}{{ if has .format "output" }}    output: |
{{ .format.output | indent 6 }}{{ end }}{{ end }}{{ if has . "pkg" }}
  pkg:
    install: |
{{ .pkg.install | indent 6 }}
{{ if has .pkg "uninstall" }}
    uninstall: |
{{ .pkg.uninstall | indent 6 }}
{{ end }}{{ if has .pkg "all" }}
    all: |
{{ .pkg.all | indent 6 }}
{{ end }}{{ if has .pkg "search" }}
    search: |
{{ .pkg.search | indent 6 }}
{{ end }}{{ end }}{{ if has . "lsp" }}
  lsp:
{{ if has .lsp "setup" }}    setup: |
{{ .lsp.setup | indent 6 }}
{{ end }}    start: |
{{ .lsp.start | indent 6 }}
{{ if has .lsp "disableDynamicRegistration" }}    disableDynamicRegistration: {{ .lsp.disableDynamicRegistration }}
{{ end }}{{ if has .lsp "init" }}    init:
{{ .lsp.init | data.ToYAML | indent 6 }}{{ end }}{{ if has .lsp "config" }}    config: {{ .lsp.config | data.ToYAML }}{{ end }}{{ if has .lsp "lang" }}    lang: "{{ .lsp.lang }}"
{{ end }}{{ if has .lsp "code" }}    code: {{ .lsp.code | strings.Quote }}
{{ end }}{{ if has .lsp "after" }}    after: {{ .lsp.after | strings.Quote }}
{{ end }}{{ if has .lsp "item" }}    item: {{ .lsp.item | strings.Quote }}
{{ end }}{{ end }}{{ if has . "skip" }}
  skip:
{{ range .skip }}    - {{ . }}
{{ end }}{{ end }} #END {{ .id }}
{{ end }}
2021-01-02 10:14:26 -08:00
Radon Rosborough 4f3633384c Factor out separate deploy script 2021-01-01 12:32:29 -08:00
Radon Rosborough a56dea0f76 Use hostnamectl 2021-01-01 12:19:11 -08:00
Radon Rosborough 86c29aef4c Add 'make sync' for my convenience 2021-01-01 12:13:59 -08:00
Radon Rosborough 87d7e6df8a Further optimization 2021-01-01 12:06:20 -08:00
Radon Rosborough 162e10d259 Configure TLS to work properly 2021-01-01 12:01:51 -08:00
Radon Rosborough e0e75d7dae Run tests in CI 2021-01-01 11:43:10 -08:00
Radon Rosborough a7e94b792e Speed up builds even more 2021-01-01 01:53:28 -08:00
Radon Rosborough 041d9ea40f Be less verbose 2021-01-01 01:18:14 -08:00
Radon Rosborough 2a656e4e50 Fix an oopsie, plus add 'make repkg' 2021-01-01 01:13:31 -08:00
Radon Rosborough 0f6d1d9923 Build packages in correct container 2021-01-01 01:02:56 -08:00
Radon Rosborough 8455d259c0 Attempt to patch more bugs 2021-01-01 00:55:33 -08:00
Radon Rosborough d36970b9de First cut at pruning unneeded downloads 2021-01-01 00:23:14 -08:00
Radon Rosborough e43b16bd12 Use smaller admin image for CI 2020-12-31 23:29:56 -08:00
Radon Rosborough be8133ee71 Don't try pulling admin image 2020-12-31 23:10:53 -08:00
Radon Rosborough 9860f8f07c Fix subtle hash calculation bugs 2020-12-31 23:07:00 -08:00
Radon Rosborough b4c5cf2763 Fix control flow bug 2020-12-31 23:05:22 -08:00
Radon Rosborough 40b6bb4a53 Fix small but very important bug 2020-12-31 22:51:51 -08:00
Radon Rosborough 0e85e00282 Start getting the incremental deploy working 2020-12-31 22:39:35 -08:00
Radon Rosborough 87793e9778 Second attempt at Dockerfile hashing 2020-12-31 22:11:24 -08:00
Radon Rosborough 0b60ad12d1 First attempt at Dockerfile hashing 2020-12-31 07:15:29 -08:00
Radon Rosborough 6ac8623216 Fix $PATH explicitly in Docker shims 2020-12-30 15:05:56 -08:00
Radon Rosborough eaf9365074 Fix more script typos 2020-12-30 11:43:28 -08:00
Radon Rosborough d2df4916dc Fix script typo 2020-12-30 11:24:18 -08:00
Radon Rosborough 7f3f95d833 Make CI run faster by fixing bugs 2020-12-30 11:15:54 -08:00
Radon Rosborough 1725811aed Fix warning from sudo 2020-12-27 13:21:58 -08:00
Radon Rosborough 37755b17a9 Run tests in CI 2020-12-27 13:14:45 -08:00
Radon Rosborough accc4da201 All tests are passing for Python 2020-12-27 13:12:29 -08:00
Radon Rosborough 2dd3e31f1a I hate writing Webpack configs. I hate it so much. 2020-12-27 12:42:06 -08:00
Radon Rosborough 8b8266c752 Forgot to build the compilation image 2020-12-26 21:20:29 -08:00
Radon Rosborough b7162b924c Be less tentative 2020-12-26 20:45:52 -08:00
Radon Rosborough d9014b2342 More fixes for CI 2020-12-26 20:36:24 -08:00
Radon Rosborough 2c12fdd04a Various CI fixes, auto reload language configs 2020-12-26 20:26:03 -08:00
Radon Rosborough 0ecc49b613 Misc fixes 2020-12-26 20:07:14 -08:00
Radon Rosborough 337e385497 It's deprecated, but apparently still required 2020-12-26 20:03:07 -08:00
Radon Rosborough ef3696e86e Work around OpenSSH being dumb 2020-12-26 20:00:54 -08:00
Radon Rosborough 63a23a6c1b Fix typo 2020-12-26 19:51:11 -08:00
Radon Rosborough 77387027f2 Set up initial CI configuration 2020-12-26 19:49:01 -08:00
Radon Rosborough 57e28ea1e2 Remove erroneous check from validate.bash 2020-12-26 11:31:25 -08:00
Radon Rosborough 2b13f8d8bf Successfully deploy to production 2020-12-26 11:28:51 -08:00
Radon Rosborough f521eda40e Fixups for Packer AMI build 2020-12-25 21:28:08 -08:00
Radon Rosborough 2e6aafbcb3 Successfully run webserver from Docker image 2020-12-25 19:34:15 -08:00