better-argo-tunnels/README.md

# Reverse SSH Tunnel Server for Traefik

A lightweight Go system that lets remote Docker hosts expose HTTP services through a central Traefik reverse proxy via SSH tunnels. The server SSHs into the Traefik ingress host to dynamically manage route configuration.

## Architecture

```
Remote Host                        Tunnel Server              Traefik Host (ingress.nixc.us)
+--------------+                   +---------------------+    +---------------------+
| tunnel-client| ---SSH tunnel---> | tunnel-server       |    | Traefik             |
| TUNNEL_KEY   |                   | allocates port,     |    | watches dynamic/    |
| TUNNEL_DOMAIN|                   | SSHs into Traefik   |--->| routes HTTPS to     |
| TUNNEL_PORT  |                   | host to write config |    | tunnel-server ports |
+--------------+                   +---------------------+    +---------------------+
```

### Flow

1. Client connects to tunnel-server via SSH using a private key
2. Client sends domain metadata (`{"domain":"myapp.example.com"}`) over a custom channel
3. Server allocates a port from the pool and sets up a reverse port forward
4. Server SSHs into `ingress.nixc.us` and writes a Traefik dynamic config file
5. Traefik detects the new config and routes HTTPS traffic to `tunnel-server:<port>`
6. Traffic flows: Internet -> Traefik -> tunnel-server:port -> SSH tunnel -> client -> local service
7. When the client disconnects, the config file is removed and the port is freed

## Quick Start

### 1. Generate SSH Keys

```bash
mkdir -p keys

# Server host key (for the SSH server that clients connect to)
ssh-keygen -t ed25519 -f keys/host_key -N ""

# Client key (for tunnel clients)
ssh-keygen -t ed25519 -f keys/id_ed25519 -N ""

# Authorize the client (install this file as the tunnel user's authorized_keys on ingress.nixc.us)
cat keys/id_ed25519.pub > keys/authorized_keys

# Deploy key for SSHing into the Traefik host
# (use an existing key that has root access to ingress.nixc.us,
#  or generate one and add its .pub to the Traefik host's authorized_keys)
cp ~/.ssh/ingress_deploy_key keys/traefik_deploy_key
```

### 2. Enable Traefik File Provider

On `ingress.nixc.us`, ensure Traefik has the file provider enabled.
If Traefik is run via Docker stack with CLI args, add:

```
--providers.file.directory=/root/traefik/dynamic
--providers.file.watch=true
```

Or via SSH (non-interactive):

```bash
ssh root@ingress.nixc.us "mkdir -p /root/traefik/dynamic"
```

Then update your Traefik stack/service to mount the directory and add the CLI args.

### 3. Start the Tunnel Server

```bash
docker compose up -d
```

### 4. Run a Client

On any remote host where your service is running:

**Using a mounted key file:**
```bash
docker run -d \
  --name tunnel-client \
  -e TUNNEL_SERVER=ingress.nixc.us:2222 \
  -e TUNNEL_DOMAIN=myapp.example.com \
  -e TUNNEL_PORT=8080 \
  -e TUNNEL_KEY=/keys/id_ed25519 \
  -v /path/to/keys:/keys:ro \
  --network host \
  tunnel-client
```

**Using raw PEM key content in an envvar:**
```bash
docker run -d \
  --name tunnel-client \
  -e TUNNEL_SERVER=ingress.nixc.us:2222 \
  -e TUNNEL_DOMAIN=myapp.example.com \
  -e TUNNEL_PORT=8080 \
  -e "TUNNEL_KEY=$(cat /path/to/id_ed25519)" \
  --network host \
  tunnel-client
```

## Environment Variables

### Server

| Variable | Description | Default |
|---|---|---|
| `SSH_PORT` | SSH listen port for tunnel clients | `2222` |
| `PORT_RANGE_START` | First allocatable tunnel port | `10000` |
| `PORT_RANGE_END` | Last allocatable tunnel port | `10100` |
| `SSH_HOST_KEY` | Path to SSH host private key | `/keys/host_key` |
| `AUTHORIZED_KEYS` | Path to authorized_keys file (tunnel user’s keys from ingress.nixc.us) | `/keys/authorized_keys` |
| `TRAEFIK_SSH_HOST` | Traefik host to SSH into **(required)** | - |
| `TRAEFIK_SSH_USER` | SSH user on the Traefik host | `root` |
| `TRAEFIK_SSH_KEY` | SSH key for Traefik host (path or PEM) **(required)** | - |
| `TRAEFIK_CONFIG_DIR` | Remote path for dynamic configs | `/root/traefik/dynamic` |
| `TRAEFIK_ENTRYPOINT` | Traefik entrypoint name | `websecure` |
| `TRAEFIK_CERT_RESOLVER` | Traefik TLS cert resolver | `letsencryptresolver` |

### Client

| Variable | Description | Default |
|---|---|---|
| `TUNNEL_SERVER` | Server host:port **(required)** | - |
| `TUNNEL_DOMAIN` | Public domain to expose **(required)** | - |
| `TUNNEL_PORT` | Local port of your service | `8080` |
| `TUNNEL_KEY` | SSH private key — file path or raw PEM **(required)** | `/keys/id_ed25519` |

## Generated Traefik Config

When a client connects requesting `myapp.example.com`, the server writes this to the Traefik host:

```yaml
# /root/traefik/dynamic/tunnel-myapp-example-com.yml
http:
  routers:
    tunnel-myapp-example-com-router:
      rule: "Host(`myapp.example.com`)"
      entryPoints:
        - websecure
      tls:
        certResolver: letsencryptresolver
      service: tunnel-myapp-example-com-service
  services:
    tunnel-myapp-example-com-service:
      loadBalancer:
        servers:
          - url: "http://tunnel-server:10042"
```

## Building

```bash
# Build both binaries locally (commit and push these for remote deployment)
go build -o tunnel-server ./cmd/server/
go build -o tunnel-client ./cmd/client/

# Build Docker images
docker compose build                                # server image
docker build --target client -t tunnel-client .     # client image
```

## Binaries and systemd (bare metal)

The repo ships compiled `tunnel-server` and `tunnel-client` for remote hosts that run without Docker. Use the included systemd units under `systemd/`.

**Keys:** Do not reuse the host’s SSH keys or share one key between hosts or tunnels. Generate a dedicated ed25519 key per tunnel (or per host). Add that key’s **public** half to the **tunnel** user’s `authorized_keys` on **ingress.nixc.us** (the reverse tunnel server).

**Multiple tunnels:** Run one systemd instance per tunnel (different env file and optional unit name), e.g. `tunnel-client@app1.service` and `tunnel-client@app2.service` each with their own env and key.

### Install from git.nixc.us (HTTPS raw)

From a host with curl, install the binary and systemd unit directly from the repo (replace `main` with your branch if needed):

```bash
REPO=https://git.nixc.us/colin/better-argo-tunnels/raw/branch/main
sudo curl -o /usr/local/bin/tunnel-client -L "$REPO/client"
sudo chmod +x /usr/local/bin/tunnel-client
sudo curl -o /etc/systemd/system/tunnel-client.service -L "$REPO/systemd/tunnel-client.service"
```

Then add a dedicated key for this tunnel, env file, and authorize on the server:

```bash
sudo mkdir -p /etc/tunnel-client
sudo ssh-keygen -t ed25519 -f /etc/tunnel-client/id_ed25519 -N ""
# Add the public key to the tunnel user's authorized_keys on ingress.nixc.us:
sudo cat /etc/tunnel-client/id_ed25519.pub
# On ingress.nixc.us as tunnel user: append that line to ~tunnel/.ssh/authorized_keys
sudo cp systemd/tunnel-client.env.example /etc/tunnel-client.env   # or curl from $REPO/systemd/tunnel-client.env.example
sudo edit /etc/tunnel-client.env   # set TUNNEL_SERVER, TUNNEL_DOMAIN, TUNNEL_KEY=/etc/tunnel-client/id_ed25519
sudo systemctl daemon-reload && sudo systemctl enable --now tunnel-client
```

### Client on a remote host (clone or copy)

1. Install the binary (from repo clone or raw URL above; repo has `client`/`server`):

   ```bash
   sudo cp client /usr/local/bin/tunnel-client && sudo chmod +x /usr/local/bin/tunnel-client
   ```

2. Copy the systemd unit and create env file:

   ```bash
   sudo cp systemd/tunnel-client.service /etc/systemd/system/
   sudo cp systemd/tunnel-client.env.example /etc/tunnel-client.env
   sudo edit /etc/tunnel-client.env   # set TUNNEL_SERVER, TUNNEL_DOMAIN, TUNNEL_KEY
   ```

3. Use a **dedicated** ed25519 key for this tunnel (not the host’s keys). Put the private key on the host (e.g. `/etc/tunnel-client/id_ed25519`) and set `TUNNEL_KEY` in env. Add the matching **public** key to the **tunnel** user’s `~/.ssh/authorized_keys` on **ingress.nixc.us**.

4. Enable and start:

   ```bash
   sudo systemctl daemon-reload
   sudo systemctl enable --now tunnel-client
   sudo journalctl -u tunnel-client -f
   ```

For a second tunnel on the same host, use a separate env and key (e.g. `/etc/tunnel-client-app2.env`, `/etc/tunnel-client/app2_id_ed25519`) and a second unit (e.g. copy to `tunnel-client-app2.service` with `EnvironmentFile=/etc/tunnel-client-app2.env`).

### Server (optional, bare metal)

If you run the tunnel server without Docker:

1. Install binary and keys under e.g. `/etc/tunnel-server/` (host_key, authorized_keys from the tunnel user on ingress.nixc.us, traefik_deploy_key).
2. Copy `systemd/tunnel-server.service` to `/etc/systemd/system/` and `systemd/tunnel-server.env.example` to `/etc/tunnel-server.env`. Set `TRAEFIK_SSH_HOST`, `TRAEFIK_SSH_KEY`, and paths to keys.
3. `systemctl enable --now tunnel-server`.

## Security Notes

- Only clients whose public keys are in the tunnel user’s `authorized_keys` on ingress.nixc.us can connect
- The server uses a stable host key for client verification
- SSH tunnels encrypt all traffic between client and server
- The server authenticates to the Traefik host with a separate deploy key
- Traefik handles TLS termination with automatic Let's Encrypt certificates