semios-build-server/INSTALL.md

# SemiOS Build Server

A web-based package build server for SemiOS. Uses SvelteKit SSR for the UI and a Rust agent for executing builds on remote machines.

## Requirements

### Build Server

- **Node.js** >= 18
- **npm**
- **Git**
- A running instance of the [packie](https://gitea.semilabs.org/semios/packie.git) package manager on the build machines

### Build Agent (per machine)

- **Rust** >= 1.85 (edition 2024)
- **Git**
- The [ports](https://gitea.semilabs.org/semios/ports.git) repository cloned on the machine
- `packie` installed and available in `$PATH`

## Quick Start

### 1. Clone and Build the Server

```bash
git clone https://gitea.semilabs.org/semios/semios-build-server.git
cd semios-build-server
npm install
npm run build
```

### 2. Configure the Server

Copy and edit `config.toml` at the project root:

```toml
[server]
host = "0.0.0.0"
port = 3000

[ports]
repo_url = "https://gitea.semilabs.org/semios/ports.git"
repo_path = "/path/to/ports-cache"
sync_interval_secs = 300

[repo]
publish_dir = "/path/to/repo-output"

[agent]
token = "your-shared-secret-token"
listen_port = 3001

[users]
admin = "scrypt$<salt>$<hash>"
```

| Key | Description |
|-----|-------------|
| `server.host` | Bind address (`0.0.0.0` for all interfaces) |
| `server.port` | Web UI port |
| `ports.repo_url` | Git URL of the ports repository |
| `ports.repo_path` | Local path where the server clones/caches the ports repo |
| `ports.sync_interval_secs` | How often to pull from upstream (seconds) |
| `repo.publish_dir` | Where published packages are collected |
| `agent.token` | Shared secret for agent authentication |
| `agent.listen_port` | Port the server expects agents to expose (for display only) |
| `users.*` | User accounts (username = scrypt password hash) |

### 3. Add Users

Only logged-in users can start builds, add packages to the repository, or delete artifacts.

Use the **registration helper** at `/register` to generate password hashes. For each user:

1. Open `http://localhost:3000/register` in a browser
2. Enter username and password, click **Generate Config Snippet**
3. Copy the `[users]` line into `config.toml`
4. Restart the server

Or generate hashes from the command line:

```bash
node -e "
const crypto = require('crypto');
const salt = crypto.randomBytes(16).toString('hex');
const hash = crypto.scryptSync('your-password', salt, 64).toString('hex');
console.log('scrypt\$' + salt + '\$' + hash);
"
```

### 3. Start the Server

```bash
BODY_SIZE_LIMIT=52428800 node build/index.js
```

Or use the npm script:

```bash
npm run start
```

The server listens on `http://0.0.0.0:3000` by default.

#### Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `BODY_SIZE_LIMIT` | - | Max request body size in bytes (set to `52428800` for artifact uploads) |
| `PORT` | - | Override the listen port |

### 4. Build and Start the Agent

On each build machine:

```bash
cd agent
cargo build --release
cp agent.toml /etc/semios-build-agent/agent.toml  # or any location
./target/release/semios-build-agent
```

Edit `agent.toml` before running:

```toml
master_url = "http://server-ip:3000"
name = "worker-1"
hostname = "192.168.1.100"
port = 3001
token = "your-shared-secret-token"
ports_repo_path = "/path/to/ports-on-this-machine"
```

| Key | Description |
|-----|-------------|
| `master_url` | URL of the build server |
| `name` | Unique machine name (used for routing builds) |
| `hostname` | Reachable hostname/IP for the server to connect back |
| `port` | Port the agent listens on |
| `token` | Must match the server's `agent.token` |
| `ports_repo_path` | Path to the ports repo on this machine |

### 5. Register a Machine

Machines register automatically when they connect. You can also add them manually via the web UI at `/machines` or by calling the API:

```bash
curl -X POST http://localhost:3000/api/machines \
  -H 'Content-Type: application/json' \
  -d '{"name":"worker-1","hostname":"192.168.1.100","port":3001}'
```

### 6. Load Packages

Click **Sync from Git** in the web UI (`/packages`) or call:

```bash
curl -X POST http://localhost:3000/api/sync
```

## Architecture

```
┌──────────────┐         ┌──────────────┐
│   Web UI     │◄───────►│  Build Server │
│  (browser)  │   HTTP   │  Node.js      │
└──────────────┘         └──────┬───────┘
                                │
                    ┌───────────┼───────────┐
                    ▼           ▼           ▼
              ┌──────────┐ ┌──────────┐ ┌──────────┐
              │  Agent A │ │  Agent B │ │  Agent C │
              │ (aarch64)│ │ (x86_64) │ │ (riscv64)│
              └──────────┘ └──────────┘ └──────────┘
```

- **Build Server** — SvelteKit SSR app with SQLite database. Manages packages, builds, artifacts, and dispatches build jobs to agents.
- **Build Agent** — Rust binary. Polls the server for pending builds, executes `./x --atombuild --target <target> build <package>`, streams logs in real-time, and uploads `.pkg` artifacts.

## File Layout

```
~/.cache/semios-build/
  ports/          # cloned ports repo (server-side)
  repo/           # published packages output

project-root/
  config.toml     # server configuration
  build-server.db # SQLite database (auto-created)
  uploads/        # build artifacts (build-<id>/)
  logs/           # build log files
  agent/
    agent.toml    # agent configuration
```

## Build Workflow

1. A user clicks **Build** on a package, selects architecture and atombuild option in the dialog.
2. The server creates a build record with status `queued` and computes the target (e.g. `aarch64-semios-linux`).
3. The dispatcher assigns the build to an idle machine with matching architecture.
4. The agent clones/updates the ports repo, runs the build, and streams logs to the server via `POST /api/builds/<id>/log`.
5. On completion, the agent discovers `.pkg` files and uploads them as artifacts.
6. The server marks the build as `success` or `failed`.
7. Artifacts can be added to the repository via **Add to Repo**, which calls `packie repo-admin --add-package`.

## Development

```bash
npm run dev    # starts vite dev server with HMR
```

## Troubleshooting

| Symptom | Fix |
|---------|-----|
| 500 errors on pages | Kill stale processes, rebuild with `rm -rf build .svelte-kit && npm run build`, restart |
| Agent not receiving builds | Check `master_url`, `token`, and network connectivity |
| Build fails immediately | Ensure `packie` is in `$PATH` and ports repo is cloned at `ports_repo_path` |
| Artifact upload fails | Set `BODY_SIZE_LIMIT=52428800` (50 MB) on the server |
| 401 on build/repo actions | Login required — use the sidebar login link or visit `/login` |
| Cannot add users | Generate hash via `/register` helper or CLI snippet, add `[users]` section to `config.toml` |