semios-build-server/INSTALL.md

SemiOS Build Server

A web-based package build server for SemiOS. Uses SvelteKit SSR for the UI and a Rust agent for executing builds on remote machines.

Requirements

Build Server

• Node.js >= 18
• npm
• Git
• A running instance of the packie package manager on the build machines

Build Agent (per machine)

• Rust >= 1.85 (edition 2024)
• Git
• The ports repository cloned on the machine
• packie installed and available in $PATH

Quick Start

1. Clone and Build the Server

git clone https://gitea.semilabs.org/semios/semios-build-server.git
cd semios-build-server
npm install
npm run build

2. Configure the Server

Copy and edit config.toml at the project root:

[server]
host = "0.0.0.0"
port = 3000

[ports]
repo_url = "https://gitea.semilabs.org/semios/ports.git"
repo_path = "/path/to/ports-cache"
sync_interval_secs = 300

[repo]
publish_dir = "/path/to/repo-output"

[agent]
token = "your-shared-secret-token"
listen_port = 3001

[users]
admin = "scrypt$<salt>$<hash>"

Key	Description
server.host	Bind address (0.0.0.0 for all interfaces)
server.port	Web UI port
ports.repo_url	Git URL of the ports repository
ports.repo_path	Local path where the server clones/caches the ports repo
ports.sync_interval_secs	How often to pull from upstream (seconds)
repo.publish_dir	Where published packages are collected
agent.token	Shared secret for agent authentication
agent.listen_port	Port the server expects agents to expose (for display only)
users.*	User accounts (username = scrypt password hash)

3. Add Users

Only logged-in users can start builds, add packages to the repository, or delete artifacts.

Use the registration helper at /register to generate password hashes. For each user:

1. Open http://localhost:3000/register in a browser
2. Enter username and password, click Generate Config Snippet
3. Copy the [users] line into config.toml
4. Restart the server

Or generate hashes from the command line:

node -e "
const crypto = require('crypto');
const salt = crypto.randomBytes(16).toString('hex');
const hash = crypto.scryptSync('your-password', salt, 64).toString('hex');
console.log('scrypt\$' + salt + '\$' + hash);
"

3. Start the Server

BODY_SIZE_LIMIT=52428800 node build/index.js

Or use the npm script:

npm run start

The server listens on http://0.0.0.0:3000 by default.

Environment Variables

Variable	Default	Description
BODY_SIZE_LIMIT	-	Max request body size in bytes (set to 52428800 for artifact uploads)
PORT	-	Override the listen port

4. Build and Start the Agent

On each build machine:

cd agent
cargo build --release
cp agent.toml /etc/semios-build-agent/agent.toml  # or any location
./target/release/semios-build-agent

Edit agent.toml before running:

master_url = "http://server-ip:3000"
name = "worker-1"
hostname = "192.168.1.100"
port = 3001
token = "your-shared-secret-token"
ports_repo_path = "/path/to/ports-on-this-machine"

Key	Description
master_url	URL of the build server
name	Unique machine name (used for routing builds)
hostname	Reachable hostname/IP for the server to connect back
port	Port the agent listens on
token	Must match the server's agent.token
ports_repo_path	Path to the ports repo on this machine

5. Register a Machine

Machines register automatically when they connect. You can also add them manually via the web UI at /machines or by calling the API:

curl -X POST http://localhost:3000/api/machines \
  -H 'Content-Type: application/json' \
  -d '{"name":"worker-1","hostname":"192.168.1.100","port":3001}'

6. Load Packages

Click Sync from Git in the web UI (/packages) or call:

curl -X POST http://localhost:3000/api/sync

Architecture

┌──────────────┐         ┌──────────────┐
│   Web UI     │◄───────►│  Build Server │
│  (browser)  │   HTTP   │  Node.js      │
└──────────────┘         └──────┬───────┘
                                │
                    ┌───────────┼───────────┐
                    ▼           ▼           ▼
              ┌──────────┐ ┌──────────┐ ┌──────────┐
              │  Agent A │ │  Agent B │ │  Agent C │
              │ (aarch64)│ │ (x86_64) │ │ (riscv64)│
              └──────────┘ └──────────┘ └──────────┘

• Build Server — SvelteKit SSR app with SQLite database. Manages packages, builds, artifacts, and dispatches build jobs to agents.
• Build Agent — Rust binary. Polls the server for pending builds, executes ./x --atombuild --target <target> build <package>, streams logs in real-time, and uploads .pkg artifacts.

File Layout

~/.cache/semios-build/
  ports/          # cloned ports repo (server-side)
  repo/           # published packages output

project-root/
  config.toml     # server configuration
  build-server.db # SQLite database (auto-created)
  uploads/        # build artifacts (build-<id>/)
  logs/           # build log files
  agent/
    agent.toml    # agent configuration

Build Workflow

1. A user clicks Build on a package, selects architecture and atombuild option in the dialog.
2. The server creates a build record with status queued and computes the target (e.g. aarch64-semios-linux).
3. The dispatcher assigns the build to an idle machine with matching architecture.
4. The agent clones/updates the ports repo, runs the build, and streams logs to the server via POST /api/builds/<id>/log.
5. On completion, the agent discovers .pkg files and uploads them as artifacts.
6. The server marks the build as success or failed.
7. Artifacts can be added to the repository via Add to Repo, which calls packie repo-admin --add-package.

Development

npm run dev    # starts vite dev server with HMR

Troubleshooting

Symptom	Fix
500 errors on pages	Kill stale processes, rebuild with rm -rf build .svelte-kit && npm run build, restart
Agent not receiving builds	Check master_url, token, and network connectivity
Build fails immediately	Ensure packie is in $PATH and ports repo is cloned at ports_repo_path
Artifact upload fails	Set BODY_SIZE_LIMIT=52428800 (50 MB) on the server
401 on build/repo actions	Login required — use the sidebar login link or visit /login
Cannot add users	Generate hash via /register helper or CLI snippet, add [users] section to config.toml