# RUNBOOK — software factory operations

Cookbook for operating the Phyllome OS factory on `git.phyllo.me`.

## Prereqs & inventory

- See [server-phyllome-cloudron.md](server-phyllome-cloudron.md) for the box
  reference and [architecture.md](architecture.md) for the layout.
- MCP servers (from the `automation` repo) used by an operator:
  `gitea-phyllome` (read), `gitea-phyllome-write` (write), `cloudron-phyllome`
  (read), `cloudron-phyllome-write` (write).

## 1. Register a runner (one-time)

The runner connects **out** to `git.phyllo.me`, so it never needs an inbound
rule. Registration needs a token from the Gitea UI:

1. **Get a registration token** (admin/owner action — cannot be done with the
   API token):
   Log in to `git.phyllo.me` → **Settings → Actions → Runners** →
   *New runner* → copy the token. Leave the window open; the token is
   one-time-use.
2. **Deploy the runner VM** on the phyllome Cloudron host. Two supported
   routes:
   - *Ansible* (existing playbook): `devops/ansible-gitea-runner` — set
     `registration_token` in `roles/runner_setup.yml`, point
     `inventory.ini` at the VM, then `ansible-playbook main.yml`.
   - *Manual*: install `act_runner` v0.2.13 on a Fedora VM, then:

     ```console
     $ sudo -u act_runner act_runner register --no-interactive \
         --instance https://git.phyllo.me --token <TOKEN> \
         --name fedora-0 --labels fedora
     ```

     Then run `act_runner daemon` under systemd (see
     `devops/ansible-gitea-runner/roles/runner_setup.yml` for the unit).
3. **Verify**: Gitea UI → **Settings → Actions → Runners** shows the runner
   **online**, label `fedora`.

> The old runner labels `fedora-cloud-42` were renamed to `fedora`
> (2026-09-15) — all workflows must use `runs-on: fedora`.

## 2. Add factory CI to a repo (pattern)

1. Push a workflow to `.gitea/workflows/<name>.yml`:

   ```yaml
   name: ci
   on:
     push:
     pull_request:
   jobs:
     checks:
       runs-on: fedora
       defaults:
         run:
           shell: bash
       container:
         image: git.phyllo.me/devops/fedora-runner-image:latest
       steps:
         - uses: https://git.phyllo.me/devops/checkout@v5
           with:
             fetch-depth: 0
         - run: make test
   ```

2. Use **host** jobs (no `container:` block) only when the job needs `mock`,
   `livemedia-creator`, or the host build cache — see the `create-iso`
   workflow for the canonical example.

## 3. Put a product pipeline under CI

| Repo | Workflow | Trigger |
|---|---|---|
| `roots/phyllomeos` | `ci.yml` (lint/test/validate) | push + PR |
| `roots/xml-definition-for-domains` | `ci.yml` (xmllint) | push + PR |
| `roots/rpm-sources` | `smoke.yml` (mock build) | push + PR (host job) |
| `devops/create-iso` | existing `build-iso.yaml` | tag `v*.*.*` |

## 4. Build & release an ISO (create-iso)

Triggered by tagging `v*.*.*` in `devops/create-iso`. The `build-iso.yaml`
workflow: `mock --init` → install `lorax-lmc-novirt` + `pykickstart` +
`livecd-tools` → `livemedia-creator --make-iso` → release the ISO via
`devops/action-gh-release@v2`. Requires a **host** runner (mock needs real
chroots).

## 5. Troubleshooting

- **Runner never comes online**: re-check the registration token (one-time use)
  and that `act_runner daemon` is running (`systemctl status act_runner`).
- **Job stuck in `queued`/`waiting for runner`**: label mismatch — the job's
  `runs-on` must exactly match a label the runner registers (`fedora`).
- **Container job can't pull the image**: runner needs network to
  `git.phyllo.me` package registry; check `docker pull
  git.phyllo.me/devops/fedora-runner-image:latest` on the VM; verify the token
  used for registry login has package write/read scope.
- **Push to `git.lukasgreve.ch` (perso) times out**: home Cloudron
  reachability is flaky from this workstation; retry — it is *not* a factory
  failure path (the factory only talks to `git.phyllo.me`).

## 6. Admin actions that stay in the Cloudron UI

Restore/clear backups, reboot, domain/user management, runner registration
tokens — the MCP write surface deliberately excludes these (see the
`automation` README).