diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..caa04e5c --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,571 @@ + + +# AppAPI agent guide + +This is the start point for working with **`app_api`**, both **operating** AppAPI (installing and managing +External Apps on a real Nextcloud) and **developing** this repo. It is written so an AI assistant can pick it +up as context and help a colleague set up, verify and troubleshoot AppAPI correctly, and so a new contributor +(human or AI) can build and change the code. + +If you are helping someone **install or run** AppAPI, start at [Quickstart](#2-quickstart-zero-to-a-working-exapp) +and [Troubleshooting](#10-troubleshooting-symptom-first). If you are helping **develop** `app_api`, start at +[Developing app_api](#12-developing-app_api-start-here). + +Applies to **Nextcloud 33, 34 and 35**. This repo's `main` is the NC35 dev line; released majors live on +`stableXX` branches. Behavior that differs by version is called out in [Version notes](#11-version-notes-nc33--34--35). +This is a **living document**: when you change AppAPI's behavior, update the relevant section in the same +change. Keep it portable (see [Keep this file portable](#15-keep-this-file-portable-and-current)). + +For building the ExApp side (Python), see the sibling project **nc_py_api** (`cloud-py-api/nc_py_api`). + +**Detailed runbooks** live under `docs/appapi/` and are linked from the sections below: +[Kubernetes](docs/appapi/kubernetes.md), [ExApps on a separate host](docs/appapi/remote-daemon.md), +[Nextcloud AIO](docs/appapi/aio.md), and the [ExApp manifest reference](docs/appapi/exapp-contract.md). They +hold depth that would bloat this hub; open the relevant one when working on that topic. + +## Table of contents + +1. [What AppAPI is](#1-what-appapi-is) +2. [Quickstart: zero to a working ExApp](#2-quickstart-zero-to-a-working-exapp) +3. [Deploy daemons: which to use](#3-deploy-daemons-which-to-use) +4. [Setup cases (topologies)](#4-setup-cases-topologies) +5. [`occ app_api:daemon:register` reference](#5-occ-app_apidaemonregister-reference) +6. [ExApp lifecycle (occ)](#6-exapp-lifecycle-occ) +7. [Operating AppAPI](#7-operating-appapi) +8. [App store / fetcher](#8-app-store--fetcher) +9. [Runtime and the ExApp contract](#9-runtime-and-the-exapp-contract) +10. [Troubleshooting (symptom-first)](#10-troubleshooting-symptom-first) +11. [Version notes (NC33 / 34 / 35)](#11-version-notes-nc33--34--35) +12. [Developing app_api (start here)](#12-developing-app_api-start-here) +13. [Key files](#13-key-files) +14. [Related links](#14-related-links) +15. [Keep this file portable and current](#15-keep-this-file-portable-and-current) + +Throughout, `occ` means the Nextcloud server console. Where it runs depends on your install: + +- **Docker / docker compose**: inside the Nextcloud container, e.g. + `docker exec -u www-data php occ ` (find the container with `docker compose ps` + or `docker ps`). +- **Snap**: `nextcloud.occ `. +- **Bare-metal / other**: `sudo -u www-data php occ ` (or `./occ`). + +Every AppAPI option below is long-form only (there are no short flags). + +## 1. What AppAPI is + +AppAPI is the Nextcloud component that enables **External Apps (ExApps)**: apps whose backend runs **outside** +the Nextcloud PHP process (usually as a Docker container), while still integrating with Nextcloud users, +permissions and the web UI. + +- **This repo (`app_api`)**: PHP backend + Vue frontend. It stores daemon configuration, manages ExApp + lifecycle (install, enable, disable, update, remove), and authorizes/routes traffic to ExApps. +- **Deploy Daemon**: the external service Nextcloud talks to in order to install, start/stop and reach ExApps. + Without a configured Deploy Daemon, AppAPI cannot deploy anything, and Nextcloud shows the admin warning + "AppAPI default deploy daemon is not set". +- **HaRP** (`nextcloud/HaRP`): the recommended daemon (NC32+), a high-performance reverse proxy that proxies + the Docker Engine, routes requests straight to ExApps (bypassing PHP, enabling WebSockets), and uses FRP + tunnels so ExApps need not expose host ports. +- **nc_py_api**: the Python framework used to write ExApps that call back into Nextcloud through AppAPI. + +ExApps are **trusted, first-class apps**, comparable to PHP apps running inside Nextcloud: they authenticate +with a per-install app secret and integrate with user sessions. Install only ExApps you trust, exactly as you +would with regular Nextcloud apps. + +AppAPI is only useful if you want to install or develop External Apps. If you do not, you can disable it +(`occ app:disable app_api`) and the "default deploy daemon" warning disappears. + +## 2. Quickstart: zero to a working ExApp + +The golden path on a Docker-based Nextcloud, using HaRP. This is the setup most colleagues want. Replace the +placeholders in angle brackets; never paste a real secret into a shared file. + +**Prerequisites**: Nextcloud 33+ with admin access; a Docker Engine reachable from where you run HaRP; HaRP +able to reach your Nextcloud URL. + +- **On Nextcloud AIO**: HaRP is auto-registered as the `harp_aio` daemon (NC33+); skip Steps 2-6 and go to + Step 7. See [`docs/appapi/aio.md`](docs/appapi/aio.md). +- **If Nextcloud itself is not in Docker** (snap/bare-metal): in Step 3 omit `--network` and publish HaRP's + ports; in Step 5 drop `--net`, set `host` to `localhost:8780` and `--harp_frp_address localhost:8782`. Run + `occ` natively (see the note above). + +**Step 1. Enable AppAPI.** + +```bash +occ app:enable app_api +``` + +**Step 2. Pick one shared secret.** HaRP and AppAPI authenticate to each other with a single shared key. Use +a strong ASCII string and reuse the exact same value in Step 3 and Step 5. + +```bash +export HP_SHARED_KEY="" +``` + +**Step 3. Start HaRP** (the Deploy Daemon). Put it on the same Docker network as Nextcloud so they can reach +each other by container name. + +```bash +docker run -d \ + --name appapi-harp -h appapi-harp \ + --restart unless-stopped \ + --network \ + -e HP_SHARED_KEY="$HP_SHARED_KEY" \ + -e NC_INSTANCE_URL="" \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v "$(pwd)/certs:/certs" \ + -p 8780:8780 \ + -p 8782:8782 \ + ghcr.io/nextcloud/nextcloud-appapi-harp:release +``` + +- Find `` with `docker network ls` (a compose install usually names it + `_default`, e.g. `nextcloud_default`). +- `HP_SHARED_KEY` and `NC_INSTANCE_URL` are the only required env vars. Set `NC_INSTANCE_URL` to this + Nextcloud's URL as reachable **from inside the Docker network** (your public URL usually works; never + `localhost`). Reuse the same value in Step 5. +- Ports: `8780` = ExApps HTTP frontend (the reverse-proxy target in Step 4). `8782` = FRP TCP frontend. + `8781` = optional HTTPS (needs `/certs`). Behind a reverse proxy, add `-e HP_TRUSTED_PROXY_IPS=""`. + +> **Security.** Mounting the Docker socket gives HaRP root-equivalent control of the host: run it only on a +> host you trust, from the official image. The `-p` mappings publish on all interfaces by default; on a single +> host prefer `-p 127.0.0.1:8780:8780` and reach `8782` over the internal Docker network, or firewall both +> ports so they are never on a public interface. Do not set `HP_FRP_DISABLE_TLS` on an untrusted network. + +**Step 4. Route `/exapps/` to HaRP on your Nextcloud reverse proxy.** Browsers reach ExApp frontends directly +through HaRP (bypassing PHP; required for WebSockets/streaming). On whatever proxy already terminates TLS for +Nextcloud, forward `/exapps/` to HaRP `:8780`. **The daemon checks and Test deploy in Step 6 pass without this +rule**, but ExApp UIs and WebSocket endpoints will be unreachable from the browser. + +nginx: + +```nginx +location /exapps/ { + proxy_pass http://:8780; # container name if on the same Docker network + proxy_http_version 1.1; + proxy_set_header Host $host; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; +} +``` + +Apache needs `mod_proxy_wstunnel` for the WebSocket upgrade (a plain `ProxyPass` proxies HTTP but silently +drops WebSockets): + +```apache +# a2enmod proxy proxy_http proxy_wstunnel rewrite +RewriteEngine On +RewriteCond %{HTTP:Upgrade} websocket [NC] +RewriteRule ^/exapps/(.*)$ ws://:8780/exapps/$1 [P,L] +ProxyPass /exapps/ http://:8780/exapps/ +ProxyPassReverse /exapps/ http://:8780/exapps/ +``` + +Then make sure HaRP has `-e HP_TRUSTED_PROXY_IPS=""` (Step 3). Full examples: +https://github.com/nextcloud/HaRP + +**Step 5. Register HaRP in AppAPI and make it the default daemon.** The positional order is +`name display-name accepts-deploy-id protocol host nextcloud_url`. + +```bash +occ app_api:daemon:register \ + harp1 "HaRP" docker-install http :8780 \ + --net \ + --harp \ + --harp_frp_address :8782 \ + --harp_shared_key "$HP_SHARED_KEY" \ + --set-default +``` + +- Use the **same** URL for the `nextcloud_url` positional here and `NC_INSTANCE_URL` in Step 3. +- The single most common failure is a **shared-key mismatch**: `--harp_shared_key` must be byte-identical to + HaRP's `HP_SHARED_KEY` from Step 2. Use `https` + port `8781` instead of `http` + `8780` only if you are + reaching HaRP across an untrusted network with certs configured. + +**Step 6. Verify the daemon.** + +```bash +occ app_api:daemon:list +``` + +In the UI, open **Settings --> Administration --> AppAPI** and use **Check connection** and **Test deploy** +(the latter installs and removes a real test ExApp, exercising image pull + run, not just connectivity). +The admin setup checks `DaemonCheck` (daemon reachable, default set) and `HarpVersionCheck` (HaRP new enough) +should be green. Note: green checks confirm the internal Nextcloud-to-HaRP path only; the **browser** path +still needs the `/exapps/` proxy rule from Step 4. + +**Step 7. Install an ExApp.** Browse installable ExApps in the UI store (**Settings --> Administration --> +AppAPI**) or at https://apps.nextcloud.com/ to find its id; there is no `occ` command +that lists store apps. Then install it (this pulls the app from the App Store and uses the default daemon): + +```bash +occ app_api:app:register --wait-finish +``` + +**Step 8. Confirm it is running.** + +```bash +occ app_api:app:list # shows: (): [enabled] +docker ps --filter name=nc_app_ # ExApp containers are prefixed nc_app_ +``` + +That is a working ExApp. From here, [Operating AppAPI](#7-operating-appapi) covers day-2 management and +[Troubleshooting](#10-troubleshooting-symptom-first) covers what to check when a step fails. + +## 3. Deploy daemons: which to use + +A HaRP daemon is **not** identified by its deploy type. Both HaRP and the legacy Docker Socket Proxy use the +`docker-install` type; a daemon is HaRP when its `deploy_config` carries a `harp` sub-array +(`HarpService::isHarp()`). The `--harp` flag on registration is what creates that array. + +| Deploy type | Class | When to use | Status | +|---|---|---|---| +| `docker-install` + `--harp` | `DockerActions` | Default. Docker host (local or remote) via HaRP | Recommended (NC32+) | +| `docker-install` (no `--harp`) | `DockerActions` | Legacy Docker Socket Proxy (DSP) | Deprecated (see [Version notes](#11-version-notes-nc33--34--35)) | +| `kubernetes-install` | `KubernetesActions` | Kubernetes cluster, always via HaRP | Supported (NC34+) | +| `manual-install` | `ManualActions` | Local development; ExApp runs outside any orchestration | Dev only | + +GPU: add `--compute_device cuda` (NVIDIA) or `--compute_device rocm` (AMD) to any daemon for AI ExApps +(`cpu` is the default). + +## 4. Setup cases (topologies) + +| Case | How to register | Notes | +|---|---|---| +| HaRP on the same Docker host as Nextcloud | `docker-install --harp`, `host` = `:8780`, `--harp_frp_address :8782` | Most common single-host setup (the Quickstart) | +| ExApps on a **separate host** | HaRP near NC + FRP-tunneled remote engine (`--harp_docker_socket_port`), or HaRP on the ExApp host | Runbook: `docs/appapi/remote-daemon.md` | +| Kubernetes (NC34+) | `--k8s --harp` (forces `kubernetes-install`) + `--k8s_expose_type` | FRP address not required for K8s; all K8s ops go through HaRP. Runbook: `docs/appapi/kubernetes.md` | +| Nextcloud AIO | auto-registered `harp_aio` daemon (`nextcloud-aio-harp:8780`) when HaRP is enabled (NC33+) | Managed by AIO. Runbook: `docs/appapi/aio.md` | +| Local development | `manual-install` | ExApp process runs on your machine; pair with nc_py_api dev mode | +| Legacy Docker Socket Proxy | `docker-install` (no `--harp`) + `--haproxy_password` | Deprecated; migrate to HaRP | + +## 5. `occ app_api:daemon:register` reference + +Source: `lib/Command/Daemon/RegisterDaemon.php`. Positional arguments (all required, in order): + +| # | Argument | Meaning | +|---|---|---| +| 1 | `name` | Unique daemon name/id | +| 2 | `display-name` | Human-readable name shown in the UI | +| 3 | `accepts-deploy-id` | `manual-install`, `docker-install`, or `kubernetes-install` | +| 4 | `protocol` | `http` or `https` (how Nextcloud connects to the daemon) | +| 5 | `host` | Where the daemon is reachable, e.g. `:8780` or a Docker socket path | +| 6 | `nextcloud_url` | URL of this Nextcloud as reachable **from the ExApps**; it becomes each ExApp's `NEXTCLOUD_URL`. On a co-located single host it equals HaRP's `NC_INSTANCE_URL`; in split topologies (K8s, remote host) it may differ from what HaRP uses | + +Options: + +| Option | Meaning | +|---|---| +| `--net` | Docker network name (default `host`; `bridge` for `--k8s` daemons) | +| `--haproxy_password` | Basic-auth password for a Docker Socket Proxy daemon (DSP only) | +| `--compute_device` | `cpu`, `cuda`, or `rocm` | +| `--set-default` | Store as the default daemon (app config key `default_daemon_config`) | +| `--harp` | Use HaRP for all Docker + ExApp communication | +| `--harp_frp_address` | `host:port` of the HaRP FRP server (the FRP port is typically `8782`); required for HaRP unless `--k8s` | +| `--harp_shared_key` | HaRP shared key; must equal HaRP's `HP_SHARED_KEY` | +| `--harp_docker_socket_port` | FRP remote port selecting the Docker Engine (default `24000` = HaRP's local engine; remote engines use `24001-24099`, see `docs/appapi/remote-daemon.md`) | +| `--harp_exapp_direct` | Advanced: disable the FRP tunnel between ExApps and HaRP (see note below) | +| `--k8s` (NC34+) | Mark as Kubernetes daemon (requires `--harp`; forces `kubernetes-install`) | +| `--k8s_expose_type` (NC34+) | `nodeport`, `clusterip` (default), `loadbalancer`, or `manual` | +| `--k8s_node_port` (NC34+) | NodePort `30000-32767` (nodeport type only) | +| `--k8s_upstream_host` (NC34+) | Override upstream host for HaRP-to-ExApp (required for `manual` expose type) | +| `--k8s_external_traffic_policy` (NC34+) | `Cluster` or `Local` | +| `--k8s_load_balancer_ip` (NC34+) | LoadBalancer IP (loadbalancer type only) | +| `--k8s_node_address_type` (NC34+) | `InternalIP` (default) or `ExternalIP` | + +Rules the command enforces: + +- `--harp` requires `--harp_shared_key`, and requires `--harp_frp_address` unless `--k8s` is set. +- `--k8s` requires `--harp` and forces `accepts-deploy-id` to `kubernetes-install`. +- Registering a plain `docker-install` daemon without `--harp` (DSP) prints a deprecation/removal warning on + NC34+ (see [Version notes](#11-version-notes-nc33--34--35)). +- `--harp_exapp_direct` drops the reverse FRP tunnel, so the ExApp must be directly network-reachable by HaRP; + `net=host` is disallowed in this mode. Only use it inside a trusted network segment. + +Kubernetes daemons (NC34+) use `kubernetes-install` with `http` against HaRP's `:8780` and need no +`--harp_frp_address`; they are occ-only (the admin UI shows them read-only). Full setup and the register +command (HaRP `HP_K8S_*` config, RBAC, expose types): [`docs/appapi/kubernetes.md`](docs/appapi/kubernetes.md). + +Verify with `occ app_api:daemon:list` and the admin UI **Check connection** / **Test deploy**. + +## 6. ExApp lifecycle (occ) + +Typical flow: **register (install) --> enable --> [use] --> disable --> unregister**; `update` in place. +Source: `lib/Command/ExApp/`. `appid` is the ExApp's id. + +| Command | Args and key options | +|---|---| +| `app_api:app:register [daemon]` | Install an ExApp. Omit `[daemon]` to use the default. Definition source: App Store (default), or `--info-xml `, or `--json-info `. `--env NAME=VALUE` (repeatable) sets container env, but only for variables the app **declares** in its manifest (undeclared names are silently ignored); `--mount SRC:DST[:ro\|rw]` (repeatable) adds bind mounts (Docker daemons; recorded but not mounted on K8s). `--wait-finish` blocks until deployed; `--silent`; `--test-deploy-mode` re-registers if already present. See `docs/appapi/exapp-contract.md`. | +| `app_api:app:enable ` | Enable a registered ExApp. No options. | +| `app_api:app:disable ` | Disable a registered ExApp. No options. | +| `app_api:app:update [appid]` | Update one ExApp, or `--all` (with `--showonly` to preview, `--include-disabled` to widen). Reuses the ExApp's stored daemon and deploy options. | +| `app_api:app:unregister ` | Remove an ExApp. `--rm-data` also deletes its persistent volume (data is **kept** by default). `--force` continues past errors; `--silent`. The Docker image is never removed automatically; prune it manually if disk space matters. | +| `app_api:app:list` | List ExApps: ` (): [enabled\|disabled]`. No options. | + +Notes: + +- **App Store install** = `app_api:app:register ` with no `--info-xml`/`--json-info`. **Manual/local + install** = supply the definition via `--info-xml` or `--json-info`. +- If `[daemon]` is omitted, the command uses the `default_daemon_config` app-config value; if no default is + set it fails. Set one with `--set-default` at daemon registration. +- `--keep-data` (on unregister) and `--force-scopes` (on register/update) are **deprecated no-ops**; do not + rely on them. Data is kept by default; use `--rm-data` to delete it. +- Unregister cleans up everything the ExApp registered (UI entries, AI providers, Talk bots, webhooks, occ + commands) but deliberately **keeps its app config and per-user preferences**, so a reinstall picks up the + previous settings. There is no purge flag for those. + +## 7. Operating AppAPI + +- **ExApp configuration**: `app_api:app:config:get|set|delete|list` inspect or modify an ExApp's stored + key/value configuration. +- **Private/mirror Docker registries**: `app_api:daemon:registry:add|remove|list` map registries for a + daemon so ExApp images can be pulled from somewhere other than the default + (`registry:add --registry-from --registry-to `). +- **Daemons**: `app_api:daemon:list` / `app_api:daemon:unregister` manage daemon configs; re-run + `app_api:daemon:register ... --set-default` to change the default. Note that `daemon:register` is a no-op if + a daemon with that `name` already exists (see [Troubleshooting](#10-troubleshooting-symptom-first)). + Unregistering a daemon is **blocked while ExApps still use it**, and there is no command to move an installed + ExApp between daemons: unregister the ExApp and reinstall it on the new daemon (its config survives, see the + lifecycle notes). That is also the DSP-to-HaRP migration path. +- **Logs**: ExApp containers are prefixed `nc_app_` (`docker logs nc_app_`); the daemon logs live in + the HaRP container; Nextcloud-side errors are in the Nextcloud log. +- **Health checks**: the shipped admin setup checks are `DaemonCheck` (daemon reachable, default set) and + `HarpVersionCheck` (HaRP new enough); they are the built-in post-install verification. An in-flight change + targeting NC35 adds ExApp-surfaced checks (`ExAppsErrorSetupCheck`, `ExAppsWarningSetupCheck`) that raise + ExApp-reported errors and "not responding" warnings into the admin overview; these are not yet in a stable + release. Background jobs such as `ExAppInitStatusCheckJob` and + `ExAppSetupChecksRefreshJob` refresh ExApp init/health state. +- **Certificates**: Nextcloud's certificate store (`occ security:certificates`) is pushed into every ExApp + container at **deploy time** (all daemon types), so ExApps trust the same CAs as Nextcloud, including + self-signed setups. After importing a new CA, update or reinstall ExApps to propagate it. Daemon connections + over `https` always verify TLS with that same store and there is no bypass flag; import a self-signed daemon + certificate into the store first. +- **Maintenance mode** (NC35+): `occ app_api:*` commands keep working (occ prints "only AppAPI commands are + loaded") and the HaRP control routes (ExApp metadata, init progress/state, logging) stay available, while + ExApp end-user traffic and the ExApp config/preference APIs are rejected until maintenance ends (blocked + AppAPI routes return 503 with `X-Nextcloud-Maintenance-Mode: 1` and `Retry-After: 120`). On NC33/34, app_api is not loaded during maintenance at all, so ExApps and + `occ app_api:*` are unavailable for the duration. + +## 8. App store / fetcher + +There are **no** occ commands for the App Store; it is code-only under `lib/Fetcher/`. + +- Default store URL `https://apps.nextcloud.com/api/v1` (`AppAPIFetcher::APP_STORE_URL`). +- Override with the Nextcloud system value `appstoreurl` (the only "custom app store" mechanism); the store is + disabled if `appstoreenabled` is false. +- ExApp catalog file: `appapi_apps.json` (`ExAppFetcher`); updates are computed by `getExAppsWithUpdates()`. + +## 9. Runtime and the ExApp contract + +``` +Browser --> Nextcloud reverse proxy (/exapps/*) --> HaRP (:8780) --> FRP tunnel --> ExApp container +``` + +- Nextcloud-to-daemon calls send the `harp-shared-key` header; ExApp URLs are under `/exapps/app_api/...` + (`HarpService::initGuzzleClient()`; `getHarpSharedKey()` decrypts the key stored, encrypted, in + `deploy_config['haproxy_password']`). +- A daemon is HaRP when `deploy_config['harp']` is set (`HarpService::isHarp()`), independent of + `accepts_deploy_id`. +- Direct-connect mode (`--harp_exapp_direct`) drops the ExApp-to-HaRP FRP tunnel; note `net=host` is + disallowed with HaRP direct mode. +- Some simpler UI integrations instead proxy through Nextcloud's own PHP route + (`/index.php/apps/app_api/proxy/...`, `ExAppProxyController`), so a trivial ExApp may work even without the + Step 4 `/exapps/` rule; WebSocket/streaming apps do not. + +### The ExApp contract + +AppAPI injects the same environment into every ExApp container (Docker and Kubernetes, +`DockerActions`/`KubernetesActions`): + +| Env var | Meaning | +|---|---| +| `APP_ID`, `APP_VERSION`, `APP_DISPLAY_NAME` | Identity, from the ExApp's `info.xml` | +| `APP_SECRET` | Per-install shared secret for ExApp-to-Nextcloud authentication | +| `APP_HOST`, `APP_PORT` | Where the ExApp backend must listen | +| `APP_PERSISTENT_STORAGE` | Path of the persistent data volume | +| `NEXTCLOUD_URL` | The daemon's `nextcloud_url` positional | +| `COMPUTE_DEVICE` | `cpu`/`cuda`/`rocm` (plus `NVIDIA_*` vars for cuda) | +| `HP_FRP_ADDRESS`, `HP_FRP_PORT`, `HP_SHARED_KEY` | FRP tunnel wiring (HaRP Docker daemons only; K8s forces direct mode and omits them) | +| `AA_VERSION` | AppAPI version | + +Lifecycle and authentication: + +- After deploy, AppAPI waits for the ExApp's `/heartbeat` to respond, calls `/init`, and the ExApp reports + init progress (0-100) back through the OCS status endpoint; at 100 it gets enabled. "Stuck initializing" + means this loop stalled (see [Troubleshooting](#10-troubleshooting-symptom-first)). +- ExApp calls to Nextcloud carry the `EX-APP-ID`, `EX-APP-VERSION` and `AUTHORIZATION-APP-API` (base64 + `userid:APP_SECRET`) headers, validated by `AppAPIAuthMiddleware`. This is a **different credential** than + the daemon's `harp-shared-key`: a 401 on an ExApp API call points at the app secret/headers, not the daemon + key. +- Through these APIs an ExApp can register UI elements (top-menu entries, Files actions, scripts/styles), + Task Processing (AI) providers, Talk bots, declarative settings, webhook listeners, and its own `occ` + commands. All of these are cleaned up when the ExApp is unregistered. + +What the ExApp itself declares (routes and their access levels, image coordinates, the env-var allow-list, +Kubernetes service roles) lives in its `info.xml` `` manifest: +see [`docs/appapi/exapp-contract.md`](docs/appapi/exapp-contract.md). + +## 10. Troubleshooting (symptom-first) + +- **"AppAPI default deploy daemon is not set"**: no default daemon. Register one with `--set-default` + ([Quickstart](#2-quickstart-zero-to-a-working-exapp)), or disable `app_api` if you do not use ExApps. +- **Daemon "Check connection" / `DaemonCheck` fails**: confirm `protocol`/`host` are reachable from the + Nextcloud container; for HaRP confirm `--harp_shared_key` equals HaRP's `HP_SHARED_KEY`; confirm HaRP is + running (`docker ps`) and on the same network as Nextcloud. +- **Shared-key errors / 401 from HaRP**: the `--harp_shared_key` and `HP_SHARED_KEY` differ, or the key has + non-ASCII characters. Re-register the daemon with the exact key. +- **Fixed the key/host but nothing changed?** `app_api:daemon:register` is a no-op when a daemon with that + `name` already exists (it prints "Registration skipped ..." and exits 0, so `--set-default` is skipped too). + Run `occ app_api:daemon:unregister ` first, then re-register. +- **ExApp installed and `[enabled]`, but its page is blank / WebSocket fails**: the `/exapps/` reverse-proxy + rule (Quickstart Step 4) is missing. The daemon checks pass without it because they use the internal path. +- **ExApp will not deploy**: image pull failing (registry/network). Check `docker ps -a` for `nc_app_*`, the + ExApp container logs, and HaRP logs; a private registry needs `app_api:daemon:registry:add`. +- **HaRP not routing / 502**: wrong `--harp_frp_address`/port (default `8782` not reachable), or the reverse + proxy is not forwarding `/exapps/` to HaRP `:8780`, or `net=host` was combined with HaRP direct mode. Check + HaRP container logs. +- **ExApp unhealthy / stuck initializing**: `ExAppInitStatusCheckJob` tracks init state. Check the ExApp + container logs and confirm `nextcloud_url` (and HaRP's `NC_INSTANCE_URL`) is reachable from the ExApp side. +- **`HarpVersionCheck` warns**: the HaRP container is older than the minimum supported version; update the + HaRP image (`:release`). +- **GPU ExApp not using the GPU**: register the daemon with `--compute_device cuda|rocm`. +- **DSP deprecation warnings**: expected on NC34+; migrate the daemon to HaRP (`--harp`). + +## 11. Version notes (NC33 / 34 / 35) + +`main` is the NC35 dev line; released majors are `stableXX` branches. There is no runtime Nextcloud-version +gating in the deploy/registration code, so these differences are which code shipped in which major. + +| Capability | NC33 | NC34 | NC35 (dev) | +|---|---|---|---| +| HaRP daemon | yes | yes | yes | +| Legacy DSP (`docker-install`, no `--harp`) | yes, no deprecation warning | yes, deprecation warning | present, removal targeted | +| Kubernetes (`kubernetes-install`, `--k8s*`) | no | yes | yes | +| `daemon:register` options | 9 (Docker/HaRP options, no `--k8s*`) | 16 (adds `--k8s*`) | 16 | +| AIO auto-daemon | `docker_aio` + `harp_aio` (neither deprecated) | `docker_aio` deprecated, `harp_aio` | `harp_aio` (`docker_aio` deprecated) | +| Connection/HaRP setup checks | `DaemonCheck`, `HarpVersionCheck` | same | same | +| ExApp-surfaced setup checks | no | no | in flight, not yet released | +| AppAPI available during maintenance mode | no | no | yes (occ + HaRP control routes) | + +- The `--harp_*` flags are stable across NC33-35. The `--k8s_*` flags and `KubernetesActions` arrived in NC34. +- `harp_aio` auto-registration exists from NC33; the NC34 change was deprecating `docker_aio` (removal + targeted for NC35). +- A hard stop that forbids registering **new** DSP daemons is planned for NC35 but is not in released code as + of this writing; treat DSP as deprecated everywhere and prefer HaRP. +- ExApp-surfaced setup checks (`ExAppsErrorSetupCheck`, `ExAppsWarningSetupCheck`) are in flight for NC35 and + not yet in a stable release; only `DaemonCheck` and `HarpVersionCheck` ship today. +- On NC35, maintenance mode keeps `occ app_api:*` and the HaRP control routes available while ExApp user + traffic is rejected (see [Operating AppAPI](#7-operating-appapi)); NC33/34 do not load app_api during + maintenance at all. + +## 12. Developing app_api (start here) + +`app_api` is a standard Nextcloud app: PHP backend in `lib/`, Vue frontend in `src/` (two webpack bundles from +`src/adminSettings.js` and `src/filesplugin.js`, built into `js/`), app metadata in `appinfo/`. HTTP routes +are declared in `appinfo/routes.php` and served by controllers in `lib/Controller/`. Deploy backends implement +`IDeployActions` (`DockerActions`/`ManualActions`/`KubernetesActions`); "is this HaRP" is `deploy_config['harp']`, +not the deploy type. See [Key files](#13-key-files) for where things live. + +### Run your changes on a local Nextcloud + +To test this checkout manually (distinct from the Quickstart, which installs the released app): + +- Place or symlink this repo into the server's apps directory (e.g. `custom_apps/app_api` or + `apps-extra/app_api`). +- `occ app:enable app_api`. +- `npm run watch` rebuilds the frontend on change; reload Nextcloud to pick it up. + +### Build, test, lint + +Use the composer/npm scripts (these are what CI runs). All from the repo root. + +```bash +# PHP backend +composer install +composer cs:check # php-cs-fixer dry-run over ./lib +composer cs:fix # auto-fix code style +composer psalm # static analysis (psalm.phar) +composer test:unit # PHPUnit (config: tests/php/phpunit.xml) +composer openapi # regenerate the OpenAPI specs (see below) +composer lint # php -l syntax check + +# Vue frontend +npm ci +npm run watch # dev build with watch (webpack) +npm run build # production build; commit the resulting js/ assets +npm run lint # eslint (src) +npm run stylelint +npm test # vitest (JS unit tests) +``` + +### CI gates (what must pass) + +- `lint.yml`: `info.xml` XSD, `composer lint`, `composer cs:check`, `composer psalm`, `npm run lint`, + `npm run stylelint` (aggregated as `Lint-OK`). +- `phpunit.yml`: `composer test:unit` on PHP 8.3 and 8.4. +- `js-test.yml`: vitest on `src/**` changes. +- `openapi.yml`: runs `composer openapi` and **fails if the committed `openapi*.json` (and, if applicable, + `src/types/openapi/*.ts`) are stale**. Regenerate and commit them whenever you touch controllers/routes. +- `node.yml`: `npm run build` and **fails if compiled `js/` assets are not committed**. +- `reuse.yml`: every file needs SPDX licensing info, via a file header or a `REUSE.toml` annotation. +- `tests-deploy*.yml`: end-to-end daemon lifecycle across Docker / HaRP / DSP and the four K8s expose types; + `tests.yml` runs nc_py_api integration (PgSQL/MySQL/APcu). CI targets the server `master` line. + +### Contributing conventions + +- **Sign off every commit (DCO)**: `git commit -s`. The sign-off name/email must match the commit author. + Nextcloud requires this to merge. +- **Commit messages**: concise, one line. Reference issues in the PR description, not the commit subject. +- **Target branch** `main`; release fixes are backported to the relevant `stableXX`. +- **PHP floor 8.2**, so do not use 8.3+ only syntax in `lib/`. Frontend engines: Node `^22`, npm `^10`. +- Before pushing: `composer cs:fix && composer psalm && composer test:unit`; if you touched the frontend, + `npm run lint && npm run build`; if you touched controllers/routes, `composer openapi`. Commit the + regenerated `openapi*.json`, any `src/types/openapi/*.ts`, and `js/` assets. +- New files need an SPDX header (see the top of this file for the format). + +## 13. Key files + +| Area | File(s) | +|---|---| +| Daemon registration CLI | `lib/Command/Daemon/RegisterDaemon.php` | +| ExApp lifecycle CLI | `lib/Command/ExApp/` | +| HTTP routes | `appinfo/routes.php` | +| Controllers | `lib/Controller/` (e.g. `ExAppProxyController`, `HarpController`) | +| Deploy backends | `lib/DeployActions/{DockerActions,ManualActions,KubernetesActions}.php` | +| AIO auto-registration | `lib/DeployActions/AIODockerActions.php` | +| HaRP logic | `lib/Service/HarpService.php` | +| Daemon config service | `lib/Service/DaemonConfigService.php` | +| App Store fetchers | `lib/Fetcher/{AppAPIFetcher,ExAppFetcher,ExAppArchiveFetcher}.php` | +| Setup checks | `lib/SetupChecks/` (`DaemonCheck`, `HarpVersionCheck`) | +| Background jobs | `lib/BackgroundJob/` | +| DB migrations / repair steps | `lib/Migration/` (`VersionXXXXXXDateYYYYYYYY.php` schema classes + repair steps) | +| Frontend | `src/` (Vue), entries `src/adminSettings.js` + `src/filesplugin.js`, built into `js/` via `webpack.js` | +| App metadata / command + job registration | `appinfo/info.xml` | +| Generated API specs | `openapi.json`, `openapi-administration.json`, `openapi-full.json` | +| Detailed runbooks | `docs/appapi/` (`kubernetes.md`, `remote-daemon.md`, `aio.md`, `exapp-contract.md`) | + +## 14. Related links + +- **nc_py_api** (build ExApps in Python): https://github.com/cloud-py-api/nc_py_api +- **HaRP**: https://github.com/nextcloud/HaRP +- **Docker Socket Proxy** (legacy): https://github.com/nextcloud/docker-socket-proxy +- **Admin docs**: https://docs.nextcloud.com/server/latest/admin_manual/exapps_management/ +- **Developer docs**: https://docs.nextcloud.com/server/latest/developer_manual/exapp_development/ + +## 15. Keep this file portable and current + +This file ships in a public repo and is read by other people's AI assistants. Keep it true for **any** +deployment: + +- **No secrets** and no real shared keys: use `$HP_SHARED_KEY` or ``. +- **No environment-specific values**: container names, hostnames, ports published only in one setup, or + compose/VM paths belong in your own local notes, not here. (For example, a dev box may publish only FRP + `8782` and reach `8780` by container name; a portable setup publishes both.) +- **Prefer verifiable facts**: cite the file/class when it helps, and prefer "how to verify" over bare + assertions. +- **Update in the same change**: when AppAPI behavior changes, update the affected section here in the same PR, + and note the Nextcloud version if it is version-specific. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..e38531ff --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,12 @@ + + +# CLAUDE.md + +Guidance for AI agents working in this repo lives in [`AGENTS.md`](AGENTS.md), the tool-agnostic convention. +This file exists only so Claude Code, which reads `CLAUDE.md`, picks that guidance up automatically through the +import below. Keep the real content in `AGENTS.md`; do not duplicate it here. + +@AGENTS.md diff --git a/docs/appapi/aio.md b/docs/appapi/aio.md new file mode 100644 index 00000000..611ea3ee --- /dev/null +++ b/docs/appapi/aio.md @@ -0,0 +1,58 @@ + + +# ExApps on Nextcloud AIO + +A short runbook (a spoke of [`../../AGENTS.md`](../../AGENTS.md)). Read this when the Nextcloud instance is +**AIO (all-in-one)**. AIO manages the deploy daemon for you: most of the manual setup in the `AGENTS.md` +Quickstart does not apply, and fighting the managed daemon is the main failure mode. + +## What AIO automates + +AppAPI detects AIO via the `THIS_IS_AIO` env var and auto-registers the deploy daemon during app +install/upgrade (`lib/DeployActions/AIODockerActions.php`, `lib/Migration/DataInitializationStep.php`): + +- With AIO's **HaRP container** enabled (`HARP_ENABLED` + `HP_SHARED_KEY` set by AIO), AppAPI registers + **`harp_aio`** (`nextcloud-aio-harp:8780`, network `nextcloud-aio`, HaRP direct-connect mode) and makes it + the default. Exists on NC33+. +- With the legacy **Docker Socket Proxy** container, AppAPI registers **`docker_aio`** + (`nextcloud-aio-docker-socket-proxy:2375`). Deprecated since NC34, removal targeted for NC35. +- If both are enabled, `harp_aio` wins the default. Registration is idempotent and AppAPI never removes a + daemon it registered: after a DSP-to-HaRP migration, `docker_aio` stays behind; clean it up with + `occ app_api:daemon:unregister docker_aio` once no ExApps use it. +- AIO's own web server container routes `/exapps/` to HaRP internally, so the `AGENTS.md` Quickstart Step 4 + proxy rule is only needed on a reverse proxy **in front of** AIO, if you run one. + +## What the admin does + +1. In the AIO interface, enable the **HaRP community container** (or add `harp` to the + `AIO_COMMUNITY_CONTAINERS` env var of the mastercontainer), then restart the AIO containers so the env + reaches the Nextcloud container. +2. That is all for the daemon: check **Settings --> Administration --> AppAPI** shows the `AIO HaRP` daemon, + and use **Check connection** / **Test deploy**. +3. Install ExApps normally (UI store, or `occ app_api:app:register --wait-finish`; on AIO run occ as + `docker exec -u www-data nextcloud-aio-nextcloud php occ ...`). + +## What NOT to do on AIO + +- Do **not** run your own HaRP container or hand-register another HaRP daemon; `harp_aio` is AIO-managed + (a second daemon competes with it, and re-registering the same name is a no-op). +- Do **not** edit `harp_aio`'s host or shared key by hand; AIO owns `HP_SHARED_KEY` and a mismatch breaks + Nextcloud-to-HaRP auth. +- Do not re-default the deprecated `docker_aio`; migrate to HaRP and unregister it. + +## Troubleshooting + +- **"AppAPI default deploy daemon is not set"** on AIO: the HaRP community container is not enabled or not + running, so `HARP_ENABLED`/`HP_SHARED_KEY` never reached the Nextcloud container and auto-registration + skipped itself. Enable the container, restart AIO, and re-check. +- **HaRP enabled but no `harp_aio` daemon**: the Nextcloud container was not recreated after enabling HaRP, + so the env vars are missing inside it. Restart/recreate the AIO containers; the (idempotent) registration + re-runs on the next app_api install/upgrade repair step. +- **ExApp installs fine but its page is blank / WebSocket fails, only behind your own proxy**: the reverse + proxy in front of AIO does not forward `/exapps/` (with WebSocket upgrade). The internal checks pass anyway; + add the rule from `AGENTS.md` Quickstart Step 4 pointing at AIO. + +For everything else (lifecycle, troubleshooting, version notes) see [`../../AGENTS.md`](../../AGENTS.md). diff --git a/docs/appapi/exapp-contract.md b/docs/appapi/exapp-contract.md new file mode 100644 index 00000000..c70a5e51 --- /dev/null +++ b/docs/appapi/exapp-contract.md @@ -0,0 +1,147 @@ + + +# The ExApp manifest (``) reference + +A detailed runbook (a spoke of [`../../AGENTS.md`](../../AGENTS.md)). Read this when writing an ExApp's +`info.xml`, preparing a `--json-info` payload, or debugging why a route, env var, or mount did not behave as +expected. The runtime side (env vars AppAPI injects, init/heartbeat, auth headers) is in the hub's +"Runtime and the ExApp contract" section. + +Parsing lives in `ExAppService::getAppInfo()` (`lib/Service/ExAppService.php`); route validation in +`lib/Service/ExAppRouteHelper.php`. Invalid route definitions abort registration with a descriptive error. + +## The element tree + +An ExApp's `info.xml` is a normal Nextcloud app manifest plus an `` section: + +```xml + + + ghcr.io + nextcloud/my-exapp + latest + + + + + ^/api/.* + GET,POST + USER + [401,429] + ["Cookie"] + + + + + + MY_SETTING + My setting + Shown in the UI + some-value + + + + + + api + API + SERVICE_ROLE=api + true + + + +``` + +With `occ app_api:app:register --json-info`, the same keys are given as JSON (`docker-install`, `routes`, +`k8s-service-roles` may sit at the JSON root). Note the naming split: elements are hyphenated +(`docker-install`, `image-tag`, `display-name`), but route fields are underscored (`access_level`, +`bruteforce_protection`, `headers_to_exclude`). + +Things the manifest does **not** declare: + +- **Port and secret**: AppAPI assigns a free `APP_PORT` and generates `APP_SECRET` at registration. +- **API scopes**: removed from AppAPI; ExApps no longer declare scopes anywhere (`--force-scopes` is a + deprecated no-op). Do not add a scopes element. +- **Mounts**: there is no `` element (see below). + +## Routes: access levels and enforcement + +Every HTTP surface the ExApp exposes must be covered by a ``; unmatched requests are rejected. + +| Field | Meaning | +|---|---| +| `url` | Case-insensitive regex matched against the request path (e.g. `^/api/.*`) | +| `verb` | Comma-separated HTTP methods the route accepts | +| `access_level` | `PUBLIC` (0) anyone, `USER` (1) any logged-in user, `ADMIN` (2) admins only | +| `bruteforce_protection` | JSON array of response status codes that count as a bruteforce attempt (e.g. `[401,429]`) | +| `headers_to_exclude` | JSON array of request header names stripped before forwarding | + +Enforcement happens in two places, depending on the path a request takes: + +- **HaRP path** (browser to `/exapps/...`): AppAPI hands HaRP the route table (url + access_level + + bruteforce_protection) via the ExApp metadata endpoint, and HaRP resolves the caller's level per request + through the user-info endpoint (no/disabled user = PUBLIC, admin = ADMIN, else USER), enforcing the + comparison itself. `verb` and `headers_to_exclude` are not part of the HaRP checks. +- **PHP proxy path** (`/index.php/apps/app_api/proxy/...`): `ExAppProxyController` matches `url` (regex) and + `verb`, enforces the access level, strips `headers_to_exclude`, and applies bruteforce throttling on the + listed status codes. + +Authoring notes: an empty element (``) is fine and means "none", but +nested sub-elements (`401`) are rejected; use +the JSON-in-text form shown above. + +## Environment variables: a declared allow-list + +`` is an **allow-list** with defaults, not free-form input: + +- Each declared `` starts with `value = default`. +- `occ app_api:app:register --env NAME=VALUE` overrides **only declared names**; undeclared `--env` values are + **silently dropped**. If the manifest has no `` block at all, every `--env` is + ignored. +- Variables whose final value is an empty string are not passed to the container at all. +- The surviving set is stored and replayed on `app:update` (no need to repeat `--env`). + +So "my `--env` did nothing" almost always means the variable is not declared in the manifest. + +## Mounts: CLI-only, Docker-only + +There is **no** mounts element in the manifest. `occ app_api:app:register --mount SRC:DST[:ro|rw]` +(repeatable, default `rw`) is the only source of extra mounts, and every given mount is applied as a bind +mount on Docker daemons. (The `--mount` help text mentions a manifest declaration; no such gate exists in the +code today.) On **Kubernetes**, mounts are recorded with the deploy options but **not** mounted into pods; +persistent data goes through the PVC/`APP_PERSISTENT_STORAGE` instead. Mounts are replayed on `app:update`. + +## Kubernetes service roles + +For multi-process ExApps on Kubernetes, each `` becomes its **own Deployment** (same image, plus the +role's `` line). Only roles with `true` get a Kubernetes Service, and the **first** +exposed role is the app's single entry point for HaRP routing; other roles are internal-only. An ExApp without +roles gets one default Deployment. On Docker daemons the roles element is ignored. + +## Minimal working example (JSON form) + +The shape used by the project's own Kubernetes tests, as an `--json-info` payload: + +```json +{ + "id": "app-skeleton-python", + "name": "App Skeleton Python", + "version": "1.0.0", + "docker-install": { + "registry": "ghcr.io", + "image": "nextcloud/app-skeleton-python", + "image-tag": "latest" + }, + "k8s-service-roles": [ + {"name": "api", "env": "SERVICE_ROLE=api", "expose": true}, + {"name": "worker", "env": "SERVICE_ROLE=worker", "expose": false} + ] +} +``` + +A complete XML reference manifest lives in the `nextcloud/test-deploy` repository (the app the admin UI's +**Test deploy** button installs). For building the ExApp itself, see **nc_py_api** +(https://github.com/cloud-py-api/nc_py_api). diff --git a/docs/appapi/kubernetes.md b/docs/appapi/kubernetes.md new file mode 100644 index 00000000..7d8aa20e --- /dev/null +++ b/docs/appapi/kubernetes.md @@ -0,0 +1,226 @@ + + +# AppAPI on Kubernetes + +A detailed runbook (a spoke of [`../../AGENTS.md`](../../AGENTS.md)). Read this when the deploy daemon is +Kubernetes; for the common Docker path, concepts, and the full command references, stay in `AGENTS.md`. +Kubernetes support is **NC34+**. + +Placeholders in angle brackets are yours to fill; never paste a real secret or a real cluster address into a +committed file. + +## Contents + +1. [How it works](#how-it-works) +2. [Prerequisites](#prerequisites) +3. [Step 1: Namespace, ServiceAccount, token](#step-1-namespace-serviceaccount-token) +4. [Step 2: Run HaRP in Kubernetes mode](#step-2-run-harp-in-kubernetes-mode) +5. [Step 3: Confirm HaRP sees the cluster](#step-3-confirm-harp-sees-the-cluster) +6. [Step 4: Register the Kubernetes daemon](#step-4-register-the-kubernetes-daemon) +7. [Expose types](#expose-types) +8. [Step 5: Verify and install an ExApp](#step-5-verify-and-install-an-exapp) +9. [HaRP `HP_K8S_*` reference](#harp-hp_k8s_-reference) +10. [Troubleshooting](#troubleshooting) + +## How it works + +AppAPI never talks to the Kubernetes API. The `kubernetes-install` deploy backend (`lib/DeployActions/KubernetesActions.php`) +is a thin HTTP client: it POSTs high-level payloads (`create`, `start`, `stop`, `expose`, `remove`, ...) to +**HaRP** at `{protocol}://{host}/exapps/app_api/k8s/...`, and HaRP does all the cluster work. + +- **HaRP runs outside the cluster** as a container that reaches the Kubernetes API server over the network, + using a ServiceAccount bearer token. It is not deployed as an in-cluster workload. +- **HaRP creates the objects**: a Deployment, a Service (except for `manual` expose), a Pod, and a PVC per + ExApp, in a namespace HaRP manages (`nextcloud-exapps` in every reference setup). `start` scales the + Deployment to 1 replica, `stop` scales it to 0. +- **A Kubernetes daemon is always a HaRP daemon** (`--k8s` requires `--harp`). The AppAPI-to-HaRP shared key + is stored encrypted and sent as the `harp-shared-key` header, exactly as for a Docker HaRP daemon. +- **ExApps are always reached through HaRP routing** at `{nextcloud_url}/exapps/{appId}`, never at a direct + pod address. + +Because HaRP is off-cluster, how HaRP reaches each ExApp depends on the Service type you choose +(see [Expose types](#expose-types)); that is the main decision on Kubernetes. + +> The exact Deployment/Service/PVC object names, and the `HP_K8S_*` default values, are internal to HaRP and +> are not defined in the `app_api` repo (in practice the objects are named `nc-app-` with a +> `nc-app--data` PVC). The stable, verifiable anchors are the namespace and the label +> `app.kubernetes.io/component=exapp` on ExApp Deployments and Services. Facts here are drawn from `KubernetesActions.php`, +> the repo's `tests-deploy-k8s*.yml` workflows, and HaRP's `development/redeploy_host_k8s.sh`; HaRP's README +> has no Kubernetes section to cite. + +## Prerequisites + +- Nextcloud 34+ with admin access. +- A Kubernetes cluster and `kubectl` access to it. +- A host to run the HaRP container that can reach the cluster's API server, and can reach the ExApp Service + addresses for your chosen expose type (see [Expose types](#expose-types)). Running HaRP on a cluster node is + the simplest way to satisfy both. +- A namespace and a ServiceAccount token for HaRP (Step 1). + +## Step 1: Namespace, ServiceAccount, token + +HaRP authenticates to the API server as a namespaced ServiceAccount. The reference setups grant it the +built-in `cluster-admin` role for convenience: + +```bash +kubectl create namespace nextcloud-exapps +kubectl -n nextcloud-exapps create serviceaccount harp-sa +kubectl create clusterrolebinding harp-admin \ + --clusterrole=cluster-admin \ + --serviceaccount=nextcloud-exapps:harp-sa +# Mint a token; use a long duration for a persistent daemon. +TOKEN="$(kubectl -n nextcloud-exapps create token harp-sa --duration=8760h)" +``` + +The cluster may cap the token TTL (via the API server's `--service-account-max-token-expiration`); check the +issued token's expiry, or use a Secret-based ServiceAccount token if you need a long-lived daemon. + +> `cluster-admin` is broad. It is what the project's CI uses, and it is fine for a dedicated or test cluster. +> The project does **not** publish a least-privilege Role. To restrict HaRP, HaRP needs to manage, in the +> `nextcloud-exapps` namespace, at least: `deployments` (apps API group) and `services`, `pods`, `pods/log`, +> `persistentvolumeclaims`, `secrets`, `configmaps` (core API group), with `create`/`get`/`list`/`watch`/`update`/`delete`. +> Author a namespaced `Role` + `RoleBinding` granting those, then validate with a real ExApp deploy: if you see +> `forbidden` errors in the HaRP logs, add the missing resource/verb. This set is inferred from +> `KubernetesActions` behavior, not from a published manifest, so treat it as a starting point. + +## Step 2: Run HaRP in Kubernetes mode + +Run HaRP outside the cluster and point it at the API server. Beyond the normal `HP_SHARED_KEY` / +`NC_INSTANCE_URL`, set `HP_K8S_ENABLED`, `HP_K8S_API_SERVER`, `HP_K8S_BEARER_TOKEN`, `HP_K8S_NAMESPACE`, and +(for the self-signed certs typical of k3s/kind) `HP_K8S_VERIFY_SSL="false"`. + +```bash +docker run -d \ + --name appapi-harp -h appapi-harp \ + --restart unless-stopped \ + --network host \ + -e HP_SHARED_KEY="" \ + -e NC_INSTANCE_URL="" \ + -e HP_K8S_ENABLED="true" \ + -e HP_K8S_API_SERVER="https://:6443" \ + -e HP_K8S_BEARER_TOKEN="$TOKEN" \ + -e HP_K8S_NAMESPACE="nextcloud-exapps" \ + -e HP_K8S_VERIFY_SSL="false" \ + ghcr.io/nextcloud/nextcloud-appapi-harp:release +``` + +- `--network host` on a **real cluster node** (k3s/kubeadm) is the simplest choice: HaRP reaches the API + server (often `https://127.0.0.1:6443`) and, with the `clusterip` expose type, routes to cluster IPs via + kube-proxy. On **kind**, a remote cluster, or a managed cluster, HaRP is off-node: ClusterIPs are not + routable and the API server is elsewhere, so choose `nodeport` (or `manual`), not `clusterip` (see + [Expose types](#expose-types)). +- Find the API server URL with `kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'`. +- `HP_K8S_VERIFY_SSL="false"` is common for self-signed cluster certs (k3s/kind). Prefer verified TLS in + production and only disable it on a trusted network. +- If ExApp pods must resolve a hostname (for example your Nextcloud host) that cluster DNS does not know, add + `-e HP_K8S_HOST_ALIASES=":"`; HaRP injects it as pod `hostAliases`. + +## Step 3: Confirm HaRP sees the cluster + +Before registering, smoke-test that HaRP is up and its Kubernetes backend can reach the API server. HaRP's +`/exapps/app_api/info` reports a `kubernetes` block with `enabled` and `reachable`: + +```bash +curl -sf http://:8780/exapps/app_api/info \ + -H "harp-shared-key: " \ + | grep -Eq '"reachable" *: *true' && echo "HaRP K8s reachable" +``` + +The authoritative validation is the register-time daemon check (and the admin UI **Check connection**), which +enforces both `enabled` and `reachable`. If the curl or the daemon check fails, fix it now (see +[Troubleshooting](#troubleshooting)). + +## Step 4: Register the Kubernetes daemon + +Kubernetes daemons are managed via `occ` only; the admin UI shows them read-only ("managed via CLI"). +The positional order is `name display-name accepts-deploy-id protocol host nextcloud_url`. For Kubernetes use +`kubernetes-install`, protocol `http`, and HaRP's `:8780` HTTP frontend as `host`. `--k8s` requires `--harp` +and forces the deploy id to `kubernetes-install`; `--harp_frp_address` is not needed for Kubernetes. + +```bash +occ app_api:daemon:register \ + k8s1 "Kubernetes" kubernetes-install http :8780 \ + --harp --harp_shared_key "" \ + --k8s --k8s_expose_type clusterip \ + --set-default +``` + +- `host` (`:8780`) is the HaRP endpoint, not an ExApp address. Use `https` + HaRP's `:8781` only if + you run HaRP's HTTPS frontend with certs. +- `--harp_shared_key` must be byte-identical to HaRP's `HP_SHARED_KEY`. +- `` becomes each ExApp's `NEXTCLOUD_URL`; it must resolve from inside the + cluster. + +## Expose types + +`--k8s_expose_type` decides what Service HaRP creates and how HaRP (off-cluster) reaches the ExApp. This is the +core Kubernetes decision. AppAPI validates the flags; HaRP maps them to Service types. + +| Expose type | HaRP creates | HaRP reaches the ExApp via | Required/related flags | Choose when | +|---|---|---|---|---| +| `clusterip` (default) | ClusterIP Service | the Service `clusterIP` (needs HaRP able to route cluster IPs, e.g. via kube-proxy on a node) | none | HaRP runs on a node or where ClusterIPs are routable | +| `nodeport` | NodePort Service | a node address and the node port | `--k8s_node_port` (30000-32767), `--k8s_node_address_type` (`InternalIP` default / `ExternalIP`), `--k8s_external_traffic_policy` (`Cluster`/`Local`) | HaRP is off-cluster but can reach node IPs | +| `loadbalancer` | LoadBalancer Service | the load-balancer IP | `--k8s_load_balancer_ip`, `--k8s_external_traffic_policy` | the cluster has a LB provider (cloud, MetalLB) | +| `manual` | Deployment only, **no Service** | `--k8s_upstream_host` (you manage the Service/address) | `--k8s_upstream_host` (**required**) | you create and manage the Service yourself | + +Only the expose flags valid for the chosen type are accepted (for example `--k8s_node_port` is rejected unless +the type is `nodeport`; `manual` fails without `--k8s_upstream_host`). + +## Step 5: Verify and install an ExApp + +```bash +occ app_api:daemon:list # the k8s daemon is listed and default +occ app_api:app:register k8s1 --wait-finish # or omit the daemon to use the default +occ app_api:app:list # (): [enabled] +``` + +Inspect what HaRP created in the cluster (ExApp Deployments carry `app.kubernetes.io/component=exapp`): + +```bash +# ExApp Deployments and Services carry the component=exapp label: +kubectl -n nextcloud-exapps get deploy,svc -l app.kubernetes.io/component=exapp -o wide +# Pods and PVCs are not labelled that way; list them for the namespace: +kubectl -n nextcloud-exapps get pods,pvc -o wide +``` + +Note: on Kubernetes, ExApp `--mount` options are recorded but not forwarded to the container; persistent data +is handled by a HaRP-side PVC (exposed to the ExApp as `APP_PERSISTENT_STORAGE`). GPU ExApps still use +`--compute_device cuda|rocm` on the daemon. + +## HaRP `HP_K8S_*` reference + +Set on the HaRP container (Step 2). Defaults are not documented in any HaRP source; the values below are what +the reference setups use. + +| Env var | Meaning | Required | +|---|---|---| +| `HP_K8S_ENABLED` | Put HaRP into Kubernetes mode (otherwise Docker mode) | yes, `"true"` | +| `HP_K8S_API_SERVER` | Kubernetes API server URL HaRP calls | yes | +| `HP_K8S_BEARER_TOKEN` | ServiceAccount token HaRP authenticates with | yes | +| `HP_K8S_NAMESPACE` | Namespace HaRP creates/manages ExApp objects in | yes (`nextcloud-exapps`) | +| `HP_K8S_VERIFY_SSL` | Verify the API server TLS cert | optional (reference setups use `"false"`) | +| `HP_K8S_HOST_ALIASES` | `hostname:ip` entries injected as pod `hostAliases` | optional | + +## Troubleshooting + +The daemon check (`occ`, and the admin UI **Check connection**) surfaces HaRP's own Kubernetes status. The +common cases map directly to HaRP env: + +- **"HaRP version is too old and does not report Kubernetes support"**: update the HaRP image (`:release`). +- **"Kubernetes backend is disabled in HaRP"**: `HP_K8S_ENABLED` is not `true`. Set it and restart HaRP. +- **"HaRP cannot reach the Kubernetes API server"**: check `HP_K8S_API_SERVER` (reachable from the HaRP + container), the bearer token, and network/TLS (`HP_K8S_VERIFY_SSL`). +- **`forbidden` errors in HaRP logs during deploy**: the ServiceAccount token lacks permissions in the + namespace. Use `cluster-admin` (Step 1) or add the missing resource/verb to your Role. +- **ExApp Deployment exists but HaRP cannot reach the ExApp**: the expose type does not match your topology. + With `clusterip`, HaRP (off-cluster) cannot route to the ClusterIP unless it runs on a node or has routes; + switch to `nodeport`/`loadbalancer`, use `manual` with `--k8s_upstream_host`, or run HaRP on a node. +- **ExApp pod cannot resolve your Nextcloud host**: add it via `HP_K8S_HOST_ALIASES`, or ensure + `` uses a name cluster DNS can resolve. +- **Image pull fails in the cluster**: private registry or missing pull secret; configure registry access in + the namespace. + +For the daemon-register flag reference and general ExApp lifecycle, see [`../../AGENTS.md`](../../AGENTS.md). diff --git a/docs/appapi/remote-daemon.md b/docs/appapi/remote-daemon.md new file mode 100644 index 00000000..8ed607a8 --- /dev/null +++ b/docs/appapi/remote-daemon.md @@ -0,0 +1,160 @@ + + +# ExApps on a separate host (remote deploy daemon) + +A detailed runbook (a spoke of [`../../AGENTS.md`](../../AGENTS.md)). Read this when ExApp containers should run +on a **different machine** than Nextcloud, for example a GPU server for AI ExApps. For the single-host golden +path, concepts, and command references, stay in `AGENTS.md`. + +Placeholders in angle brackets are yours to fill; never commit a real secret. + +## Contents + +1. [Pick a topology](#pick-a-topology) +2. [Model A: HaRP near Nextcloud, remote Docker Engine over FRP](#model-a-harp-near-nextcloud-remote-docker-engine-over-frp) +3. [Model B: HaRP on the ExApp host](#model-b-harp-on-the-exapp-host) +4. [Verification](#verification) +5. [Troubleshooting](#troubleshooting) + +## Pick a topology + +Nextcloud never talks to a remote Docker Engine directly. Its Docker API calls go **through HaRP's HTTP +frontend**, with a `docker-engine-port` header selecting which engine: `24000` is HaRP's own local engine (the +mounted socket), and each additional engine connects over an FRP tunnel and gets a unique remote port in +`24001-24099` - that is what `--harp_docker_socket_port` selects (up to 99 engines per HaRP). + +| Model | Where HaRP runs | ExApp engine | Choose when | +|---|---|---|---| +| **A** | Next to Nextcloud | Remote; its Docker socket tunnels back to HaRP over FRP (mutual TLS) | One central HaRP, one or many ExApp hosts; the remote host only needs **outbound** access to HaRP | +| **B** | On the ExApp host | Local to HaRP (mounted socket) | One remote ExApp host, and Nextcloud can reach its ports `8780`/`8782` directly | + +Model A is the setup the HaRP README documents for external engines and scales to many hosts; Model B is +simply the single-host Quickstart placed on the remote machine. + +In both models the ExApps run on the remote host and their FRP clients dial back to HaRP's `8782`, so the +`nextcloud_url` positional (and HaRP's `NC_INSTANCE_URL`) must be reachable **from the remote host**; never +`localhost`. + +## Model A: HaRP near Nextcloud, remote Docker Engine over FRP + +**Step 1. Run HaRP next to Nextcloud** exactly as in the `AGENTS.md` Quickstart (Steps 2-4: shared key, +`docker run`, `/exapps/` reverse-proxy rule). `--harp_frp_address` (and HaRP's FRP port `8782`) must be +reachable from the remote host, so use a real hostname/IP, not a Docker-internal name. If HaRP should manage +**only** remote engines, you may omit the `/var/run/docker.sock` mount. + +**Step 2. Copy the FRP client certificates to the remote host.** HaRP generates mutual-TLS certs at +`/certs/frp` inside its container: + +```bash +mkdir -p harp_frpc_docker/certs/frp && cd harp_frpc_docker +for f in client.crt client.key ca.crt; do docker cp appapi-harp:/certs/frp/$f certs/frp/; done +# transfer the harp_frpc_docker folder to the remote host +``` + +**Step 3. Create `frpc.toml` on the remote host** (in that folder): + +```toml +serverAddr = "" # HaRP's address as reachable from this host +serverPort = 8782 +loginFailExit = false + +transport.tls.certFile = "certs/frp/client.crt" +transport.tls.keyFile = "certs/frp/client.key" +transport.tls.trustedCaFile = "certs/frp/ca.crt" +transport.tls.serverName = "harp.nc" # do not change + +metadatas.token = "" + +[[proxies]] +remotePort = 24001 # unique per engine, range 24001-24099 +name = "deploy-daemon-1" # unique per engine +type = "tcp" +[proxies.plugin] +type = "unix_domain_socket" +unixPath = "/var/run/docker.sock" +``` + +**Step 4. Run the FRP client on the remote host:** + +```bash +docker run -d --name harp_frpc_docker \ + --restart unless-stopped \ + -v "$(pwd)/frpc.toml:/etc/frpc.toml" \ + -v "$(pwd)/certs:/certs" \ + -v /var/run/docker.sock:/var/run/docker.sock \ + ghcr.io/fatedier/frpc:v0.61.1 "-c=/etc/frpc.toml" +``` + +**Step 5. Register the daemon** on Nextcloud. `host` stays HaRP's frontend; the remote engine is selected by +`--harp_docker_socket_port`. `--net` names a Docker network **on the remote engine** (`bridge` unless you +created one there): + +```bash +occ app_api:daemon:register \ + remote1 "Remote GPU host" docker-install http :8780 \ + --net bridge \ + --harp \ + --harp_frp_address :8782 \ + --harp_shared_key "$HP_SHARED_KEY" \ + --harp_docker_socket_port 24001 \ + --set-default +``` + +Add `--compute_device cuda|rocm` if the remote host has a GPU. + +> **Certificate lifetime.** The FRP certs are valid for `HP_FRP_CERT_VALIDITY_DAYS` days (default `5000`) and +> are **not renewed automatically**; an expired cert silently kills the tunnel. To renew: stop HaRP, delete its +> `/certs/frp` folder, start HaRP, re-copy the three files to every remote engine and restart each `frpc`. +> ExApps embed these certs at **install** time, so after regenerating you must remove and re-install each +> ExApp; a restart is not enough. + +## Model B: HaRP on the ExApp host + +Run HaRP on the remote machine exactly as in the `AGENTS.md` Quickstart Step 3 (it mounts that host's +`/var/run/docker.sock`; publish `8780` and `8782`), then register it from Nextcloud by address: + +```bash +occ app_api:daemon:register \ + remote1 "HaRP (ExApp host)" docker-install http :8780 \ + --net \ + --harp \ + --harp_frp_address :8782 \ + --harp_shared_key "$HP_SHARED_KEY" \ + --set-default +``` + +`--harp_docker_socket_port` stays at its default `24000` (HaRP's local engine). Your Nextcloud reverse proxy +must forward `/exapps/` to `http://:8780` (Quickstart Step 4), and Nextcloud must be able to +reach `8780` on the remote host; consider TLS (`https` + `8781` with certs) or a private network for that leg, +since ExApp traffic crosses hosts. + +## Verification + +```bash +occ app_api:daemon:list # shows the daemon with its frp_address and docker_socket_port +``` + +- Admin UI **Check connection** and **Test deploy** (Test deploy pulls and runs a real test container on the + remote engine, so it validates the tunnel + registry path end to end). +- Model A tunnel check, on the HaRP host: `docker exec appapi-harp curl -fsS http://127.0.0.1:24001/_ping` + (expects `OK` from the remote Docker Engine; use your `remotePort`). +- Install an ExApp, then on the **remote** host: `docker ps --filter name=nc_app_`. + +## Troubleshooting + +- **Check connection fails / tunnel dead (Model A)**: the remote `frpc` cannot reach `:8782` + (firewall/outbound), the certs were not copied, or the `metadatas.token` does not equal `HP_SHARED_KEY`. + Check the `frpc` container logs on the remote host and HaRP's logs. +- **`_ping` on the remote port fails but frpc is connected**: wrong `remotePort` in `frpc.toml` vs + `--harp_docker_socket_port`, or two engines claimed the same port. +- **Deploy works but the ExApp stays unhealthy**: `nextcloud_url`/`NC_INSTANCE_URL` is not reachable from the + remote host (DNS/firewall), or the ExApp's FRP client cannot reach `:8782`. +- **Everything worked for years, then all remote deploys break at once**: FRP cert expiry; see the certificate + lifetime note above. +- **Re-registering after a fix changes nothing**: `daemon:register` is a no-op for an existing name; + `daemon:unregister` first (see `AGENTS.md` Troubleshooting). + +For daemon flags and general lifecycle, see [`../../AGENTS.md`](../../AGENTS.md). diff --git a/lib/DeployActions/AIODockerActions.php b/lib/DeployActions/AIODockerActions.php index 909b6d09..49c022f9 100644 --- a/lib/DeployActions/AIODockerActions.php +++ b/lib/DeployActions/AIODockerActions.php @@ -32,7 +32,7 @@ public function __construct( } /** - * Detecting AIO instance by config setting or AIO_TOKEN env as fallback + * Detecting AIO instance by the THIS_IS_AIO environment variable */ public function isAIO(): bool { return filter_var(getenv('THIS_IS_AIO'), FILTER_VALIDATE_BOOL);