# POC server cutover — bare-metal Apache, self-applying

The repo deploys as one unit to the web root, so the layout is:

```
https://<host>/                         → portal hub        (Next, 127.0.0.1:3000)
https://<host>/product-ai-automation/   → product-ai front  (Next, 127.0.0.1:3001)
                                          → its Symfony backend (php8.3-fpm via Apache 127.0.0.1:8000, internal)
```

Target is a dedicated **Ubuntu 24.04** box (PHP 8.3 is native) at **34.65.188.113**. After the
one-time prerequisites below, **merging an MR is all it takes** — the CI runs
[`server-setup.sh`](server-setup.sh) on the box after rsync.

---

## Automatic on every deploy (CI → `server-setup.sh`)

1. Build the portal and the product-ai frontend (latter with `BASE_PATH=/product-ai-automation`).
2. Backend: `composer install` (run under **php8.3**), `cache:clear`, schema create/update, `app:seed`.
3. Symlink secrets from `data/secrets/`; ensure `data/` dirs.
4. Render + install the systemd units + the Apache vhosts; enable required mods.
5. Restart the web tier (portal, product-ai-frontend, php8.3-fpm) behind a health gate, then
   `apache2ctl configtest` + reload. A failed build/health-check aborts **before** touching the
   running site or Apache.

The **worker/poller** units are installed but **not started** (real Vertex cost).

---

## One-time prerequisites (per box)

This box is an upgraded clone, so Apache, the TLS certs, the CI SSH key, and sshd on :1338 all
came along with it. You mainly need the runtimes the POC adds.

**1. Runtimes** (PHP 8.3 itself is native on 24.04 — just add the extensions + Node + Composer):
```bash
apt-get install -y \
  php8.3-fpm php8.3-cli php8.3-common php8.3-opcache \
  php8.3-sqlite3 php8.3-gd php8.3-intl php8.3-mbstring php8.3-xml php8.3-curl php8.3-zip
curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && apt-get install -y nodejs
apt-get install -y composer
```

**2. Secrets** — kept in the rsync-excluded `data/secrets/` (server-setup symlinks them to each
app's `.env.local`, which Next + Symfony auto-load):
```bash
mkdir -p /srv/www/digt/<host>/data/secrets && cd "$_"
# portal.env               : AUTH_SECRET, HUB_ADMIN_EMAIL, HUB_ADMIN_PASSWORD [, GOOGLE_CLIENT_ID/SECRET]
# product-ai-frontend.env  : ADMIN_API_TOKEN, AUTH_SECRET [, GOOGLE_*, ADMIN_ALLOWED_EMAILS, ALLOW_DEV_LOGIN=true]
# product-ai-backend.env   : APP_ENV=prod, APP_SECRET, ADMIN_API_TOKEN (same as frontend) [, VERTEX_*]
chmod 600 *.env
```

**3. TLS** — the cloned box should already have `/etc/letsencrypt/live/<host>/`. If not, obtain it
*without* letting certbot touch the vhost (server-setup owns the vhost):
```bash
certbot certonly --webroot -w /var/www/html -d <host>      # or --standalone
```

**4. Retire the old vhost** — whatever currently serves `<host>` at the web root must be disabled,
or Apache keeps serving it (same ServerName) instead of the new `poc-<host>.conf`:
```bash
apache2ctl -S | grep <host>          # find the current vhost file
a2dissite <old-site> && systemctl reload apache2
```

**5. CI access** — cloned, so the CI key is already in `/root/.ssh/authorized_keys` and sshd is on
:1338. The pipeline IP is already updated to `34.65.188.113`.

---

## First cutover — test on the box before moving DNS

1. Do the prerequisites above.
2. Either merge `feat/centralized-data` → `stage`, or run it by hand once to shake out box
   specifics: `bash /srv/www/digt/<host>/deploy/server-setup.sh <host>`.
3. **Test before DNS** — point your laptop's `/etc/hosts` (or `curl --resolve`) at
   `34.65.188.113` for `<host>`, then check `/` (portal login) and `/product-ai-automation/`.
   You can also smoke-test the upstreams directly on the box: `curl localhost:3000/login`,
   `curl localhost:3001/product-ai-automation/`, `curl localhost:8000/api/health`.
4. When it's good: sync the live data and flip DNS (below).

---

## Final data migration + DNS flip

Everything but data is already on the box (code via CI, config via the clone). The only cutover
step is the **data delta**:
```bash
# from the OLD box, sync the centralized data dir to the new one:
rsync -avz -e 'ssh -p 1338' /srv/www/digt/<host>/data/  root@34.65.188.113:/srv/www/digt/<host>/data/
```
Then point `<host>` DNS at `34.65.188.113`. (`data/` is rsync-excluded by the deploy, so deploys
never overwrite it.)

> If the **legacy** app also lands on this box, run it on **PHP 7.4 from the ondrej PPA** (it
> supports noble) via a per-vhost `SetHandler …php7.4-fpm…`; the POC stays on native **8.3**. They
> coexist cleanly through per-vhost FPM handlers.

---

## Turning on processing (REAL Vertex cost)
```bash
systemctl enable --now poc-product-ai-worker poc-product-ai-poller
```
Leave both off until the Vertex creds are set and you intend to process.

## Auth model
**Start-simple:** each app gates itself — the portal enforces its own login wall and
product-ai-automation has its own login; Apache just reverse-proxies. To enforce a single hard
wall at the edge later (Apache has no nginx-style `auth_request`), route everything through the hub
and have it proxy the POC subpaths.

## Rollback
`server-setup.sh` backs the prior vhost up to `…/sites-available/poc-<host>.conf.bak.*`:
```bash
cp /etc/apache2/sites-available/poc-<host>.conf.bak.<ts> /etc/apache2/sites-available/poc-<host>.conf
apache2ctl configtest && systemctl reload apache2
# or just re-enable the old vhost:  a2ensite <old-site> && systemctl reload apache2
```

## Troubleshooting
| Symptom | Check |
|---|---|
| Pipeline red at the `server-setup.sh` step | The job log echoes each phase — usually a missing package, a missing `data/secrets/*` file, or `apache2ctl configtest` (often the TLS cert). |
| 502 at `/` | `systemctl status poc-portal` · `journalctl -u poc-portal -n50` |
| 502 at `/product-ai-automation/` | `systemctl status poc-product-ai-frontend` · was it built with `BASE_PATH`? |
| Backend 500s | `journalctl -u php8.3-fpm` · `data/secrets/product-ai-backend.env` present? · `php8.3 -m \| grep -E 'sqlite3\|gd'` |
| `apache2ctl configtest` fails | TLS cert for `<host>` missing → `certbot certonly`; or an old vhost still claims the ServerName |
