docs: sync README + SPEC with the shipped reality

Both docs still described the pre-provider-layer single-source app:

- README: anidb-only framing, missing stats page, 'upstream.py is the
  ONLY Cloudflare-facing code', and a self-check command that does not
  work (plain script execution breaks the providers' relative imports;
  the asserts are anidb-specific under the hianime-first default) —
  now documents ANI_SOURCE=anidb + the -m form. Dropped the stale
  'episode titles are not available' note (Jikan titles shipped).
- SPEC: 'no database' -> SQLite, architecture -> 3 providers, full
  endpoint list (intros/searches/events/stats/cdn, DELETE history,
  ?all=1, {sub}), auth hardening (per-IP budget, X-Aniweb on /login),
  tests/run.py -> pytest + ANIWEB_LIVE_TESTS gating, out-of-scope list
  (countdown/intro-skip/titles shipped), risks updated for the
  fallback layer, milestone 8 pointer.
This commit is contained in:
jozamudi committed 2026-09-16 21:14:26 +00:00
1 parent 4aca78cab8
commit 0f6bd0a52a
2 files changed
+110 -53

No files matched your search

+78 -32
View File
@@ -1,16 +1,20 @@
# ani-web — Web Front-End for ani-cli
Browse, search, and stream anime from the browser, using the same
anidb.app data source that ani-cli scrapes.
Browse, search, and stream anime from the browser. Sources are pluggable
providers (`server/providers/`): hianime.at primary, anidb.app (the
original ani-cli source) and anikototv.to as fallbacks.
**Status: implemented and running.** Deviations from the original
draft: hls.js is vendored locally (CDN was a silent failure point),
history is server-side with watch-position resume (was: localStorage,
last 20), and the front-end lives in `static/` with the server code in
`server/`. See README.md.
last 20), storage is SQLite (was: JSON files), the front-end lives in
`static/` with the server code in `server/`. See README.md.
## Verified data flow (spiked 2026-07, all 200 OK with Chrome TLS impersonation)
The anidb.app flow — one of the three providers (hianime.at is primary;
see `server/providers/hianime.py` for its flow).
| Step | Request | Yields |
|---|---|---|
| 1. Search | `GET anidb.app/browse?q={query}` | HTML cards: `href=".../anime/{slug}-{id}"`, `title="{name}"`, poster `img src` |
@@ -33,41 +37,70 @@ or segments directly — **all HLS traffic must go through a backend proxy**.
Browser (vanilla JS + hls.js)
│ /api/* JSON /stream/* proxied HLS
▼
Python backend (FastAPI, uvicorn)
Python backend (FastAPI, uvicorn, SQLite)
│ curl_cffi (Chrome impersonation)
▼
anidb.app + hls.anidb.app
hianime.at / anidb.app / anikototv.to + their CDNs
```
Two moving parts, no database, no framework on the frontend.
Two moving parts, SQLite (users/history/searches/events), no framework
on the frontend.
## Backend
`server/app.py` + `server/upstream.py`. Python 3.12, deps:
`fastapi`, `uvicorn`, `curl_cffi`, `itsdangerous`.
`server/app.py` (routes), `server/upstream.py` (shared HTTP, TTL
cache, provider dispatch), `server/providers/` (hianime, anidb,
anikoto), `server/db.py` (SQLite), `server/auth.py` (login throttling),
`server/users.py` (accounts). Python 3.12, deps: `fastapi`, `uvicorn`,
`curl_cffi`, `itsdangerous`.
Source dispatch: `ANI_SOURCE=hianime,anidb,anikoto` — ordered fallback;
an exception falls through to the next provider, a `None` from
`master_m3u8` ("no dub") is final.
- `GET /api/search?q=` → `[{id, slug_id, name, poster}]` (parse step 1)
- `GET /api/anime/{slug_id}` → `{id, name, synopsis, mal_id, seasons:[{slug_id,name,id}]}`
- `GET /api/anime/{id}/episodes` → `[{id, number, filler}]` (step 3, passthrough)
- `GET /api/episode/{ep_id}/stream?lang=eng|jpn` → `{token}`
(steps 5+6; token is base64url of the master m3u8 URL)
- `GET /api/history` / `POST /api/history` → **per-user** watch
- `GET /api/intros/{mal_id}` → per-episode MAL intro timings + episode
titles (keyless Jikan, 30-day cache; empty list when unavailable,
never an error that can touch the player)
- `GET /api/episode/{ep_id}/stream?lang=eng|jpn` → `{token, sub?}`
(steps 5+6; token is base64url of the master m3u8 URL; `sub` is the
external .vtt URL when the provider serves subs separately)
- `GET /api/history?all=1` / `POST /api/history` /
`DELETE /api/history?slug_id=&ep_id=` → **per-user** watch
history with position, persisted to
`server/data/aniweb.db` (SQLite; upsert per anime+episode).
`server/data/aniweb.db` (SQLite; upsert per anime+episode, newest 50).
- `GET /api/searches` / `POST /api/searches` → per-user recent
searches (newest 8, case-insensitive).
- `POST /api/events` → client self-diagnostics (playback failures,
step timelines, capability probes) + the position stream the stats
are derived from (90-day retention).
- `GET /api/stats?days=&slug=` → watch minutes, episodes
started/completed, active days + best streak, weekday/per-day
heatmaps, top-10 shows, sub/dub split — derived from the
position-event stream.
- `GET /api/cdn` → CDN + origin health (background 64KB probe every
3 min, ~1h history) — check whether "playback broken" is the CDN.
- `GET /stream/{token}` → proxied HLS. Token = base64url of the
upstream URL. Rewrites every non-comment line of m3u8 bodies to
point back at `/stream/`; proxies master, variants, and segments
(bytes, no transcoding). Responses are `Cache-Control: public,
max-age=86400` — VOD playlists/segments are immutable and the token
is the exact upstream URL, so the browser disk cache serves re-opens
and re-seeks without touching the proxy.
upstream URL (host-allowlisted against the providers' self-registered
CDN hosts — anti-SSRF). Rewrites every non-comment line of m3u8
bodies to point back at `/stream/`; proxies master, variants, and
segments (bytes, no transcoding), segments across parallel byte-range
lanes with single-stream fallback. Responses are `Cache-Control:
public, max-age=86400` — VOD playlists/segments are immutable and the
token is the exact upstream URL, so the browser disk cache serves
re-opens and re-seeks without touching the proxy.
- `GET /login` / `POST /login` / `POST /logout` → multi-user auth:
PBKDF2-SHA256 passwords (`server/users.py`; SQLite `aniweb.db`, 0600),
signed 30-day cookie carrying the username, 5-fail/15-min
in-memory lockout **per user**. Accounts are CLI-created only
(`python -m server.users add|rm|list`) — no self-signup. One-time
migration: legacy single-password `credentials.json` → user
`admin`; `history.json` → `history/admin.json`.
in-memory lockout **per user** plus a per-source-IP failure budget
(username rotation can't dodge it; XFF trusted only behind the
proxy). All mutations — including `POST /login` — require the
`X-Aniweb: 1` header (cross-site form POSTs can't set it). Accounts
are CLI-created only (`python -m server.users add|rm|list`) — no
self-signup. One-time migration (done): legacy single-password
`credentials.json` → user `admin`.
- Session hardening: cookies are `HttpOnly`, `SameSite=Lax`, and
`Secure` when `ANIWEB_SECURE=1` (set by `run.sh`); the middleware
re-verifies the account still exists per request, so deleted
@@ -82,11 +115,15 @@ Two moving parts, no database, no framework on the frontend.
produced black screens.
Flow: search box → results grid (poster + title, click) → detail view
(synopsis under the episode grid, season tabs) → episode grid (number
(synopsis and season tabs where the source provides them, under the
episode grid) → episode grid (number
+ filler badge, sub/dub toggle) → `<video>` with hls.js pointed at
`/stream/{token}`; single quality — the proxy fetches each segment
across parallel byte-range lanes (single-stream fallback).
Views: home (continue watching), search, detail+player, full history
(`?history`), stats (`?stats`); PWA-installable.
- Default play: opening an anime resumes the last watched episode of
the season (seeking to the saved position unless ≥95% through),
else autoplays episode 1. If the browser blocks autoplay, the native
@@ -103,22 +140,28 @@ uv venv .venv && uv pip install --python .venv/bin/python -r requirements.txt
```
Use the app through your HTTPS proxy (with `Secure` cookies, plain
remote HTTP can't hold a session). `python tests/run.py` runs the
full suite against a throwaway server on :8399.
remote HTTP can't hold a session). `.venv/bin/pytest tests/ -q` runs
the hermetic suite against a throwaway server on :8399; the
live-upstream browser e2e is opt-in (`ANIWEB_LIVE_TESTS=1`).
## Out of scope (v1)
Downloads (`-d`), next-episode countdown (`-N`), syncplay, intro skip,
ani-schedule integration, episode titles (not in anidb's API — needs
a MyAnimeList API key). Add when someone asks.
Downloads (`-d`), syncplay, ani-schedule integration. (Next-episode
countdown, intro skip, and episode titles — via keyless Jikan —
shipped since this list was written.) Add when someone asks.
## Risks
- **Cloudflare policy change** → upstream breaks. Mitigation: swap
`impersonate="chrome"` for `"safari"`/binary curl-impersonate;
all upstream access is isolated in one module (`upstream.py`).
- **anidb.app layout change** → parsers break. The 4 API endpoints
(steps 3–6) are clean JSON; only steps 1, 2, 5 are HTML/JS scraping.
all upstream access is isolated in `upstream.py` + `providers/`,
and the ordered multi-source fallback turns one dead site into a
degraded result set, not an outage.
- **Provider layout change** → parsers break. For anidb the 4 API
endpoints (steps 3–6) are clean JSON; only steps 1, 2, 5 are
HTML/JS scraping. hianime/anikoto are fully scraped (higher rotation
risk), so they self-register CDN hosts and re-roll servers until one
serves.
- **ToS** — anidb.app is a third-party scraper site; personal use,
no redistribution, bind to LAN only by default.
@@ -133,3 +176,6 @@ a MyAnimeList API key). Add when someone asks.
states, autoplay default, `run.sh`.
7. Auth — login screen, then multi-user CLI-created accounts with
per-user history + test suite.
8. Later (see git log, `docs/CHANGES-2026-09.md`): provider layer with
hianime primary, SQLite storage, stats/events, 2026-09 security
audit batch.