From d02cb017d302afafef5a60a06f2b18958b610cfa Mon Sep 17 00:00:00 2001 From: Josue Zamudio Date: Wed, 16 Sep 2026 02:41:25 +0000 Subject: [PATCH] gate live-upstream browser e2e behind ANIWEB_LIVE_TESTS=1: provider rotation/takedowns were turning the suite red for non-regressions; default suite is hermetic (33 tests) again --- README.md | 19 ++++++++++++------- docs/CHANGES-2026-09.md | 8 ++++++++ tests/test_browser.py | 11 ++++++++++- 3 files changed, 30 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 5b9e6ae..929f8ca 100644 --- a/README.md +++ b/README.md @@ -131,17 +131,22 @@ SPEC.md design + the verified anidb.app data flow state flips to `server.log`; `GET /api/cdn` (public) returns the latest throughput + ~1h history. Use it to check if "playback broken" is the CDN again. -- The front-end is version-pinned: bump `?v=N` on `app.js`/`style.css` - in `index.html` on every static change — phones/proxies otherwise - serve stale JS (index.html itself is `no-store`). +- The front-end is version-pinned: the server injects the git SHA as the + `?v=` cache-bust on `app.js`/`style.css` at startup (index.html itself + is `no-store`), so a restart is a cache bump — nothing to hand-bump. +- Mutations need the `X-Aniweb: 1` header (CSRF: cross-site form POSTs + can't set it), `/login` POST included — forced-login is dead. +- Login throttling (server/auth.py): per-username 5 fails -> 15 min lock, + plus a per-source-IP budget; both sliding windows, in-memory. - Playback failures self-report: the client posts a step timeline + capability fingerprint to `/api/events` (see the `events` table), so a broken browser is diagnosable from the server. -- `.venv/bin/pytest tests/ -q` runs the full test suite against a +- `.venv/bin/pytest tests/ -q` runs the test suite against a throwaway server (:8399, temp data dir — never touches the real - data). The API tests use stdlib only; the browser tests need - playwright (`uv pip install playwright && playwright install - chromium`) and are skipped if absent. + data). The default suite (API + auth + dispatch tests) is hermetic + and deterministic; the Playwright browser tests scrape and play LIVE + upstream content, so they are opt-in: `ANIWEB_LIVE_TESTS=1 + .venv/bin/pytest tests/ -q` (needs playwright chromium installed). - Episode titles are not available: anidb's episode API only returns `{id, number, number2, filler}`. Adding titles needs a free MyAnimeList API key. diff --git a/docs/CHANGES-2026-09.md b/docs/CHANGES-2026-09.md index 65d7baa..fc48ca7 100644 --- a/docs/CHANGES-2026-09.md +++ b/docs/CHANGES-2026-09.md @@ -42,3 +42,11 @@ mutation, and login.html sends the header. `curl_cffi` version churn = impersonation churn = silent upstream 403 waves. requirements.txt becomes a full freeze; bump deliberately. + +## 6. live-upstream e2e gated off by default + +The Playwright tests scrape and play real provider content — they go red +whenever a provider rotates or a title is taken down (observed repeatedly). +A suite that fails for non-regressions can't gate anything, so the default +run is now hermetic only; run the live e2e deliberately with +`ANIWEB_LIVE_TESTS=1`. diff --git a/tests/test_browser.py b/tests/test_browser.py index 3c055d8..ccd7e5e 100644 --- a/tests/test_browser.py +++ b/tests/test_browser.py @@ -2,15 +2,24 @@ Run via pytest (see conftest.py for the throwaway server). Covers what HTTP tests can't: the login UI, the user chip, and real video playback -behind auth. Skipped if playwright is not installed. +behind auth. + +These tests scrape and play LIVE upstream content, so they fail whenever a +provider is mid-rotation or a title is taken down — that is not a code +regression. They therefore stay OFF by default (the default suite must be +deterministic); run them deliberately with ANIWEB_LIVE_TESTS=1. """ import asyncio import http.cookiejar import json +import os import urllib.request import pytest +if os.environ.get("ANIWEB_LIVE_TESTS") != "1": + pytest.skip("live-upstream e2e — run with ANIWEB_LIVE_TESTS=1", allow_module_level=True) + async_playwright = pytest.importorskip("playwright.async_api").async_playwright BASE = "http://127.0.0.1:8399"