Proxy: stream HLS segments through instead of buffering
fragLoadTimeOut root cause: /stream buffered the whole segment (r.content) before sending, so the browser's time-to-first-byte = the CDN's entire upload time. On slow CDN moments that trips hls.js's first-byte timer even though the segment would have completed. - upstream.get_stream(): opens the upstream response without reading the body (stream=True), same retry/timeout rules as _get - segments now flow to the browser as the CDN emits them (256KB chunks); the 12s cap becomes a total-transfer cap; cache is populated on completion only (a truncated body never poisons the LRU) - playlists keep the buffered path (body needed for URI rewriting) - cache-hit fast path restored for both (stream() checks the LRU first) Verified: segment byte-identical to direct CDN (sha256), sequential keep-alive 3/3 OK, 5 concurrent fetches 200+identical, cache hit 0.148s miss -> 0.001s hit, 17/17 API tests green. Skipped: raising hls.js retry counts — the first-byte race was the binding constraint, not the retry budget.
This commit is contained in:
1 parent
27de918cb0
commit
19bc2afb32
2 files changed
+72
-22
No files matched your search
@@ -64,6 +64,25 @@ def _get(url, **kw):
|
||||
time.sleep(0.3)
|
||||
|
||||
|
||||
def get_stream(url, timeout=12):
|
||||
"""Segments: open the upstream response WITHOUT reading the body, so the
|
||||
proxy can forward bytes as the CDN emits them. (A buffered fetch made the
|
||||
browser's time-to-first-byte = the CDN's whole upload time, which tripped
|
||||
hls.js's fragLoadTimeOut on slow CDN moments.)
|
||||
ponytail: skips the 'Just a moment' body check — challenge pages arrive
|
||||
as 403/503 (handled below), and reading the body would defeat streaming."""
|
||||
for attempt in range(2):
|
||||
r = _session().get(url, **{**H, "timeout": timeout, "stream": True})
|
||||
if r.status_code in (403, 429, 503) or r.status_code >= 500:
|
||||
r.close()
|
||||
if attempt:
|
||||
raise RuntimeError(f"upstream {r.status_code}: {url}")
|
||||
time.sleep(0.3)
|
||||
continue
|
||||
r.raise_for_status()
|
||||
return r
|
||||
|
||||
|
||||
def _search(q):
|
||||
"""Step 1: browse page -> [{id, slug_id, name, poster}]"""
|
||||
html = _get(f"{BASE}/browse", params={"q": q}).text
|
||||
|
||||
Reference in new issue
Block a user