Proxy: 16 parallel lanes (was 4), like yt-dlp -N 16

Lane floor dropped 256KB -> 32KB so typical 0.35-2.2MB segments get the
full 16 lanes. Live check: 16 lanes used, 719KB in 0.55s, sha256
identical. Same fallback guarantees as before.
This commit is contained in:
jozamudi committed 2026-08-31 15:00:21 +00:00
1 parent ee2a888aca
commit 4c5dbab4b0
1 file changed
+4 -3
+4 -3
View File
@@ -83,15 +83,16 @@ def get_stream(url, timeout=12):
return r
def fetch_parallel(url, lanes=4, lane=256 * 1024, timeout=12):
"""One upstream file across N parallel byte-range lanes.
def fetch_parallel(url, lanes=16, lane=32 * 1024, timeout=12):
"""One upstream file across N parallel byte-range lanes (16, like yt-dlp's
-N 16 / ani-cli's download path).
Per-connection CDN throttling is the common slow-segment cause; N lanes
turn one 60s crawl into N concurrent crawls (I/O-bound, GIL irrelevant).
Returns (body, content-type); raises on ANY failure so the caller falls
back to the single-request path — never worse than before.
Verified 2026-08 against hls.anidb.app: HEAD -> content-length,
ranges -> 206, 4 lanes -> byte-identical stitch (sha256)."""
ranges -> 206, multi-lane -> byte-identical stitch (sha256)."""
from concurrent.futures import ThreadPoolExecutor
head = _session().head(url, **{**H, "timeout": timeout})
head.raise_for_status()