Veil

A web scraper that uses the lightest method that works

Open Source Developer Tools GitHub发布于 2026年6月23日

什么是 Veil？

Veil is an open-source Python web scraper that tries the cheapest method first and only escalates when a site blocks it so you're not running a full browser for every page. • 3 tiers: plain HTTP → TLS impersonation → stealth browser • Resumable concurrent crawler (checkpoint, JSONL, dedupe) • Proxy rotation, rate limiting, robots.txt by default • Honest README: documents what it can't beat Python, MIT, tested. A learning project built in the open.

截图

还没有评论，来抢沙发吧！