Crawl4AI (opens in new tab)

(docs.crawl4ai.com)

2 pointsbjornroberg3d ago1 comments

1 comments

Promising foundation if you’re willing to own the policy layer + quality gates. Also worth calling out the boring production bits: robots/ToS, rate limiting, bot mitigation, login/session handling, and not accidentally hoovering up PII.

j / k navigate · click thread line to collapse

Crawl4AI (opens in new tab)

(docs.crawl4ai.com)

2 pointsbjornroberg3d ago1 comments

1 comments

bjornrobergOP3d ago

Promising foundation if you’re willing to own the policy layer + quality gates. Also worth calling out the boring production bits: robots/ToS, rate limiting, bot mitigation, login/session handling, and not accidentally hoovering up PII.

j / k navigate · click thread line to collapse