undefined | Better HN

0 pointsnurettin6mo ago0 comments

I worked for an Amazon scraping business and they used Luminati (Now Brightdata) for a few months until I figured out a way to avoid the ban hammer and got rid of their proxy.

They indeed provided "high quality" residential and cellular ips and "normal quality" data center ips. You had to keep cycling the ip pool every 2-3 days which cost extra. It felt super shady. It isn't their bots, they lease connections to whoever is paying, and they don't care what people do in there.

0 comments

yomismoaqui6mo ago

> ... until I figured out a way to avoid the ban hammer ...

You had my curiosity ... but now you have my attention.

walletdrainer6mo ago

Without bothering to check on Amazon, I successfully scraped meta stuff for years at rates exceeding 20gbit/s without any proxies but just rotating IPv6 addresses on the same couple of blocks for every request

There are usually silly bypasses like this that easily work even with bigco stuff

j / k navigate · click thread line to collapse