I don't see many people bothering to be honest, developers who are out there scraping are mostly just doing a job, they want to be done as quick as possible and most don't care about things like this unless there are repercussions
Being an "ethical web scraper" is about your own ethics, not abusing other peoples servers/data, and preserving the open internet for everyone.
Yes, if you slam someones website you will probably get the data you want but it will damage it for the collective. If everyone just slammed websites then:
1) the website owners will just get pissed off and either shut data behind logins or make it inaccessible. This is basically what happened with LinkedIn.
2) make it much harder and costlier to scrape by using more advanced anti-bots, requiring everyone to use more expensive residential proxies and headless browsers to get past them.
Web scraping can be a burden for websites, so everyone should approach it in as responsible and ethical way as possible.
I just feel like the whole thing is a chicken and egg problem - both sides could be nicer to the other but no one wants to go first!
I do like the idea of identifying web scrapers in the user agent but I'm not sure how many websites would use it...
