I made a opensource alternative for these services. Although these worked very well, I was not so confident what they do. So I made my own and opensourced it.
It is written in Golang and is fully customizable.
You mean like Bypass Paywall Clean?
https://gitlab.com/magnolia1234/bypass-paywalls-chrome-clean
javascript:location.href='https://archive.is/?run=1&url=%27+encodeURIComponent(documen...
It's a shame Google won't let this addon be in the store.
Edit : The Digital Millennium Copyright Act (DMCA) prohibits circumventing an effective technological means of control that restricts access to a copyrighted work. I guess that would apply here.
I remember some guy that wrote a WoW bot and got sued using the DMCA, with the argument that his bot was circumventing the anti-cheat and the anti-cheat could be seen as a 'mechanism protecting copyrighted material', because it was safeguarding access to the game servers, the servers were generating parts of the game world (such as sounds) dynamically, and those were under copyright... Wild stuff.
Or, looking at it the other way, if you put a small sticker that says "do not do X" and even one person follows that, isn't that therefore an "effective" method?
It doesn't if you're not in the US.
chrome and firefox extension for removing paywall: https://github.com/iamadamdev/bypass-paywalls-chrome
If you want an alternative that only requests permissions for sites with paywalls, this one is better: https://gitlab.com/magnolia1234/bypass-paywalls-firefox-clea...
I tried a Bloomberg article which gave me a "suspicious activity from your IP, please fill out this captcha" page, only the captcha was broken and didn't load.
Then I tried a WSJ article which loaded basically the same couple of paragraphs that I could get for free, but did not load any of the rest of the content.
javascript:window.location.href="https://archive.is/latest/"+location.href
It will usually open up the archived version of article without the paywall.
The ladder applies custom rules to inject code. It basically modifies the origin website to remove the Paywall. It rewrites (most of) the links and assets in the origins HTML to avoid CORS Errors by routing thru the local proxy.
The ladder uses Golangs fiber/fasthttp, which is significantly faster than Python (biased opinion) .
Several small features like basic auth ...
I have a feeling that this performance difference is practically imperceptible to regular humans. It's like optimizing CPU performance when the bottleneck is the database.
* I say "yet" because there could conceivably be ways to mitigate this, but afaik most would involve individual deals/contracts between every search engine & every subscription website - Google's monopoly simplifies this somewhat, but there's not much of an incentive from Google's perpsective to facilitate this at any scale.
Is it actually illegal anywhere to bypass a paywall?
The obvious thing is to mock Googlebot, but site owners can check that the request isn't coming from a Google-published IP and see that it's a fake, right?
> https://github.com/kubero-dev/ladder#environment-variables
> USER_AGENT User agent to emulate Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
> X_FORWARDED_FOR IP forwarder address 66.249.66.1
> RULESET URL to a ruleset file https://raw.githubusercontent.com/kubero-dev/ladder/main/rul... or /path/to/my/rules.yaml
just because they can doesn't mean they will... also most "site owners" are (by this point) a completely different people than "site operators" (who I take to be the 'engineers' who indeed can check this IP things)
But I feel a bit unconfident to let someone inject code to sites i view.
> Remove CORS headers from responses, assets, and images ...
I use txtify.it
One single downside was the intransparency. It was not clear which code was added or removed on the site you where looking at.
Wouldn't we always require a paid account to cache the HTML through (the SciHub model)?
Edit: Access to other projects & domains was apparently restored some time after: https://twitter.com/thmsmlr/status/1719480558932148272
I think it’s quite reasonable that they blocked the account rather than the project. You wouldn’t have got that level of service from big tech.
> Hey Thomas. Your paywall-bypassing site broke our ToS and created hundreds of hours of support time spent on all the outreach from the impacted businesses.
> Our support team reached out to you on Oct 14th to let you know this was unsustainable and to try to work with you.
“Other than the completely unreasonable thing, they seemed pretty reasonable”.
I mean he's not being a complete asshole in the discussion here[1], but nuking the entire customer's account for a ToS violation on one specific product still isn't a reasonable move. Yes Google and Amazon do it routinely, but if you're not a trillion dollar monopoly and you care about your business' reputation, you shouldn't behave like those.
[1] CEOs behaving like assholes in Twitter discussions isn't supposed to be the norm.
> Freedom of information is an essential pillar of democracy
However, this reads like this tool saves democracy by letting you bypass a crappy pay wall on a site you visit once a year, and that whoever wants to get paid for their published content online is an enemy of democracy.
Maybe even better: magnolia outperforms archive.is on paywalls
Does anyone have any insight into how it would take Vercel hundreds of hours of support time? https://twitter.com/rauchg/status/1718680650067460138
Access to private property is an essential pillar of democracy and the safe proliferation of ideas. While property owners have legitimate financial interests, it is crucial to strike a balance between property and the public's right to access property. The proliferation of locks on doors raises concerns about the erosion of this fundamental freedom, and it is imperative for society to find innovative ways to preserve access to people's homes and workspaces without compromising the sustainability of property ownership.. In a world where property should be shared and not commodified, locks should be critically examined to ensure that they do not undermine the principles of an open and informed society.
That said, I work in news media (and have been involved in building paywalls at different orgs - NYT and New Yorker). I know how money for these directly support journalism - salaries and the costs with associated with any story.
If you are skipping paywalls a lot, I would encourage you to pay for a subscription to at least one or two news sites you respect - bonus points if its a small or medium local newsroom that benefits!
For me that has been; NYTimes, New Yorker, Wired, Teen Vogue, and my wife's hometown paper in Illinois.
I do wish there was a better way for me to share an account across multiple news sites that let me properly pay for good journalism without these issues. I do subscribe to a very local news source that seems to handle this a lot better, but they also don't paywall (most) of their primary content.
In the meantime I do find it strange that so many sites wish to gain the advantage of advertising that they have put up an article on the web, without actually providing that article. I have no issue with paid content, but when that content gets listed in search engine results and social media links like a web page, but clicking on it does not behave like a web page, It feels something feels like something has broken from the idea of the linkable World Wide Web.
Instead I just don't pay anyone, turn back when I encounter a paywall and look for someone's summary if I'm really interested.
The problem creating such a service is that most media houses believe that their content is the best thing since sliced bread and thus they often don't want to partner. Even though most of their content isn't that unique. Of course, some publications do have unique content, e.g. nyt, bloomberg.
I could see artifact being an interesting company to tackle this though (https://artifact.news/). They are already sending traffic to news sites and only serving what the user wants. If they now let me bypass paywalls for $20 that would be nice.
Edit: apparently it is down now.
402: PAYMENT_REQUIRED Code: DEPLOYMENT_DISABLED ID: fra1::8wkv2-1699275385535-39dedae23d6a
To remove paywalls 12ft<Archive.org<Archive.today is my opinion.
It was just an effective way to get through substack/medium in my experience.
> Freedom of information is an essential pillar of democracy and informed decision-making. While media organizations have legitimate financial interests, it is crucial to strike a balance between profitability and the public's right to access information. The proliferation of paywalls raises concerns about the erosion of this fundamental freedom, and it is imperative for society to find innovative ways to preserve access to vital information without compromising the sustainability of journalism.
