1) malware is often very aggressive and fast-spreading, and once it's on a user's computer it's hard to get off, therefore...
2) the system to detect it and stop access to the site has to be automated, not a human-in-the-loop system that might take hours or days to shut off access to a site which is infecting many users per minute, and...
3) the more clarity there is on how exactly that automated system works, the more certain we can be that malware will be able to evade it; it's much like how spam detection or search page rankings are opaque, because the incentives to game the system are very great
I'm not saying Google's system is perfect, but I am saying it's a very hard problem to solve in a way that doesn't give us an even worse time stopping malware spread than we already have. So while it is hard to feel sorry for a company as wealthy and powerful as Google, I think the issue is not as clear-cut as some comments on this thread seem to suggest.
But that being said, by far the biggest problem is just the lack of recourse and communication. Compare this to email spam prevention and the like, which solves a very similar problem but if you accidentally get blacklisted you can just talk to the SpamHaus people or whatnot and get the problem sorted.
It's not hard to imagine how Google could improve here: send better notifications when something is blacklisted, provide a reason why, and offer a better procedure to get your problem fixed.
Yes, this will cost time and money due to the large scale of things. But if you have the ability to block parts of the internet for much of the population then you also have some responsibility here; you can quite literally kill companies with this. Email spam prevention usually step up to this responsibility. Google ... not so much.
Mistakes will still happen, and that's okay. I appreciate the hard job they're doing, which does provide a lot of value. It's how you deal with those mistakes that matters, and Google deals with them terribly across all of their products.
> One common criticism is that there was no way to contact SPEWS. According to the SPEWS FAQ: "Q41: How does one contact SPEWS? A41: One does not..."
https://en.wikipedia.org/wiki/Spam_Prevention_Early_Warning_...
It also doesn't seem like sites like Facebook, Reddit, Youtube, Google photos, etc. run into this problem, even though they allow user uploaded content so there is some kind of bias against smaller companies.
PS. Twitter still does not allow me to share links to OP's website.
I was surprised this wasn't part of the lessons learned. But it seems the monitoring basically failed but that wasn't a lesson.
I feel like majority of uptime monitors are falling for this same trap. One of the reasons why for my monitoring service I choose to do full page load monitoring via Chrome instead of just a http request via Curl or whatever. Main reason, people care if the webpage loads or not. People care how long it takes for their webpage to load. Having a website respond in 200ms is great but if it takes 8000ms for all the JS to load and process your website is still slow. I get why sites are just doing curl requests because it's way cheaper but really you're monitoring one part of the stack while really caring about all of it. If your website starts producing javascript errors you want to know, etc.
[1] https://www.ootliers.com (The landing page and everything are terrible and I'm working on improving that)
For example:
- a frequent/simple check dealing directly (on the internal network) with the webserver ("does it work well yes/no, what's the raw response time, etc..."). Here is where I would definitely use "curl".
- another less frequent test involving as well the DNS and the external network.
- another end-to-end test (e.g. once every 10 minutes?) involving as well one or more real browsers (this would test as well for example revoked SSL certs).
=> all these infos/metrics should be quite helpful to identify problems, or at least to shrink the potential area that is causing it.
The main offer is for order monitoring but I am in the middle of creating a just page load monitoring offer for others since I think that service by itself is super useful.
But the key thing for me is the "goal" montioring. So for ecommerce it's orders but for other systems it's different. That's the thing I really want to monitor. If other things break they'll affect those so you can detect lots of failures. The only issue is finding out what the cause is. But first I'll improve the anomaly detection a bunch before looking into root cause detection.
To be fair, I've not had this happen yet so I am going to try and find a site that chrome won't let me visit and see what happens when I visit it programmatically.
Why would they flag that as a phishing site?
I’m just having a difficult time determining how this situation is not the fault of the site/app; we don’t even know that any of this is true and it looks more scripted than an offended rant.
They could use Google's safe browsing api to check if they're on that list as well as curl.
One the other hand it seems, based on this and many other posts, that there isn't much communication from Google to its "clients" to 1) explain what's wrong and 2) quickly/directly ask for a reevaluation (e.g. after the problem has been fixed, to question the validity of the problem, etc)?
I understand that there might be bad actors around doing everything on purpose on their website/app and that therefore #1 (basically telling the bad people why they got detected) would be a bit of a gray zone, but at least #2 should be a no-brainer (e.g. in the case of the previous ".ass"-files-case anybody in any support desk could have immediately whitelisted that "problem")?
Then that's a problem with the rules, which need clarified to better encode the "spirit" thereof. Hiding the rules entirely is a poor substitute for that.
What are your saying?
Google should have been regulated years ago, instead, they have been allowed to snap up every smaller company to solidify their position in the market and ensure they and only they are allowed positions of power, control and authority.
If Google dislikes you (or their baseless algorithms that are detached from reality) then you are toast. How long before Google's algorithm results in an actual human death? Doesn't seem totally far fetched and entirely plausible.
Yet, you let this happen, or rather, it seems this isn't concerning enough for it to warrant a massive protest, after all, Big Tech controls protest online and can just shut it down. Amazon seems to have been mightily effective at stopping any "union" movement, so we know the censor machines are fine tuned and ready to fire at any moment.
We need to be talking about this daily, in needs to be front and center for weeks and weeks, and we need to demand accountability. We are ruled and governed not by elected officials but by faceless, nameless and non-human machines. They do not Think. They do not Talk. They do not care.
Yet this thread will disappear in a few short hours, and this will be just another episode of the weekly "Google's systems are out of control and one developer got caught out, too bad I hope they are okay".
This is happening to thousands of others undoubtedly that do not make hackernews or have the resources/energy to fix it.
We should demand better.
Of course they know. Everybody knows, it's just a series of tubes.
But that's not the point. The people in charge also know:
> If Google dislikes you (or their baseless algorithms that are detached from reality) then you are toast.
Replace here Google with FAANG, and see how whole countries are completely depended on those companies. At this point those companies can blackmail any government on earth into almost anything they want. FAANG are actually even richer than most countries on this planet.
I think if FAANG didn’t already control so much of our communications you might see such advocacy groups, but as it is...
Do you want to be the face of a campaign that will piss off FAANG?
We are getting stories like this on a weekly basis now.
Google is clearly causing measurable harm to your company and you. And apparently to thousands before you.
Considering how much money patent trolls manage to extract from Big Tech with considerably weaker cases, how is it that everybody is treating Google like a fragile grandmother with dementia, going out of their way not to hold them responsible in court?
This is not a rhetorical question. I really don't get it.
America is the land of getting millions in settlement when McDonald's gives you coffee that is hotter than you anticipated. How the hell is Google getting away with their behavior?
The coffee was not merely "hotter than you anticipated" (although that's at least sort of right), it was near boiling: McDonald's required franchisees to hold coffee at 180–190 °F, much closer to actual boiling than what other establishments hold coffee at, which is typically twenty degrees below that in that area. She had third degree burns on six percent of her body, six rather sensitive percent. She needed an eight day hospital stay just for skin grafts. I once dug up the photos, by the way, they're rather unpleasant, and I say that as someone who has attended autopsies.
Of course, the temperature differences may not seem like much, but a ten degree drop at that point changes the time from "skin graft city" from three seconds to perhaps four or five times that.
Final verdict, before settlement, was $640,000, not "millions." The parties settled out of court for an undisclosed final amount less than $600,000.
1. publishers want to be able to put content on the Web without undergoing background checks
2. everyone wants to be able to discover content with as little friction as possible
3. consumers don’t want to drown in unwanted crap
The incomprehensible Algorithm is the result of trying to square that circle. Give up any of those requirements, and the arms race would end:
Give up #1, and it’ll be possible to do all of the rules enforcement reactively, with no algorithms and no inhumane call centers, because when someone is banned, they’ll stay banned. The ban will be tied to a legal name and anyone caught ban-dodging can be sued.
Give up #2, and it won’t matter how much spam you make available on the web because nobody will fall victim to it. The web becomes less like a publishing platform and more P2P, because you basically only find content on there through your in-person social contacts.
Give up #3, and you don’t need Safe Browsing any more. Good luck selling that to everyone, though.
In order to sue them, you need to come up with something that they should’ve done but didn’t. Having a human review every web page that’s ever published is obviously dumb, so they’re going to have to go with the algorithmic approach.
This doesn't actually work because the people doing bad stuff are criminals with no qualms about committing crimes, like identity theft. Some large fraction of spam is sent from compromised but otherwise legitimate mail servers.
> Give up #2, and it won’t matter how much spam you make available on the web because nobody will fall victim to it. The web becomes less like a publishing platform and more P2P, because you basically only find content on there through your in-person social contacts.
This is the one you can actually fix because it's a spectrum rather than binary. It's also something that doesn't need to be a monopoly, and not being a monopoly would significantly reduce the consequences of mistakes.
Discovery is also fundamentally a search issue. Not putting something you suspect of being spam in the first page of your search results is a world away from shutting down some guilty until proven innocent third party's DNS or hosting.
How so? If you sue for damages, you only have to prove you were harmed by Google's actions, no? And actively misrepresenting your website as dangerous and deceptive to your customers is sort of libelous and clearly damaging.
Currently they have all the benefits of their monopoly with none of the responsibility which is exactly the way they like it.
They have enough money to influence the USA government if anything changing the situation were to be introduced.
It's unlikely that any claimant would be able to show a contractual provision that enables them to claim for damages against Google (thus allowing them to sue in contract), so a cause of action for tort would be the usual way to sue Google - except unless Google makes you suffer some form of physical harm or damages your property, you're unlikely to be able to recover any damages for your website suffering these consequences, in the UK at least. I understand US law may be quite different.
There's a testable argument to be made about the requirement for "damage" to your property (the website) being inflicted by the certificate warning, but policy arguments on the matter of "ripple effect" liability makes it seem likely the courts would hold that Google isn't liable.
Also Google is probably far better placed to weather lawsuits than most ordinary people; they can probably afford to induce the other party to settle out of court, and presumably the relevant monopoly and abuse of market position laws only allow a regulator to take legal action (the ordinary consumer being restricted to contract and tort lawsuits).
I'm probably misunderstanding your argument here, but if, say, Google steals your bike that would be purely economic damage. Surely the UK legal system would still punish that...!?
Yeah, it's a really good question. We got all these fully staffed insanely rich companies causing measurable harm to people. They just insist there's nothing they can do to stop it. Why does everyone believe them?
Users won't sue as long as there's no meaningful harm to the users. And there's essentially no meaningful harm to the users by dropping a single site. As a user, I don't care if any particular site hosts a malware and gets blocked - that's what I want. If that site gets back slowly, I don't care either. That's the website owner's loss.
Government doesn't have standing to sue, as long as there's no discriminatory effect - and as long as the selection criteria is fair (malware/phishing), and they are not negligent in fixing false positives, government will have hard time finding a leg to sue.
It comes down to - as a society, safe browsing APIs are critically important and they have been working reasonably well. You'll have to show they are mismanaging, or malicious, or doing damages to the users. There's no evidence for any of those.
Regulatory Capture.
The dividing line between big tech and big gov is far thinner than most people consider.
Mcdonald's burned off a woman's labia after burning the flesh of several people with coffee tens of degrees hotter than is safe, and then refused to simply pay her medical bills, prompting a lawsuit.
Has Google burned your labia?
They interfered with the contract OP has with their customers.
That’s some dumb shit right there.
Help HN: Google just blocked my site as deceptive site - https://news.ycombinator.com/item?id=26326528 - March 2021 (20 comments)
I just looked over the site a little more. The business idea seems to be to have a widget to add to your site that can be used to upload arbitrary files to it. The real advantage looks to be that they have a bunch of integrations set up with Facebook, GDrive, Dropbox, Instagram, etc so that all just works without you having to set up and manage developer accounts with 10 different services. Plus built-in image resizing and such things that all just works. Pretty cool, and I might use it if I built a site that needed to do that.
One way you can frame the point of this business is that they worry about the details of integrating with these other services so that you don't have to. As they found out, hosting external content is inherently dangerous, and it pays to have someone responsible for that who knows the risks and has experience in mitigating them. If a site owner wasn't using this service, they would have to take that responsibility on for themselves and re-learn these same lessons. So that's just another advantage of using this service - "we have experience in mitigating the risk of hostile users abusing upload services to serve malware, so you don't have to worry about it".
Site owner has not confirmed they screened all uploaded content for malware - this is a major issue these days and google and others will flag you if you host viruses and pump out malware.
And no - you cannot sue google to force them to allow users to be infected.
It’s not clear that all customer content is hosted on a separate domain, and each customer on a separate sub domain . Your reputation will be trashed pretty quickly if you host content on main domain blindly.
It’s not clear that all uploaded content is protected from being linked too or downloaded. Google admins and other virus vendors can setup screens on downloads.
Anyways - see plenty of shady / scam and incompetent website owners hosting malware - not much sympathy in most cases.
This is actually a big issue sometimes for folks who use google drive, because malware will infect their files, they will then be synced to google, then blocked from downloading them ever again!
That leads to support requests list this:
https://support.google.com/a/thread/60528209?hl=en
Even if you pay the ransomeware fee, google WILL NOT let you access your own files again. So years worth of files - GONE.
They do use different origins for these services. Google DOES actively ban users (everything, youtube, drive and email) from their service even users using google services (vs a third party upload service). Ie, if you are going to run a phishing scam, host the image on this service, not google, or your drive account + a lot of other stuff is at risk. I've even seen google follow links to other accounts your google account is an admin on, so can have business impacts and more.
I don't know where the idea comes from that google is very hands off on this stuff, they run a major spam fighting op that blocks lots of even potentially legit email, they do tons of scans through chrome, they do advanced stuff for opt-in domains on their paid platforms (even more intrusive but let's them pick stuff up behind password locked pages etc).
This last one can really confuse site owners, when NON PUBLIC content results in site bans.
Disclaimer: I'm a Google SRE. But never supported anything reachable from the outside.
I would have thought this would be an excellent way to not host malware.
Has anyone tried, genuinely curious how this would turn out.
You are warned: the above link is a phishing website that when used will spam you whole Facebook friends with the same link via message. Google Chrome, still today, shows it as a normal website: https://imgur.com/a/1bsFutr
I was trying to buy a school bus to make a schoolie out of, the Craigslist add directed me to a seemingly innocuous eBay motors link that looks pretty close to the real thing. I was busy and clicked, totally intending to drop $5k. I got distracted and had to come back to it later, when I did, credit card in hand, the page showed the red screen with a huge warning. A closer look revealed the bad url.
Saved by google? Oh god, I think I need a shower now.
I just ran ads with headlines like Nextjs + TailwindCSS Landing Pages
Apparently somehow I ran afoul of their Circumventing Systems policy. I don't know how this qualifies and when I appealed they came back saying the same thing.
> (from a screenshot) I work at Google [...] so I escalated your issue [...]
> I believe the HN thread getting on the homepage tremendously helped me and somebody from Google saw it and expedited the review after all
So, once more an issue with FAANG could only be fixed because somebody knew somebody else and went out of his way to get this to the right eyes.
This could easily have gone another way and OP would have received no help whatsoever and would have waited for days or weeks to get this issue cleared and lost his business.
Maybe it's only me but I find it unbearable that you'll usually not be able to reach any real person at all for issues like these and it's pure luck what happens to you.
I reached a real person at Google Domains and managed to get things escalated to "Specialist". Their response: "we can't help you, post about it on the community forums" (which I had already done 20 days prior).
This account "owns" digital goods, thousands of songs, and many domain names. Google is actively stealing these things, but they don't care and, "can't help".
Funnily enough that password leaked and someone managed to take over my account (I wonder how they manage to bypass the security questions, sounds like a security vulnerability on Apple) and they're using it to register (I assume) stolen devices and install software. I get email notifications every time these people do something.
I reached Apple support but they're unwilling to help, they even refuse to nuke the account as a last resort.
I wonder how many people lost their account like me because of these overzealous security measures.
I long for the day that they cross the wrong person with means to take them to court over their negligence.
It's always the same story. Some guy gets on Twitter or HN who happens to get noticed, then FAANG releases a statement saying they made a "mistake". Mistakes in the aggregate that affect millions of people aren't "mistakes." That's deliberate malfeasance at scale.
Funny they never ask you that design question in interviews. "Design a system which will harm at most 5% of your users while scaling up to billions of people." Maybe if more people understood the sobering dark side of scale, they would stop gleefully promoting runaway scale-at-any-cost engineering.
Just kidding. Profit is God.
I'm also reminded of the dystopian movie Brazil. You're always at danger of getting eaten by the bureaucratic machine today, with only the most absurd recourse available. Just read the passive indifference of the email that Google sent this guy. "Google has received and processed...", "Google systems indicate...". This is one shit dystopia are are living.
If you know important people at the emperor's court, you have a chance to get yor problem solved.
Even with HN it's a complete lottery what contents reaches the front page, so getting issues like these resolved is a matter of extreme luck for a common person.
If enough companies contribute to this, it might put some pressure on FAANG to take things seriously.
They kind of do
> So after a lot of brainstorming and ideas from HNers I finally figured out the culprit(s).
> We have a live demo on our home where people can upload a test file. [...]
> We also give all users a 20MB test storage. [...]
> I believe that somebody signed up for our service (it’s free to sign up) and then uploaded a malicious file on our test storage and abused this feature.
If that is correct, Google was completely in the right to flag the domain as malicious and warn visitors.
edit: just noticed, their comp.lang.c archives are back up now
NodeBB does host a demo instance to allow people to kick the tires. I don't believe we allow people to upload images, but it is worth double checking just in case.
https://github.com/publicsuffix/list/blob/master/public_suff...
https://symantec-enterprise-blogs.security.com/blogs/feature...
Way less nowadays due to all the employee shaming.
I have gotten warnings from google multiple times about hosting NSFW images (that is not the purpose of the site) that have ads on the page. This isn't google disliking NSFW content - it's google not liking NSFW content and ads together. Due to multiple warnings, and worried about bans, I now actually manually review each image. This is actually easier than it sounds. I wrote myself a batch script and review in chunks before I allow google to view any images.
"Potential phishing attempt. This web page tries to trick visitors to submit sensitive personal information such as login data or credit card numbers."
Is this somehow related to the Google situation?
The question I have is what can anyone do to really change things? If we all agree this is a major issue why can't we find a reasonable solution to it.
Especially if you can show damages/customers cancelling service, I think this would be a hill to die on. Google et al have too much power, even over people and orgs that aren't even customers. Its high time we reign their powers in, find them strongly culpable for what they do (and what they change and then refuse to do), and consider breaking up these monster companies up when they show they are against the public interest.
Were you, uploaderwin, given a notice prior (say to abuse@uploader.win , admin@uploader.win or other appropriate mails) to being effectively banned WRT google? I'd go on a limb and say you didnt. No, you have to be aware of the right page at Google, register you as an admin to the site, and hope they share what they consider abuse.
And frankly, you were lucky you got the social media escalation. You should have never had this happen... But here we are.
The website owner's theory is that someone used their demo to upload a genuinely malicious file, and presumably then shared it to others. Adding the site to their blocklist immediately is a reasonable action taken in defense of Google's users. It's certainly not libel for them to truthfully say the website is hosting malicious content. Well, not in the US; other jurisdictions don't necessarily have truth as a defense. (Tortious interference is complicated, but typically requires that the person interfering knows about the business relationship they're obstructing, and is taking the action for the purpose of obstructing it. It seems like a stretch here.)
As always with Google, the real issue here is their awful communication and slow responses to people who can't find a way to go outside the normal channels.
EDIT: and the article has some useful suggestions for practices to follow if you need to let people upload files as a demo. I hadn't really considered the purpose of a separate domain for such things from this angle before.
Even it is correct, we can't assume it will be always correct.
> As always with Google, the real issue here is their awful communication and slow responses to people who can't find a way to go outside the normal channels.
Real problem is their slow repsponse can kill business (or may be people). If they are yielding this much power, there must be atleast some paid support service. I guess, it is time, all govs should look into this and regulate FAANG.
1. The poster was hosting malicious content from their domain (user uploaded no doubt, but still on the domain they control).
2. On one hand, it is desirable that people who are not malicious be given enough information as fast as possible to rectify their sites.
3. On the other hand, this same sort of information can make it easier for malicious users to evade detection.
That is, it seems to me like there is an inherent tension between #2 and #3 that make a simple solution difficult.
Seems to me that:
1. As the poster discovered, user content should always be hosted on a separate domain. Google should recommend this as a standard good practice.
2. Perhaps I'm missing something, but when Google blocks an entire domain, I don't see the harm in telling the site owner which subdomain is causing the flag, which could let good users identify the problem faster.
I never bought that excuse. That sounds like saying we should be secretive about legal charges brought against a person, lest that information help criminals evade detection.
For SVG just paste the markup on your HTML. Browser support is excellent and it will weight less than using a base64 encoded string.
You will be able to style it using CSS as if it was regular HTML, use JS, etc.
https://news.ycombinator.com/threads?id=daave
What the?
Really a disgusting company.
that's quite a strong word. For the average Joe, google has immesurably improved their internet experience. The vast majority of people are perfectly happy with google and love it for gmail, youtube etc. Just because they are good at destroying some peoples lives, most users don't really care at all. You might find it hard to recruit supporters just because google are horrible sometimes.
