You should have all you backups in a different location and terraform tested with a different cloud provider, otherwise you're risking the company.
[Edit] Where I come from: That doesn't say anything about Hetzner, I have been with them for 20+ years, they have stopped individual servers in that time frame, but haven't cancelled my whole account.
First they wanted us out of on-premise, and told us costs wouldn't matter.
Then they wanted us to be 'cloud agnostic', but when given deadlines changed to 'get it working in AWS ASAP, doesn't matter the tech debt'
Now they're freaking out about AWS costs, and we're back to juggling 'cloud agnostic' and 'reduce cost to serve in all clouds' priorities on top of features and maintenance, both of which are 10x slower due to tech debt and the plethora of bugs.
I really need to find a new job soon. Its insane how badly the execs and upper management are running this company. Every day is a knee jerk reaction from someone so detached from the reality of things or with so little understanding how it works, they do nothing but add process problems that barely address the issues they think they're solving.
There are so many providers, and therefore examples, of physical tin being accessible in under a minute with cost:hardware ratios that blow Cloud out if the sky (pun! ha!) OVH have a server for USD $95/month (with no commitments) that can be brought up and made available 120 _seconds_ that has six 3.8GHz cores, 32GB of RAM, 2x960GB NVMe SSDs, and 1Gbit/s of UNMETERED, guaranteed bandwidth... that's absolutely insane, and that's fully managed from the hardware down, so arguments like, "bUT yoU haVe to MAintAin hardWARE!" are just not true _at all_.
But also, the execs are the ones making the business risk decisions. Just make sure they have the correct info to make those decisions, the. Your responsibility is done.
When the hard drive failed, they restored the customer to the latest backup. Which was the tape still sitting in the tape drive in the server. It was from the first Sunday night after the system was installed years ago
> Regular backups seem to also only be taken once per 24 hours, though team-member-1 has not yet been able to figure out where they are stored. According to team-member-2 these don’t appear to be working, producing files only a few bytes in size.
As with code deployment, it's not so scary when it's something you do so frequently that it's just a little script you run.
I’m doing that to linux, and then the Linux box is furthermore backed up with nakivo.
Not my favorite but the price was okay and I can run the whole director on Linux, unlike all their other competitors. [veeam’s next major release 13 or 14 should do this in the next year or so too.]
While nakivo backs up s3 buckets, NFS shares, and local file servers… to your point, I don’t trust it (or any other backup software I can’t extract and unpack the resulting backup by hand) as far as I can throw it. So I rsync or mirror it to a local Linux box with aws-cli and then back THAT up.
I think you can do all this with windows stuff too but I don’t know it that well
Additionally you can take servers that are linux vps’es and do the reverse: mirror THEIR content to an s3 bucket.
You can also run minio open source/free on your fileserver and set up s3 to s3 sync. Cloudflare for example will ingest and replicate your minio server automatically and you can firewall it all off to their address ranges. It’s not free but it actually prices out favorably compared to veeam and nakivo if that’s all you need backed up.
Of course it turns out that the restore can only happen to a _new database name_ not the original, and the code had in multiple places hardcoded the assumption of what the db was called.
So restoring also involved patching the code and rolling that out; you can't "roll back" because to roll back the db the code must roll forward.
Google Cloud accidentally wiped an Australian super[annuation] (pension) fund's entire cloud deployment earlier this year. I think that if you really want durable backups, they have to be reducible to object storage and put in someone else's cloud.
1) Expensive
2) Not straightforward, e.g. is there a 1:1 setup in another cloud for your system?
3) Likely to go untested and be useless when you need it most
The most expensive part is going to be maintaining an up-to-date offsite data backup. Running a few VMs for a handful of hours is basically free.
https://rsync.net/pricing.html
That said I was once a CTO for a company with 10 photo studios and we had a large amount of new (raw, DSLR) photos per minute, so cost was an issue and also upload speed for offsite backups.
I manually pulled a backup of everything but jeez, not good.
What is the most reasonable point that meets the criteria of 'as soon as possible'?
Because I imagine out of the gate doing this could be a net negative, not a net positive.
On the other hand, I'm not sufficiently well versed enough on the absolute latest devops techniques that may make this whole thing trivial, but I thought all the major cloud providers had just enough quirks in their Terraform support you can't write once standup / deploy anywhere
It's very easy to do if you don't do the absolute latest devops techniques.
Of course an accident is different than just randomly terminating service.
If you had a bunch of retailers in a shared space (like a market), and one of them was setting off fireworks, using all the power/water in the space, and scaring away customers, I'd expect them to get kicked out pretty quickly.
Now it may be that this is a false positive, I'm sure they happen, but in the case where it's a legitimate bad actor that is actively harming both the company and other customers on those servers, what's the course of action the company should take?
Over the past years there have been numerous people online which have claimed that Hetzner closes accounts without giving a reason. I'm sure most of those claims intentionally omit some details to make it look like they didn't infringe the T&C.
However as a Hetzner customer (a small one, to be fair), I'd still like to know that those complaints are baseless, and that I can still trust your company.
2.7. Furthermore, we reserve the right to terminate the contractual relationship without notice for good cause.
This below is where we got started - the ref number should make it easy for you to sort: > Procedure: L0020649F > Person: [redacted] / Kiwix > Cause: Hello, > > Starting this morning (December 1st at 00:00 UTC), our servers went down. > We received zero email nor notification of any kind from you. > Looking for a way to contact you, I looked into this Unlock tab that list an incident > that matches the time the problem started. > > It's been close to (12) hours already, without a single message from you. Our services > are down. > > In the Robot dashboard, there is no server listed. In the Traffic statistics page, it > says we have no IP. > In the Cloud dashboard, we cant even enter, it says Access Denied. > > What's going on? The billing page is reachable and it indicates we paid all our > invoices and the next one is to come in 5 days. So it's not a payment issue. > > I checked > https://docs.hetzner.com/robot/dedicated-server/troubleshoot... > > I am not sure if we're locked because the traceroute does not lead to > blocked.hetzner.com > Because the server is not listed, we cant use the whitelist or any other tool. > > Please restore the service immediately. > Please let us know what kind of issue there is if there is one. > > Only restoring SX65 #2453510 (135.181.224.247) is urgent. The two cloud ones can be > sorted out later.
We got two more emails from Hetzner the day after that (Monday 2) but none addressing the root issue. Our account access had been locked by then anyway so we had to call up Germany; you should be able to document that as well.
Not sure HN is the best place to compare notes but hey, happy to meet you where you feel comfortable responding.
What are you talking about? The comment you're replying to says:
> There was a notice of termination via email with a deadline in accordance with our T&C, on 30 October 2024.
> Our team has already been in contact with this customer several times and we also have the transmission protocol of the communication.
I have never been a customer of google cloud for this reason and i sure as hell wont deploy new servers on hetzner until they provide a clear statement on what went wrong and what they will do to make sure they never screw up like this again.
Hetzner, the ball is in your court.
They're a budget host, you should always proceed with caution and never rely on for production. It's the same as buying a second hand eBay server to host users upon. I learnt that the hard way.
> Hetzner, the ball is in your court.
Not really. If you read the T&C, you'll find that they can do anything with the server.
From their T&C:
2.7. Furthermore, we reserve the right to terminate the contractual relationship without notice for good cause.
--
Any server with any company can do the same. There's been numerous stories of Amazon has doing the same. Same with Google.
Unless it's colocation or where you own the hardware you can be screwed in many ways.
I would never trust a dedicated server host.
Yes, but that doesn’t mean that they have to.
Yeah, if. Why are you so certain that Hetzner "screwed up"?
But this seems to be about Kiwix (which in short is "offline Wikipedia" in various ways) and doesn't seem to be about questionable content in any way.
Eventually I guess we'll get Hetzner's perspective on this, as they tend to start writing publicly about issues once the other side starts writing publicly about it as well.
Personally I've been a happy user of Hetzner for many years, with no issues that weren't my own doing. But reading about people having their servers deleted in the middle of the night on a Sunday (Berlin time) and all data wiped immediately, with no recurse, does sound a bit aggressive. Luckily it seems like both me and Kiwix has mirrors for the data we care about.
Or hosting content that Hetzner misclassified as against their ToC. Or that they decided was because of a string in a random file name. Or, in one Mastodon instances case recently, because Hetzner saw that users could upload their own images and decided that was risky (nevermind that this is common and they have moderation and a strategy for if someone tries to host anything illegal, but that one employee reviewing it was twitchy that day and there is no recourse), etc.
Or, in one Mastodon instances case recently,
because Hetzner saw that users could upload their own images
> Users first download Kiwix (or a browser extension), then download content for offline viewing with Kiwix. [1]
> Our main storage backend became entirely unreachable. For the average user that meant not being able to access the library and download files, and for us that meant not being able to connect to it and see what was wrong. [2]
Maybe some odd photos landed on WikiMedia which then got automatically synced to Hetzner's servers and then triggered some alarms.
I can't judge about Hetzner deleting the data, but them not attempting to really get in touch with the Kiwix team -- after all they should know that they are trying to do some good in this world -- is a really horrible move. In the same category as Google blocking access to user's accounts without any word, or German companies suing security researchers for notifying them about a security flaw in their systems.
Shame on Hetzner.
[0] https://en.wikipedia.org/wiki/Kiwix#Available_content
"the hoster deleted my stuff without warning" is up there with "the dog ate my homework"
On one hand you have these comments in this submission, saying Hetzner is too trigger-happy and takes down things too quickly. On the other hand, you have people like you using the process from the other side who feel like nothing is being done and it takes forever to get through them when needed.
I feel like it's very hard to have a balanced perspective unless you have experience of both sides of the process, which unfortunately I'm guessing most people are missing. I certainly am, as I've never tried to get someone else's servers taken down on Hetzner, so I have no idea how that process works, I've only ever been on the receiving side.
There are several comments under this thread from people reporting essentially that happening to them.
OP claims everything got immediately wiped without warning. That would be against Hetzner's own TOS.
OP also doesn't elaborate further, and is posting this in a position where he has to explain his own downtime of multiple days. Make your own judgmenent what is realistic here
That’s actually insane and business killing. Both for Hetzner’s reputation and potentially for their customer.
I would not recommend them for a serious, money-on-the-table business.
[citation needed]. Even when they shut down Russian customers they gave advance warning. This is the first time I have heard of service being shut off (and data deleted) without any warning.
That’s what the post said. But of course we have no idea if it’s true or not. No evidence was provided, and we are only hearing one side of the story.
And in that hackernews thread we have dozens of people relating similar stories.
The reality that they have this power, and that they'd delete data irretrievably, scares me.
Last year I had a misconfigured port on a Docker service, and someone was able to exploit it and run a port scanner. It was during a period that I was away from home, so if I hadn't seen their service abuse emails in time, I could have returned home after a few days to find all my data wiped out (or uptime monitors complaining).
as much as we like to hammer on EU (lack of) companies, one potential improvement point is customer service
German companies are awful at customer service. Even within the EU
one would have to reconsider a century of stereotypes if they weren't.
True also from my experience. I've noted several potential reasons why that is from my time in Germany.
Government provided customer protection laws are quite lax and disputes tricky to win and don't represent a big enough deterrent for the scammers when they're just a slap on the wrist and therefore part of the cost of doing business. Sure, you can get sued and you loose once, but if 80 of the 100 customers you scammed don't sue you or don't win, then you're still at a net positive and therefore it's profitable to keep doing that.
Also that Germany doesn't have common law, so lawsuits aren't arbitrated based on precedent, so customers who got screwed need to sue and win individually for the same issue which is favorable for the companies doing the screwing as without the precedent of common law that minimizes their risk of loosing by slam dunk every time. Also, some German judges art just tech illiterate boomers who will throw out a case they don't even understand unless you're Axel Springer.
(some) Rental agreements, internet, telco and gym memberships are my favorite infamous examples. They're almost universally regarded as anti-consumer, with tonnes of sketchy clauses, but German lawmakers do nothing to improve that for the consumer.
Secondly, Germans aren't used to being very demanding and lighting a brand on fire on social media the way Americans/Anglophones do on Twitter when they don't like something, partly because of cultural reasons where making a fuss in public is discouraged/shamed, partly because of legal reasons where a company can sue your or at least send you scary legal letters for libel if you damage their brand online like that in Germany. Or at lest, the company can simply demand the social media platform take down the offending posts, and by German law they have to comply which the likes of Google/Meta will comply automatically without any arbitration.
Also, culturally, the conservative Germans seem to have have gaslit themselves into believing everything "Made in Germany" is perfect without fault, while everything made abroad is of poor quality or at least worthy of scrutiny, so they just default to using German products without looking across the fence to check out the foreign competition. This way of thinking is more typical of manufactured goods but not sure how much it applies to SW products and services.
Couple these with the difficulty of starting and scaling a business in Germany as a small entrepreneur and with the legal and bureaucratic hoops designed to keep foreign competitors out, mean that German companies operating in Germany who became established players, have litte incentive to improve beyond the bare minimum, so they can keep providing poor quality services while still staying in business. It's classic of an economy of well connected dinosaurs sitting on old money.
For what now?
Hi,
I don’t have a mastodon to reply directly to you.
But i have had some issues with content being taken down by VPS providers as well.
What I’ve found works well is to use a VPS provider that the public is unaware of. And for some time I had used OVH based on the unlimited bandwidth and the reasoning that Wikipedia and Julian assange (who have far more enemies than I ever will) were using OVH.
I don’t know if that’s true any more because I subsequently moved my content to ENS and IPFS.
Anyway regardless of where your content is actually hosted or lives,
What I had done was turn my “real” servers into content origins , which were concealed form the rest of the world and lock it down in the firewall so it could only be reached by disposable squid proxy servers with a 10-liner config file
Then I pointed DNS , cloud flare etc at the squid nodes
And couldn’t care less if they were taken down.
Because I could deploy new ones in minutes elsewhere.
I didn’t have “bad content”, just ruthless business competition that kept coming at me like Tonya Harding.
And I’m sharing because your content didn’t seem too offensive either.
In the front end VPS nodes you’d just put the real address of your content as the remote origin.
And then nobody but you will ever know where it is.
Then generally your hosting company shouldn’t be aware of what it is either unless they’re snooping around in your files, and if they are, hell with them too.
You’re welcome to pass this along as a remark on avoiding censorship, or keep it to yourslelf as proprietary information I don’t mind. Let me know if you want or need an example squid conf. It’s seriously 10 lines at most and many examples found on google.
Great if it works for you, congrats. But I don't think this solves issues for many people, I doubt it solves an actual issue for you and it's basically the same as using cloudflare/akamai/similar but with a manually setup proxy on a VPS.
- Ask HN: Hetzner banned me with no explanation. What can I do? (https://news.ycombinator.com/item?id=32318524)
- Hetzner didn't even provide a detailed info on why they deactivated my account (https://news.ycombinator.com/item?id=40781617)> Dear Mr David Allison
> After reviewing your updated customer information, we have decided to deactivate your account because of some concerns we have regarding this information. Therefore, we have cancelled all your existing products and orders with us.
> Best regards
> Your Hetzner Online Team
And why would anyone need anything but code in monospace? Please don't do that.
i am always angry if i see articles about them here on HN because such a vendor should be blacklisted and not promoted.
That’s not my experience. We get these emails about once every 6 months, we act and respond, and they don’t take anything down.
Is it possible that maybe others had a different experience than you, and those experiences are as valid as your own?
Besides, what was your website about? I've received notices I had to reply to within 24-hours, otherwise they delete the servers. But I've always replied and complied, so never had any servers deleted.
If it’s a single email - then even if it doesn’t get caught in a Spam filter that’s still a short period of time to notice and respond when the stakes are so high.
If that email goes to junk, or you’re unwell and not checking emails as frequently (given - I assume - that many of Hetzner’s customers are individuals) or any other number of reasonable situations, you’ve effectively had no warning before service termination and deletion of data.
I don’t mind cloud providers acting on suspicious usage patterns or abuse reports but there has to be some kind of due process or it just ends up unnecessarily destroying goodwill in a brand/provider.
some random app vendor didn't like the free promotion on our website https://macupdater.net/
we can delete any "offending" page within a few hours, but taking the whole server offline first and asking questions later is not OK by Hetzner.
others had better experiences and got a 24-hour timeframe. just asking but is this during business hours or can they send you a notice on saturday and you'll be offline by sunday? doesn't seem much better.
I was considering them for a small project, but as this project will be nobody's fulltime job, I can't guarantee that I or anybody else would necessarily see that email within 24 hours.
Even if most people will have no problem with them, I’d say that knowing how a company handles edge cases like this is much more valuable than knowing how the handle things when everything is fine.
They offered to "recover" the account, which was basically just an account shell with my info. All of the assets and backups had been permanently erased.
Now we need to know the full story. Did you have a public DMCA takedown link and actually handle requests and the complainers just ignored that and went over your head to Herzner? or did you just wing it running a server with UGC thinking it's surely gonna be OK?
I am not saying you were wrong but you only tell a small part of the story
However this is more related to EU regulation rather than Hetzner itself.
Hosting things within the EU has become really tough.
I, as a European, using mostly dedicated servers within the EU (including Hetzner) haven't noticed this at all. What are you referring to specifically?
Some "use cases" like building marketing profiles and alike certainly has gotten harder, but that's a feature so I'm guessing you're not referring to that. I don't think general "hosting things" has become any harder than before, assuming you're not trying to slurp up as much data as possible.
they did NOT give any 24 hours.
I have no idea if that was the reason, though.
f.ex. the situation with egress costing money in the US, but it's free on all EU location.s
Aren't you confusing Hetzner Cloud with Hetzner Robot (dedicated servers) here? AFAIK, Cloud has egress costs while Robot is usually unmetered.
They are exceptionally fast at detecting things like that though.
Out of pure spite I built my own data center.
I would be curious about any details you can share.
Of course it wouldn't work for all cases but I find it beats having a vps somewhere that can be taken down for no reason at all.
Hardware: 4x Old decommed 19" dells on Ebay with plenty of DDR4 memory, HP Proliant G10+ are also good
Ups Eaton Pro
Gigabit Fiber Internet, which is more than enough. 10-50mbit can suffice for compute nodes too.
Bought ssds and m2 storage plus some spinning ols rust drives
Temp and humidity monitoring
Google Nest Protect smoke detector
TP link 16amp smart plugs on all, to have a control plane to turn it all off remotely
Workloads:
Most are LXC
Some Docker
KVM virtual machines
Zero trust: Some Cloudflare
Tailscale
Proxmox backup server to back it all up, lots of retention
Monitoring:
Deployed remote uptime monitoring on fly.io
Read and experiment a lot Hang out on /r/homelab /r/homedatacenter and /r/selfhosted for learning, community and inspiration
We cancelled our account last month because of that.
I cannot imagine the world of hurt that we'd be ushered in, had they actually dropped our data wholesale like they did for OP.
https://lowendspirit.com/discussion/comment/191966#Comment_1...
Would definitely be good to hear hetzners side of the story because all the cases I’ve seen thus far turned out to be a case of initial telling being understandably upset but leaving out crucial details.
They definitely are trigger happy with telling customers to find someone else & generally don’t elaborate on why
Last time I used them (pre-2020) they were going as far as requesting customer's ID and rejecting them on the basis of country of origin, and I assume this also includes facial features that may resemble "an average scammer". Obviously this did not happen to European/American IPs so they never faced such issues, and as such this practice was invisible to the world.
I can say for sure OVH and Scaleway would try to negotiate with you before erasing your data - this may have changed over the years.
Dodged a bullet.
I am always recommending to not build on Hetzner.
Ok but on topic, who is this guy and why did they do this to him?
The funny bit was I paid the invoice, and then my account remained suspended. When support finally got back to me a few days later, they said (and I quote)
Dear Client
We want to give you one last chance as a gesture of goodwill, so we revoked the cancellation for you.
Kind regards
Seeing it happen to a reputable project such as Kiwix [0] definitely damages my perception of Hetzner. I've read numerous complains on Reddit a few months ago but they mostly boiled down to breaching the ToS in obvious ways. Still, not giving a heads up before cancelling a service and no option to recover data is just bad business practice.
[0] (I've deployed Pi's with Kiwix in remote areas in Africa, it's an amazing project)
Same goes for having your domain with the compute provider.
If you're locked on some proprietary services like with AWS it is much bigger issue.
Given one of kiwix.org's goals on their homepage is "access to internet ... [due to] ... outright censorship" (which is legally required in some countries), likely they are in violation of T&C in
"8.1. The Customer is obligated to check and comply with the legal provisions arising from the use of the contractually agreed services"
and
"8.4. If we become aware of illegal activities, we are obligated under Art. 6 Abs. 1 DSA (Digital Services Act) to request that the Customer immediately removes the offending content and we are entitled to lock the Customer’s access to their Hetzner services or account."
Of course there is usually a bit of a chicken and egg issue with this sort of thing. Many companies only respond at all when complaints go viral on sites such as hn.
None of this is to say that Hetzner responded in an ideal manner, and whatever reasons exist for the termination are still not known, but it seems likely at least some of the OP's criticisms of the process are not valid.
Of course now it's too late as many will never come back to review this thread. As often happens on hn when somebody complains about something, the pitchforks come out before there is enough information to really understand the situation fully.
1) There is some fundamental data aspect Kiwix hasn't mentioned (or is entirely unaware of). I.e. CP or some other super illegal stuff.
2) Hetzner is profoundly incompetent, deleted production servers by accident, and the "But we sent you an email!" thing is a lie to cover up the mistake.
3) There is some kind of interaction that happened prior to this that we aren't privy to. Perhaps a series of late bills, legal threats, or some other inter-personal issue.
Predicted outcome:
I either expect Kiwix to get a knock by federal/national authorities. Or the more likely outcome in my opinion: some frustratingly vague statement by Hetzner PR about its customers being "mistaken" in regards as to why data-go-poof.
I mean seriously, let us assume it's something illegal: Sure, fine, whatever. Wouldn't it make more sense for that material to not be deleted, so whoever the guilty party is would be arrested for/prosecuted by it? Deleting the servers would be like police being informed about a murder weapon and asking the tipster to destroy the weapon before an arrest is even made. It doesn't make any sense to me. Surely if some bad thing were discovered, there would be some method to encrypt/restrict illicit material without destroying it.
Either bad blood, or unpaid bills, or simple incompetence seems like the most likely culprits to me.
As an example, you run any crypto related operation, even if it's a mere 5% of your workload, you will have this happen to you. You don't even have to be hosting anything at all.
OVH had a datacenter burn down a few years ago, so think about that what you will :)
Imo it makes sense to spread out, and have backups with a different provider than the one your main servers run on. They should all over S3, which is a standardized format for easy syncing of backups
There's also Webtropia I have used in the past, also German, without issues.
OVH would be the safest bet but their support is worst than Hetzner.
"""" We have detected that your server is using different MAC addresses from those allowed by your Robot account.
Please take all necessary measures to avoid this in the future and to solve the issue. We also request that you send a short response to us. This response should contain information about how this could have happened and what you intend to do about it. In the event that the following steps are not completed successfully, your server can be locked at any time after DATEHERE.
How to proceed: - Solve the issue - Please note, in case you have fixed the problem, please wait at least 10 minutes before rechecking: https://abuse.hetzner.com/retries/?token=TOKENHERE - After successfully testing that the issue is resolved, send us a statement by using the following link: https://abuse.hetzner.com/statements/?token=TOKENHERE
Please visit our FAQ here, if you are unsure how to proceed: https://docs.hetzner.com/robot/dedicated-server/faq/error-fa... """
I was just using standard Docker to host a web app. No proxmox or KVM of any sort. I would just wait the 10 minutes, click their link https://abuse.hetzner.com/retries/?token=TOKENHERE, which would retry and would come back fine and my response would be "I changed nothing and the retry came back solved. I've done tcpdumps over a weeks time to see if any MAC addresses leak from the OS and none have while a similar ticket like this gets opened every couple days." The ticket would close shortly after I submitted.
I inquired to them at least twice about this and they just kept telling me I was leaking a MAC address that I wasn't allowed to even when I had proof of tcpdumps over a week time period. I found someone else who had this issue with them (most issues around this that I found were people hosting Proxmox) and they had Hetzner replace the NIC and it fixed the issue. Well, Hetzner wouldn't replace my NIC because "it was working" even though I referenced these abuse tickets. I ended up getting another dedicated server, migrated my app over there, and I haven't had issues since.
Their support is seriously not very good. Since that experience, I have had backups elsewhere and test restoring those backups regularly. The price to performance I get from them is unbeatable and like I said, I haven't had issues since getting a new machine. But, I'm definitely cautious and don't exactly trust things to not go sideways even though it's been 2 years since that experience.
Every half decent switch made in the last 25 years can be configured to allowlist MAC addresses. Either that, or dropping customers onto their own VLANs is the standard way of managing this.
Some of the files they host are pretty big, so maybe Hetzner just decided it wasn't worth hosting any more.
I've been using Hetzner for years though and never had an issue. But I don't get anywhere close to the 20TB traffic limit.
This reminds me that I should set up some backups though.
After all, Hetzner is now priotizing shareholder value and is removing smaller customers wasting their compute resources.
Crypto validators can be quite noisy neighbours which is a problem on fair use VPS
Dont think it relates to small or not
But I am not in a position to take those decisions anyway.
I have stopped relying on instances being secure and map out a just-in-case strategy (that I also regularly oversee or exercise) to quickly reset/restore and get back on track.
It does raise an interesting question of how to reliably contact a customer if email is broken?
aws, azure, and gcp aren’t cheap , but they offer better stability—both technically and operationally.
AWS has lots of problems. But they have a team of real humans that respond to tickets around the clock and actually understand stuff. To many businesses, that's worth the extra cost clone.
Even if you're a nobody spending $30 a month, AWS are extremely responsive and helpful.
https://www.theblock.co/post/182283/1000-solana-validators-g...
That said, I host on them too. But some stuff is on nearlyfreespeech.net.
I know a few others who have suffered the same from this company.
> 6. Termination
> The Provider may terminate this Agreement at any time, for any reason, with or without notice to the Adopter. Adopter may terminate the Services upon notice to the Provider.
Theres certainly other (German) legislation in place that might be relevant, but whats the point of speculating if we can’t blame the EU for it.