Skip to content

Top New Best Ask Show Jobs

Last Mile Redis | Better HN

Last Mile Redis (opens in new tab)

(fly.io)

260 pointspw4y ago59 comments

59 comments

ruuda4y ago

I'm still not entirely convinced that serving apps closer to users solves more problems than it creates, but I am always impressed by the fly.io blog posts. The writing style makes them a joy to read.

_6pvr4y ago

Most pieces of technology create as many problems as they're meant to solve. The cost-benefit is always around whether or not the solved problems are worth the cost of the new problems. If latency is such a problem that you'd want to deploy servers and caches as close as possible to a user, you'd probably gladly pay for the reduction of that latency at the cost of the new class(es) of problem.

It was an interesting decision to make a SaaS out of this solution, though, as I think 99% of problems are not solved by having machines closer to their users.

simonw4y ago

My favourite kind of technology is technology that either takes something that was impossible and makes it possible, or that takes something that was too hard to do and reduces the difficulty to the point that you chose to do it when previously you would have chosen not to.

Fly.io fits that latter category perfectly. Global distribution was generally Too Hard to consider for my projects. Now it isn't.

nine_k4y ago

Not only latency but possibly also the cost / availability of massive egress. This is why e.g. YouTube has local caches around the world. Other heavily-hit caches with heavy content may make sense, too; I can't easily imagine a case beside serving video, though.

victor1064y ago

> Most pieces of technology create as many problems as they're meant to solve. The cost-benefit is always around whether or not the solved problems are worth the cost of the new problems.

Well said.

fprct4y ago

Too bad their blog doesn't expose RSS feed!

Whoops. It used to. Going to fix that soon!

Fly is the only company that consistently makes me want to reconsider my negative views on "the cloud", and it's absolutely 100% due to their incredible blog posts.

As for this particular post, it's great as always, but I would have liked to see more specifics on how applications might write to the main replica vs their local Redis instance.

mrkurt4y ago

Well that's nice of you to say. Would you believe we also have negative views of the cloud?

You're right, we kind of glossed over how to do that. People usually just keep two Redis connections in their code, something like `regionalRedis` and `globalRedis`. It's cheap to keep Redis connections around.

I can't really think of a better way to handle it, it's kind of a weird problem because not _all_ writes need to go a certain place, just writes you deem global.

Since I personally believe fly.io has the most potential in redefining how apps are deployed (though, with no disrespect I don’t believe it’s quite there yet) … is there any plans to providing a PaaS like offering where I can deploy my favorite web framework to your edge network and you handle the OS, Database, App server, etc.

Said another way, I just write my web app code and fly.io handles literally everything else. (I don’t even mess with docker etc. Just my app code and be done)

Just holding the two Redis connections makes sense. I remember the Postgres post said something about modifying HTTP headers (a solution I liked), and I didn't know if this would be something similar.

Actually, I just looked back, and that was to solve the problem of only being able to write to a single postgres instance of the globally distributed cluster. For just caching, with every instance writeable, that probably wouldn't apply here.

manigandham4y ago

Fly has become a serious contender. I remember the early days where they were more of a reverse proxy with javascript workers but have since evolved into a fantastic platform, solving for both compute and data.

I might have to launch a startup just to get a chance to use them.

gingerlime4y ago

yes it looks really interesting. I wonder about the reverse proxy (with caching) aspect since we’re looking at caching some of our Rails pages, and to purge them would mean upgrade to Cloudflare enterprise which is expensive for us. Is fly a more flexible CDN contender then? or would we be better off with bunny or keycdn for this use-case? where does the reverse proxy route fit in this picture?

manigandham4y ago

Fly is no longer a CDN or reverse proxy. It lets you run anything packaged into container in several regions around the world and serve traffic to them all through a single domain.

You can build a CDN on top but if you just need basic CDN features then you should probably look at something else. Cloudflare allows purging all of the content, or individual URLs, for free through the UI or API. You only need enterprise for the more advanced tag-based purging. You can also look at Fastly which is another configurable CDN. BunnyCDN is also good. Start simple and then move as you need more.

taf24y ago

We do this to locally distribute our javascript… from our main rails app we publish out to each region in aws where we have a cluster of redis and nginx and a little bit of lua to grab and render data out… it’s super fast and very easy to maintain … you get your nice normalized database with all its complex queries and then on the client side you get your nice big blob of context all precomputed and perfect for rendering exactly what the client needs… bonus you cache the context data for a shorter time in each local nginx worker for even faster response times then serving from disk…

These posts always strike a nice balance between doing a deep dive on a topic while still staying accessible and building the reader up to the "Aha!" step by step although I have things deployed on Fly so maybe I'm a little biased at this point

> The most boring way to exploit geographic cache locality is "just a bunch of Redii".

I'm not a latin expert, but 'is' is not a latin ending. The closest would be 'es', but then the plural would be Redia or Reda.

"-is" is typical 3rd declension. If it's not neuter (how do you pick a gender for loanwords in Romance languages?), it'd be Redes. If it's neuter it'd be Reda as you said.

Or maybe it'd follow the -eris pattern and be Rederes or Redera.

I thought 3rd declension is just 's' for the nominative singular. So we'd need to know what the stem is.

mrkurt4y ago

This might surprise you, but I am also not a latin expert. Maybe the plural of Redis is just Redis.

:)

Sorry I was just being pedantic on a Friday afternoon. The rest of the blog post was great and I agree with all your points. I've been advocating for immutable cache keys for years, for the exact reason you mention.

twic4y ago

Surely Redare.

Clavis “key”, plural with -es. “Redii” would be plural of “Redius”; “Redes” is the correct one. (In English, “Redises” would be just as good.)

devoutsalsa4y ago

If we're playing Warhammer 40K, the chapter keeping all these Redises around the galaxy alive might be called the Cachess Redisari. Now I wanna see what happens when get a little too twisted by the chaos.

friseurtermin4y ago

Great post! I'm not familiar with all of fly's inner workings but I'm guessing that each app gets its own Redis instance at each location (rather than all of your customers sharing instances) because "global" item replication is a bit cheaper if "global" == locations where the app actually exists. I wonder if there's a way for an application to write to a subset of locations (i.e. locations all over the US, I just want to update something in California). Obv I could have each of my CA locations do a local update but maybe I don't want to

The Redis instances we're talking about here are "just" Fly.io apps; they're not like a special platform feature or anything. Which is good news; it means you can do whatever you want with Redis. You could, I guess, do a replica-of-replicas tree configuration, with app nodes holding connections (Redis connections are really cheap) to their local node, a macro-regional node, and the global node and selectively synchronizing. Or you could go the other way and lose some Redis servers, keeping Redis servers only in certain regions. We make it sort of straightforward for apps to suss out the topology with DNS queries.

> Fly runs apps (and databases) close to users, by taking Docker images and transmogrifying them into Firecracker micro-vms, running on our hardware around the world.

Any reason you run the apps on micro-vms? Why not directly on a container runtime?

href4y ago

A container runtime would not provide proper isolation. The VMs run on bare metal along with the VMs of other users.

See https://fly.io/blog/sandboxing-and-workload-isolation/

Thank you. That was a great post.

alexchamberlain4y ago

From https://fly.io/docs/reference/architecture/,

> MicroVMs provide strong hardware-virtualization-based security and workload isolation, this allows us to safely run applications from different customers on shared hardware.

ipsum24y ago

I'm curious what the moat of fly.io is. Wouldn't Cloudflare use the same Firecracker VMs to replicate the same experience here if it becomes successful?

mike_d4y ago

The moat is they have super smart folks who keep iterating on the right problems.

Cloudflare is too busy trying to build a unified stack for boiling the entire ocean.

ipsum24y ago

Smart people aren't a moat, even if they're super smart.

What do you mean?

I'm sure others will be curious about their concrete locations:

Locations: https://js.fly.io/docs/#datacenter-locations

(TLDR: mostly local in the US so far, with sparse presence globally)

alexchamberlain4y ago

I think that link is out of date? There is a different list here: https://fly.io/docs/reference/regions/

j / k navigate · click thread line to collapse