Firecracker – Lightweight Virtualization for Serverless Computing (opens in new tab)

Want to run Ruby on Rails on Lambda, with no changes from the servers I run on my laptop. Or maybe I want to run Crystal. Or maybe I’m writing my own language. Doesn’t really matter.

Lambda works great as a deployment and execution model. This allows anything to run on Lambda, not just specially prepared runtimes.

Your container = your environment.

When your container is not running (say, 99% of time), other customers' containers are running. No need to ever boot the kernel, etc.

One might say that an unikernel has advantages over it. But it also has a higher barrier to entry.

I need to run 1+ Fargate containers 24/7, which is useless and wasteful.

With Fargate-Lambda crossover I wouldn't be running anything 24/7, and it would be a lot less resource intensive than one Lambda-Container per request as well.

Google's App Engine gets / got this right when they first launched, but to make it work they had to demand apps be written for their sandbox (like AWS Lambda), because of which the model isn't as general purpose. Firecracker would allow regular containers to be used this way, making a Firecracker service the first service to allow general purpose servers to be started and stopped (all the way to zero) based on incoming traffic.

Another missing piece in addition to the billing aspect mentioned elsewhere, is all the existing event based integrations lambda provides. E.g. react to kinesis, sqs, sns etc. events. having aws manage the event plumbing in addition to start/pause/stop is really nice.

Pricing model is not serverless. The basic serverless principle is no-use-no-pay.

> I'm curious why they don't provide a comparison

They do, if you read the FAQs: https://firecracker-microvm.github.io/#faq

From memory the original version of Intel Clear Containers had its own kvm based vmm but they moved back to qemu (or a more minimal patched version they maintain). They are working on containerd support so should be similar to Kata soon.

OK I had missed the kata containers blurb in the FAQ, thanks for pointing it out. In fact the tweets make my point: we are all so blinded by new shiny releases that we forget their highly incremental nature.

Also cloudflare announced workers using isolates.

Right. In the end, AWS saw containerization as an existential threat, and serverless is its response to the commoditization of AWS (and other specialized cloud vendors) by containerization technology. Serverless helps AWS re-couple your application back into the specialized AWS vendor environment, once more requiring you to keep specialized and costly AWS-specific knowledge on-hand in order to build and deploy your application. It gives them plenty of room to advertise and provide all of those edge case services to you once more (and their usage charges) and also helps them prevent you from treating AWS like a rack of networked CPUs to serve as the substrate for your application containers.

People (mostly AWS folks -- dig a little deeper into who is writing much of the serverless blog posts out there) keep pushing the "serverless is containers" but that's just a tactical response. Add a layer of abstraction and it's very clear why AWS is betting so hard on serverless. Originally, AWS commoditized the old datacenters by providing the same network/CPU substrate, but at a higher cost because you outsourced the management of those resources to AWS. And AWS slowly dripped out new and convenient services for your application to consume, allowing you to outsource even more of your application needs to this one vendor. And while the services offered by AWS were just a little bit different, they were functionally similar. And that's how you locked yourself into using AWS instead of CoreColoNETBiz or whatever datacenter you were using before. I remember one of the first major outages of us-east-1, which caught most of the internet with its pants down (interestingly, the answer was just to give more money to AWS for multi-region redundancy). AWS had a pretty good thing going: Outsourcing the management of all those resources to AWS is expensive! But that's when containers came along and people at AWS started to take notice. With containers, people could de-couple their applications from Dynamo and Elastic Beanstalk and VPC and all those specialized services that cost so much time/money. Instead, you could just cram all that shit into containers, without needing to set up IAM roles or pore over Dynamo documentation or dump so much time into getting VPC set up just right. And that's the whole point of containerization: Easily build your services in a homogeneous environment with exactly the software you want to use and eliminate that technical debt of vendor lock-in and the enormous cost center of specialized vendor knowledge (e.g. Dynamo, IAM, VPC, etc etc). Treat the cloud -- any cloud -- like a bunch of agnostic resources. Docker commoditized the commoditizers.

And serverless is how AWS plans to get you to re-couple your application tightly to their specialized web of knowledge and services. They get to say that you're still using containers, but they need to gloss over the fact that you're locked into the AWS version of containers. You cannot "export" your specialized AWS-only knowledge of Fargate or Lambda or API Gateway or ECS to Google Cloud or Azure or some dirt cheap OVH bare metal. You're tightly re-coupled to AWS, having bought into their "de-commoditization" strategy. Which I need to stress is totally fine, if you're okay with that. It just needs to be made clear what you are trading off.

The VPC startup times are insane, so we quickly move our lambdas out of that, accepting the trade off.

Our normal cold starts are in the 1-2 second range, and the app initialization comes after. Too high for an API facing users :/

We felt the same way about QEMU before we started crosvm. Glad to see you all found some use out of it.

Katacoda has nothing to do with Kata Containers...

I'm not sure how you can make any of the conclusions anyway, unless you know a lot of seemingly private details about how KataCoda is implemented.

Which is exactly the problem.

That’s why the attack surface is way larger and harder to keep an eye on.

Yeah, jokes aside I simply don’t think it makes sense to run full processes on the edge. Not yet, anyway.

Script isolates makes a lot of sense with current hardware limitations, but full processes at the edge are coming sooner or later.

Yeah but one of the main ways in which gVisor provides security is by intercepting system calls and strictly limiting which calls can be made. Firecracker may use KVM instead of running entirely in usermode, but as far as most of us are concerned, that's an implementation detail. The pertinent question is whether the price of security is limiting the possible system calls, which means that Firecracker won't be able to run arbitrary containers, just as gVisor doesn't guarantee that it can run arbitrary code (which may require filtered system calls).

KVM supports Windows just fine, which is why you can run Windows on GCP and Openstack. And Firecracker seems to support enough of a machine to boot Windows as long as the windows instance has support for libvirt disk devices and a libvirt NIC.

However, it seems they boot in a slightly unconventional way. They take a elf64 binary and execute it. This works for Linux and likely some other operating systems that can produce elf64 binaries. Windows supports legacy x86 boot and UEFI, but likely not elf64 "direct boot".

So if you can get windows into an elf64 binary and have it run without a GPU you could have it boot. So, likely not. But the reason isn't due to KVM.