I am developer, but mostly work on desktop apps, or embedded devices or lataly on some MVC applications. But reading things like
"A Container-Native Stack for Modern Applications and Operations Increase development velocity while simplifying operations."
I have no idea what should I imagine and what is it good for...
any good introduction or explanation into what is it they actually do?
In the ancient past, like when I'm from, you'd write up a few different bash scripts to help you provision each server type. But setting this all up, you'd still have to run around and create 20 servers and provision them into one of 5 different types, etc.
Then there's chef/puppet, which takes your bash script and makes it a little more maintainable. But there are still issues: huge divide between dev/prod environments, and adding 5 new nodes ASAP is still tedious.
Now you have cloud and container orchestration. Containers are like the git repos of the server world. You build a container to run each of your apps (nginx, redis, etc), configure each once (like coding and committing), and then they work identically on dev and prod after you launch them (you can clone/pull onto hardware). And what's more, since a container image is pre-built, it launches on metal in a matter of seconds, not minutes. All the apt-get install crap was done at image build time, not container launch time.
Things are a lot easier now, but you still have a problem. You're scaling to 30, maybe 50 different servers running 6 or 7 different services. More and more you want to treat your hardware as a generic compute cloud, but you can't escape that, even with docker, your servers have identities and personalities. You still need to sit and think about which of your 50 servers to launch a container on, and make sure it's under the correct load balancer, etc.
That's where Kubernetes steps in; it's a level of abstraction higher than docker, and works at the cluster level. You define services around your docker containers, and let Kubernetes initialize the hardware, and abstract it away into a giant compute cloud, and then all you have to do is tell kubernetes to scale a certain service up and down, and it automatically figures out which servers to take the action on and modifies your load balancer for that service accordingly.
At the scale of "a few servers", Kubernetes doesn't help much. At the scale of dozens or hundreds, it definitely does. "Orchestration" isn't just a buzzword, it's the correct term here; all those containers and services and pieces of hardware DO need to be wrangled. In the past it was a full time sysadmin job, now it's just a Kubernetes or Fleet config file.
Disclosure: I'm currently writing a book on Docker. Disclaimer: I have not had my coffee yet.
Edit: Since someone asked, I'm writing a book called "Complete Docker" which will be published by Apress. I don't know the exact pub date that Apress will launch it on, but I expect it'll be available in October.
Now I wonder.. how many projects actually needs these kind of solution when even StackOverflow can do without it (they are in the range of few servers)? I would imagine it would be only few top popular web apps/services, but by popularity of these posts it looks like it is probably a lot more...
Something I never understood with containers: where do they store persistent data, e.g. MySQL's /var/lib/mysql - and how does upgrading work, i.e. when the apt-get postinstall script runs transformation on the persistent data, how is the transformation applied to the "clones"?
And hopefully your book will mention SmartOS and Triton and zones providing full isolation for Docker.
"At the scale of "a few servers", Kubernetes doesn't help much. At the scale of dozens or hundreds, it definitely does."
So now I know I don't need to learn much about Kubernetes unless I have "dozens or hundreds" of servers to manage.
Now, think about it from a reductionistic engineering perspective. What do I really want this server to do? Well, it accepts TCP connections, parses a request to figure out which file (at the time, it was all files) to server, sticks an HTTP header on the file, and shoves it down the socket.
This task is so simple that a skilled network programmer can nowadays literally bash together a 1990s-level static HTTP server in an day, with nothing but a socket library and some basic string handling. (It may not be great and it probably is insecure, but, well... see also "1990s web server"....) The code to do this is perhaps in the dozens or hundreds of kilobytes.
But that's not what I had. I have a full computer that physically needs to live somewhere. It has hardware ethernet and hardware graphics cards and a physical monitor and a power supply and RAM and, basically, hardware hardware hardware, the failure of any one of which means the system is either difficult to change or outright down. I have an entire Windows operating system, which even in the 1990s was hundreds of megabytes of code, endless code. Code for a windowing system, for pete's sake. Code for the audio subsystem. Code for accessing the hard drive. Code for access code that accesses code. Code code code code code, a bug in any one of which means the system may be down or insecure. My website, which at the time was quite likely in single-digit megabytes in size, was a tiny directory lost in a sea of files on the hard drive.
Over the past 20 years, the commodity hardware world [1] has been slicing away at the fact that several dozen kilobytes of code are being accompanied by hundreds of megabytes of support and literal pounds of physical hardware. Hardware went first with VMs. VMs got lighter and lighter. Lightweight hypervisor solutions sliced away at the heaviness of the VM. Containers slice away at the OS. Things like Kubernates slice away at the idea of a container living somewhere physically.
We're trying to free that several dozen kilobytes of code to be just several dozen kilobytes of code, as flexible and easy-to-deploy as several dozen kilobytes should be, if you weren't mired in the world of hardware and OSes and code and strong physical connections.
(Data storage is more complicated, but in a lot of ways, the same principles are in play.)
Operationally, containers are very exciting. However, in terms of "magic technologies I don't understand", I don't think they're worth stressing about "getting old" or anything. It's mostly "just" a big pile of practical considerations in trying to not just build that world, but in some sense also undo decades of grinding-in of the physical world to our operational considerations. If you want worry about getting old and out-of-date, worry about that let you do something that you couldn't before, like GPU programming or deep learning.
[1]: Which must be specified because mainframes beat us all here decades ago, in a lot of ways.
Cloud orchestration: the logic for how your platform provisions and launches instances (think ec2) into your fleet of resources.
Container orchestration: the logic for how your applications (containers) are distributed across your fleet of instances
Kubernetes: Software that runs on your fleet and manages how/where/when container orchestration happens. There are many players in this space such as mesosphere, swarm, ecs and so on.
1. Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works.
2. Anything that's invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.
3. Anything invented after you're thirty-five is against the natural order of things.
- Douglas Adams
i'd argue that this situation is if you're a developer at a large company or enterprise. developing can be a nightmare because testing and/or deploying is so difficult. sometimes this is by design, sometimes by virtue of incompetence. other times people want to change it, they just don't know how.
tools like this claim they can change this. whether that's true or not. well, that depends on more than just what tools you use but it is part of the equation.
[1] https://www.joyent.com/blog/samsung-acquires-joyent-a-ctos-p...
Please, please continue to develop your public cloud offerings. Having options other that the myopic, me-too, feature-matching, monoculture that is AWS/GCE/Azure is incredibly important.
That said, for my use profile, you guys need to work on your price competitiveness. Hopefully Samsung will inject the necessary cash for economies-of-scale.
I'm sure they will. They are a top contributor to open source:
Wiki: "In journalism, lede is an alternative spelling for lead paragraph."
We use Joyent for nodejs microservices, chiefly any server side DOM manipulations (using jsdom), and it's been solid! Currently evaluating docker container management and will give Triton a go...
Just saying, there's a bit of bias that's probably worth recognizing.
I don't remember the exact Korean wording, but a common phrase said by Samsung engineers (under their breath) in Korea basically translates to "I'll do what you say, but it's only because I'm at <name of a Japanese labor camp that I forget now>." My time there really made me appreciate how much better we have it here!
Can you explain the cultural aspects (and management shortcomings) in more detail for those of us who haven't experienced Korea? Thanks.
That's Samsung proper. Not their incubator setup, which is different. Companies like Mapzen were bought by Samsung, but have a completely different culture.
This is good news for node.js since what this means is more money gets poured into node.js development.
For example Microsoft pouring money into JS seems to have lead directly to Typescript.
https://www.glassdoor.com/Reviews/Samsung-Research-America-R...
This could just be clashes between US and Korean management culture, or perhaps just some instability in a new research lab.
Microsoft is a software company. Samsung is not. If you think they are, then you probably haven't used one of their bloated, buggy phones that they stop updating way too soon.
jk.
There's a reason Samsung seems to be not that great as a software shop. I wrote up a long post on HN almost 2 years ago (God, it's been 2 years!!). I'm reposting the link here.
We were offered either one or three free years of hosting on a Joyent SmartMachine, depending on what your initial investment was. After this free trial, you would be converted to a regular customer.
Adding my voice to the bitter-parade. Joyent wouldn't be where they are today without us early bootstrap investors. It was basically a Kickstarter campaign before Kickstarter existed. Not only do I feel like I failed to receive my early investor benefits, but WORSE off was how the whole situation was handled. We had to beg and plead to get our data back. The company essentially went dark to all of their oldest, most loyal customers.
I totally take the "lifetime" deal with a grain of salt but it wasn't like the company folded because it ran out of money or wasn't successful. It was more like we got successful so we forgot about the little people.
Congrats to all my former colleagues who are absolutely amazing at their jobs and wonderful people to work with. Samsung looks like a very good match. Hope the transition goes well.
So that's how I saw it progressing. Samsung was looking into how to get in on the Internet of Things, stumbled across Manta, and it went from there.
If you want to store objects in a distributed way, why not just use something like RethinkDB, which has proxying/sharding etc. built in? RethinkDB even has a way to execute code with 'js'.
Or for serverless processing, why not AWS lambda or the crop of clones?
I'm not sure where this will take things with Samsung, likely backend services to support their own "store" on their devices separate from Android. If it's internalized, I only hope they release more of their infrastracture tooling, which is very cool to say the least.
It's a good product fit, but the fit is so good that Microsoft basically built a more-complete competitor in the form of Azure. They built Joyent instead of buying it.
LinkedIn was different because Microsoft could never build LinkedIn itself.
[1] https://azure.microsoft.com/en-us/services/app-service/web/
edit: I should have been more specific.. when an open source project is under a company's wing and it gets acquired, you don't know what can happen, even under MIT. Look at express recently. Since it's the under the Node Foundation now, this is not a big deal. Had it happened a short time ago, there may have been further turmoil in the community.
So actually all of Cantrill's worries with Oracle exist with Node.js's license. I am sad when I see large free software projects licensed under MIT, we are doing ourselves a disservice by allowing proprietary software to take advantage of the free software we've developed.
[1] It's part of his Fork Yeah! talk about the fork that is illumos.
"By bringing these two companies together we are creating the opportunity to develop and bring to market vertically integrated mobile and IoT services and solutions that deliver extraordinary simplicity and value to our customers."
Given how well React Native tends to work, and I have to be honest, I like the ecosystem structure better than most apps I've worked on in general, I can see it actually working out better in many cases.
- AOT compilation for a subset of JS: http://www.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-13...
- "A JavaScript engine for Internet of Things": http://samsung.github.io/jerryscript/
Its corporate culture only allows the most cunning, politically savvy person to stay alive and move up the rank, and thus most executives (all if I limit it to small sample of executives I've personally met) fit that model.
And shit literally flows downwards, where goals/promises set by them would be pushed downwards and engineers have to take the burden.
It doesn't help that Korean society is very hierarchal and based on Confucius principles, where you don't usually challenge older persons and/or someone higher in the rank. This is one example that describes serious problem - http://thediplomat.com/2013/07/asiana-airlines-crash-a-cockp....
For those of you who are intrigued and have time, I suggest watching Misaeng with English subtitles (https://www.viki.com/tv/20812c-incomplete-life). Samsung isn't as bad, but the same hierarchy, verbal abuse, social dynamics, and strict rules on paper format exist.
The best outcome would be if they leave Joyent's management and culture alone. But I doubt it.
I also have the first-hand experience of their applying the same "consumer electronics" mentality to completely different business which required high-touch sales.
There is no denying success of Samsung - multi-billion, international corporation. However, Samsung is only good at generating quality hardware products at mass scale. There have not been success in any sort of software and services. Perhaps they are trying to expand beyond their strengths, and I applaud that effort and they actually do need it, since it's only matter of time Chinese companies will catch up and produce as quality products as Samsung, as Samsung did to Sony. I hope it bears fruits. I hope they can allow Joyent to succeed and thrive, and learn from that.
I will see what happens next few years.
"Samsung will immediately benefit from having direct access to Joyent’s technology, leadership and talent. Likewise, Joyent will be able to take advantage of Samsung’s scale of business, global footprint, financial muscle and its brand power."
Their fears have come true (and now it's even happening to Apple). Hardware is an incredibly difficult, risky business, and differentiation is now in the services/software arena. That's why Samsung, in particular, needs this acquisition (and others like it).
Disclaimer: I'm not involved in that space, so maybe they've started?
I don't expect to see Samsung take on GCE, Azure or AWS directly though... and would be really unsure if I were a Joyent customer right now.
That said, since SmartOS is based on illumos, and illumos is designed from the ground up to be portable, running SmartOS on ARM would be cool, since it would open the doors for illumos on embedded devices.
I would so love me an illumos based, iPad Pro-like tablet...
It seems that NodeJS has moved out of Joyent. They have hosted container support that seems to run on solaris, which seems interesting, but a bit too much of buzz-wordy from their website.
I am not very familiar with this, so will be great if someone can explain a little bit. I read the comments around orchestration, but am more interested in Joyent's value proposition.
If anybody deserve it, it's you guys.
