Windows Subsystem for Linux Architectural Overview (opens in new tab)

(blogs.msdn.microsoft.com)

193 pointsjackhammons10y ago40 comments

40 comments

The Drawbridge[1] stuff is pretty interesting looking. I really wish MS would ship a simple sandboxing solution, so I can run arbitrary binaries and restrict them. The new Metro app stuff is cute, but doesn't help with the millions of existing binaries. Nor does it seem very user-friendly or useful, for that matter.

1: http://research.microsoft.com/en-us/projects/drawbridge/

voltagex_10y ago

A UWP sandboxing solution was demoed at Build this year and will ship as part of the Anniversary Update (SDK?). Apparently even Age of Empires II worked under it.

contextfree10y ago

Centennial isn't really a sandbox in the sense I think the poster above you meant - while processes have their reads/writes to certain filesystem/registry locations virtualized by default, this isn't a security boundary as they ultimately run at medium trust and can do anything the user can do.

AppContainer is the security sandbox used by modern apps (aka Metro/UWP). It can be used independently of other aspects of the modern app model - e.g., Chrome uses it to sandbox content processes - although this isn't documented very well (which I guess was what GP was complaining about?) and it seems like trying to sandbox apps that weren't designed to be sandboxed, as GP was wanting, would have inherent compatibility problems?

novaleaf10y ago

it's not what you want, but i use Sandboxie to run untrusted stuff. it's a freemium product.

http://www.sandboxie.com/

chris_wot10y ago

This is fascinating :-) I was wondering if they leveraged their environment subsystem framework - nice to have this confirmed!

A long time ago I wrote the Wikipedia article on the architecture of Windows NT. It obviously needs an update, but I think it's still quite relevant and explains in a vastly simplified manner how Windows Fitz together.

It can be found here:

https://en.m.wikipedia.org/wiki/Architecture_of_Windows_NT

There is a block diagram that I think is also helpful:

https://en.m.wikipedia.org/wiki/Architecture_of_Windows_NT#/...

CurtHagenlocher10y ago

It's interesting that this was enabled by the Drawbridge work. I was somewhat under the impression that Drawbridge was dead and/or superseded by other container-related projects. It's often weirdly hard to find out what's going on with some particular Microsoft project or technology, even for those of us who are employees. I was pretty excited about Drawbridge ~3 years ago when I was working with it but it subsequently seemed to have vanished.

Josteniok10y ago

I created an account just to reply to this. You said, "It's often weirdly hard to find out what's going on with some particular Microsoft project or technology..." I was just thinking that very thing today. Why is that?

13of4010y ago

It's not as bad as Amazon, apparently. I've known people there who say everything is compartmentalized. At Microsoft, it's typically only the bleeding edge stuff they keep secret internally.

JdeBP10y ago

Keeping things secret is only a part of the problem. Corporate utter forgetfulness of the past is another. Consider the tale related by Stephen Walli (referenced at https://news.ycombinator.com/item?id=11560510) about the Microsoft people who didn't even know that Microsoft had the SFUA/SUA with Windows NT.

> "I had to explain to the [Windows High Performance Computing team] that they already owned the technology they needed, but to no avail. They couldn't get their head around the idea."

killbrad10y ago

I'd be interested in specifics on either side.

sievebrain10y ago

Organising and propagating information throughout a large organisation takes significant effort.

xgbi10y ago

I cannot stop thinking that soon they will announce a native implementation of Docker on top of all that.

justincormack10y ago

They have not implemented namespaces or cgroups in the Linux emulation, so I doubt it. Docker on Windows will only run Windows binaries.

Sanddancer10y ago

Given they're working with docker and pretty much announced all but that, I think that that's more a guarantee than ponderance.

morekozhambu10y ago

So, its kind of LINE for Windows like WINE for Linux.

jacobush10y ago

There is, or was a program called LINE doing just that.

superobserver10y ago

While this is an interesting technical overview, could anyone give me an idea of how to reinstall lxrun? The blasted thing doesn't work for me anymore and there's basically no documentation on how to fix it other than reinstalling Windows 10. Yes, I've tried lxrun /uninstall /full followed by lxrun /install.

Edit: in fact, I think the problem with my installation has to deal with Windows' NTFS not natively being capable of understanding the lxss file attributes...

tacos10y ago

Delete the directory. And don't touch it from the Windows side ... don't copy anything into it. It's definitely early alpha quality.

superobserver10y ago

By delete, do you mean I can delete %APPDATA%\Local\Lxss from the Windows side? I don't think that will work, as I've already tried that.

cremno10y ago

Have you stopped lxssmanager?

https://msdn.microsoft.com/en-us/commandline/wsl/faq#how-do-...

1 more reply

zrm10y ago

> The primary role of SUA was to encourage applications to get ported to Windows without significant rewrites.

Is this a supported use of the Linux subsystem?

Running native Linux apps directly is great if it works but there are going to be cases where the app would 99% work except for that one thing Linux has and the Linux subsystem doesn't. Maybe Microsoft doesn't provide a tun/tap driver so you need to use TAP-Windows.

It would be convenient to be able to change only that without having to worry about the subtle differences in the Windows version of inet_ntop() and the call to make a socket non-blocking and that Unicode on Windows is UTF-16 instead of UTF-8 and so on.

venomsnake10y ago

Microsoft are insane about providing compatibility when it suits them. It will work - if they see a business case for it.

cududa10y ago

I was just thinking after reading this - sure there were some engineering marvels that enabled this, but for the implementation it was just rote work with the sheer manpower Microsoft can muster up. That in itself is just really impressive.

Edit: giving it more thought, a Mozilla engineer recently demo'd an immediate mode rendering engine that him and one other dev wrote in 9 months, with feature parity of modern browsers. So either way - big engineering effort or a couple '10x developers' - pretty impressive .

the_why_of_y10y ago

Assuming you're talking about WebRender?

http://pcwalton.github.io/slides/webrender-talk-022016/

... which is notable because it is not immediate mode but the first retained mode renderer in a browser. Also, they used Rust to cut down on the time spent debugging threading issues.

1 more reply

mamon10y ago

>> big engineering effort or a couple '10x developers' - pretty impressive

Or, being Mozilla engineer for quite some time he had enough domain knowledge to do that.

Or he found a clever way to reuse good parts of Mozilla code with just enough changes to avoid copyright infringement lawsuit.

2 more replies

netheril9610y ago

So `fork` is still expensive?

ygra10y ago

I think I read somewhere else that they use a better implementation here. Cygwin and SUA being user-mode pieces were limited to a slow fork, but in kernel mode they can emulate things properly.

viraptor10y ago

Does the fork performance really matter these days anymore? I mean comparing from one side to the speed of loading the app itself, fork is usually super cheap; on the other comparing to per-process async event handling, it's so slow pretty much nobody uses it this way for network apps anymore. What's the use case for a fast fork today?

jibsen10y ago

I don't know if it is due to various fork implementations being slow on Windows, but programs that spawn a lot of subprocesses like build tools can be really slow [1].

I would love to see some numbers for those from Bash on Windows.

[1]: https://gist.github.com/jibsen/7ebeddde3bc2bfd421b96ae53a824...

ulber10y ago

One example: The HyPer in-memory database from TUM uses fork to provide fast snapshots for running OLAP queries on fresh data.

[1]: http://hyper-db.de

poooogles10y ago

Redis uses fork to save to disk.

viraptor10y ago

Does fork on windows actually block parent for a long time, or does it only delay the child? I thought it was the latter, which shouldn't affect Redis. (but I may be wrong)

baq10y ago

note that they say 'no documented way'.

Slix10y ago

Does this enable running Docker containers of Linux things in Windows? Would that enable an improvement on Docker's VirtualBox workaround?

slmyers10y ago

I tried to use bash on my windows machine... signed up for insider preview etc.

My machine consistently states a restart will force an update that will allow me to use bash, but it doesn't happen.

I gave up and went back to my ubuntu.

partiallypro10y ago

Not to be a dick, but you're probably not doing it correctly. It asks you to reboot on the spot, for one. When I first tried to do it, I couldn't figure it out without instruction. It's a fairly long process considering it's just a simple item. It should be as easy as switching on Hyper-V...but it's not. Granted it's still in beta.

http://www.howtogeek.com/249966/how-to-install-and-use-the-l...

finishingmove10y ago

I've had the update issue with a "normal" Windows 10 installation (upgraded from 7 without reinstall, which went well, seemingly). It always found updates but then failed (silently) to install them, and so on.

j / k navigate · click thread line to collapse

40 comments

MichaelGG10y ago

1: http://research.microsoft.com/en-us/projects/drawbridge/

voltagex_10y ago

A UWP sandboxing solution was demoed at Build this year and will ship as part of the Anniversary Update (SDK?). Apparently even Age of Empires II worked under it.

contextfree10y ago

novaleaf10y ago

it's not what you want, but i use Sandboxie to run untrusted stuff. it's a freemium product.

http://www.sandboxie.com/

chris_wot10y ago

This is fascinating :-) I was wondering if they leveraged their environment subsystem framework - nice to have this confirmed!

It can be found here:

https://en.m.wikipedia.org/wiki/Architecture_of_Windows_NT

There is a block diagram that I think is also helpful:

https://en.m.wikipedia.org/wiki/Architecture_of_Windows_NT#/...

CurtHagenlocher10y ago

Josteniok10y ago

13of4010y ago

It's not as bad as Amazon, apparently. I've known people there who say everything is compartmentalized. At Microsoft, it's typically only the bleeding edge stuff they keep secret internally.

JdeBP10y ago

> "I had to explain to the [Windows High Performance Computing team] that they already owned the technology they needed, but to no avail. They couldn't get their head around the idea."

killbrad10y ago

I'd be interested in specifics on either side.

sievebrain10y ago

Organising and propagating information throughout a large organisation takes significant effort.

xgbi10y ago

I cannot stop thinking that soon they will announce a native implementation of Docker on top of all that.

justincormack10y ago

They have not implemented namespaces or cgroups in the Linux emulation, so I doubt it. Docker on Windows will only run Windows binaries.

Sanddancer10y ago

Given they're working with docker and pretty much announced all but that, I think that that's more a guarantee than ponderance.

morekozhambu10y ago

So, its kind of LINE for Windows like WINE for Linux.

jacobush10y ago

There is, or was a program called LINE doing just that.

superobserver10y ago

Edit: in fact, I think the problem with my installation has to deal with Windows' NTFS not natively being capable of understanding the lxss file attributes...

tacos10y ago

Delete the directory. And don't touch it from the Windows side ... don't copy anything into it. It's definitely early alpha quality.

superobserver10y ago

By delete, do you mean I can delete %APPDATA%\Local\Lxss from the Windows side? I don't think that will work, as I've already tried that.

cremno10y ago

Have you stopped lxssmanager?

https://msdn.microsoft.com/en-us/commandline/wsl/faq#how-do-...

1 more reply

zrm10y ago

> The primary role of SUA was to encourage applications to get ported to Windows without significant rewrites.

Is this a supported use of the Linux subsystem?

venomsnake10y ago

Microsoft are insane about providing compatibility when it suits them. It will work - if they see a business case for it.

cududa10y ago

the_why_of_y10y ago

Assuming you're talking about WebRender?

http://pcwalton.github.io/slides/webrender-talk-022016/

... which is notable because it is not immediate mode but the first retained mode renderer in a browser. Also, they used Rust to cut down on the time spent debugging threading issues.

1 more reply

mamon10y ago

>> big engineering effort or a couple '10x developers' - pretty impressive

Or, being Mozilla engineer for quite some time he had enough domain knowledge to do that.

Or he found a clever way to reuse good parts of Mozilla code with just enough changes to avoid copyright infringement lawsuit.

2 more replies

netheril9610y ago

So `fork` is still expensive?

ygra10y ago

I think I read somewhere else that they use a better implementation here. Cygwin and SUA being user-mode pieces were limited to a slow fork, but in kernel mode they can emulate things properly.

viraptor10y ago

jibsen10y ago

I don't know if it is due to various fork implementations being slow on Windows, but programs that spawn a lot of subprocesses like build tools can be really slow [1].

I would love to see some numbers for those from Bash on Windows.

[1]: https://gist.github.com/jibsen/7ebeddde3bc2bfd421b96ae53a824...

ulber10y ago

One example: The HyPer in-memory database from TUM uses fork to provide fast snapshots for running OLAP queries on fresh data.

[1]: http://hyper-db.de

poooogles10y ago

Redis uses fork to save to disk.

viraptor10y ago

Does fork on windows actually block parent for a long time, or does it only delay the child? I thought it was the latter, which shouldn't affect Redis. (but I may be wrong)

baq10y ago

note that they say 'no documented way'.

Slix10y ago

Does this enable running Docker containers of Linux things in Windows? Would that enable an improvement on Docker's VirtualBox workaround?

slmyers10y ago

I tried to use bash on my windows machine... signed up for insider preview etc.

My machine consistently states a restart will force an update that will allow me to use bash, but it doesn't happen.

I gave up and went back to my ubuntu.

partiallypro10y ago

http://www.howtogeek.com/249966/how-to-install-and-use-the-l...

finishingmove10y ago

j / k navigate · click thread line to collapse