Support running Windows Containers on linux #8136
Comments
|
This would definitely would require community to work on. Not something the core developers are likely to work on. |
|
A friendly reminder that this issue had no activity for 30 days. |
|
As much as I don't want to deal with Windows Containers, we have many Windows application workloads in the pipeline that we need to containerize. I would like to use Quay's builders to facilitate some of this work. However, until podman/buildah/libpod support this it won't be possible. |
|
A friendly reminder that this issue had no activity for 30 days. |
|
No progress and no volunteers. |
|
A friendly reminder that this issue had no activity for 30 days. |
|
I am closing this issue in favor of #8452 |
|
#8452 is about running podman on windows/mac. This issue about running windows containers on podman (on linux). Not the same issue. |
|
Thanks, @seanyoung! I missed that you want to run Windows containers on Linux. |
|
A friendly reminder that this issue had no activity for 30 days. |
|
This would require something like kata to run qemu within a container. |
|
@rhatdan no this would not require qemu. It would require a wine. I imagine there would be base layer with wine, and then the windows image as /c/. Then wine runs The wine can be stripped down version as no gui components are required. Windows doesn't allow gui things to run in windows containers either. |
|
@seanyoung crun has the ability to load shared libraries when running containers, we are doing something with this when setting up libkrun. If we had a similar library around wine, we might be able to pull this off. @giuseppe WDYT? Then we could launch crun with a wrun OCI Link. @seanyoung Would this be something you would be interested in working on? |
|
@rhatdan this is something I'd like to work on, but realistically I don't know when I'll find the time. I'm already working on other open source projects. Not enough time in the day is one of life's greatest disappointments. Having said that, I had started working on making work, but this required removing some assumptions across the code. Should pull have a |
|
Hmm looks like |
|
$ podman pull --os windows mcr.microsoft.com/windows/servercore:ltsc2019 |
|
Sadly this failed. Not sure why. |
|
Yes, file links seem to work differently. I haven't quite figured out what the difference is. The image also includes a copy of the windows registry. Lastly the filesystem is case-insensitive. The linux kernel has support for this, but I'm not quite sure what modifications podman needs to set this up. |
|
@rhvgoyal WDYT? |
that is something we could add. Is it possible to load wine from a |
True, but have you explored a maintaiener partnership with the Wine ecosystem? |
|
I agree we don't have people to support this in the RHEL sense, but if we could get Wine experts to contribute and work with this, I think it would be a nice Upstream feature for Podman. |
|
My comment above focused on upstream only. I don't see how we can even do community support without any expertise. I personally prefer to not ship a feature upstream rather than give the impression that incoming issues can be triaged or get any priority. In other words: I don't think we have time for it. |
|
The only way to build expertise is to experiment. We don't have great expertise in WASM and yet crun-wasm exists. If someone steps up to do the work of getting WINE to work. we would expect that person to help with support of Windows clients. If such person disappears we can slowly deprecate the |
|
I've spent a little time looking at how windows containers work. There are two modes Windows containers can be run in, using the windows docker command line tools it looks like this: or In process mode, the container runs like you would expect: the windows kernel services system calls from the container directly. Emulating this mode requires implementing Windows system calls or using wine. The Hyper-V mode is interesting. There a small "utility VM" which runs in Hyper-V (a hypervisor). In the docker container there is a https://forums.docker.com/t/use-of-container-utility-vm-folder-docker/118403/6 |
|
So does it means that it could theoretically boot with qemu hyperv emulation? |
I suspect so. Probably require some debugging to figure out exactly how to set things up. |
|
Are there any topics where a person without the ability to read code could help? |
|
It seems, implementing Windows containers could fill a market with no competitors. Quoting a bit of discussion from kde-devel
If Podman could run Windows containers, there would be no reason to adhere to such arbitrary limitations. Which would make it the preferred (the only one really) choice for a number of tasks. |
|
Hello folks, MSFT person here - first, wanted to make myself available to answer questions on Windows containers. Keep in mind I have close to zero knowledge on Linux. I focus on Windows containers, Kubernetes, Docker, etc. but everything running on the Windows OS (aside from the needed K8s control plane stuff). I understand the desire to run Windows containers on Linux, so let me just put something out there: While that might be possible with Wine (or whatever method) that would not be supported by Microsoft. Of course, something working and being supported are different things and if this works, I'm sure some users will end up using it - I'm more just making this clear than anything else. The other thing I wanted to understand is: Wouldn't be easier to actually have Podman support Windows containers on Windows? Today, docker, moby, and containerd interact with Windows (for Windows containers) via HCS. https://learn.microsoft.com/en-us/virtualization/api/hcs/overview |
Isn't it a supported case? That said, I'm not sure I get your point: do you want a Linux running a Windows VM running a Windows container or how do you want that to work on Linux? And even if so, that still requires buying a license, does it not…? I don't see how that would solve the case. |
|
Seems the idea of windows containers on windows was mentioned before #11809 |
|
AFAIK Podman doesn't support Windows containers, just Linux containers. Did I miss something?
Well, the idea would be for Podman to run on Windows - if that's on a Windows host directly or Linux running a Windows VM, it wouldn't make difference.
Well, Windows containers need to be license anyway. If you manage to run a Windows container on Linux machine, you are using a Windows OS, so that would have to be licensed anyway. |
|
@1480c1 Thanks for noting that. That's unfortunate. I could see huge advantages to having Podman support for Windows containers. |
I think there's a confusion. The issue is about Linux host to run Windows container. E.g. for CI purposes or for testing purposes, or just to run some service… So I am trying to figure out how your suggestion fits into a solution, and running a whole VM so far the only one I see. If that's indeed your suggestion, then needless to say that sounds very heavyweight.
No, people have been running Windows apps on Mac OS and Linux via WINE with no license required for ages. Admittedly, Idk what a Windows container contains, but I've never heard of anyone requiring to pay to launch a container. |
No, I get that. I was proposing a different approach, which based on the comment above has been rejected. I was proposing that Podman could run Windows containers on Windows - they can still be in a CI/CD pipeline, just a different OS. I get this is not the original proposal, but based on the comments I felt this was becoming a harder task than anticipated, hence my suggestion.
That's the thing. If you get an MSI/EXE package that was compiled for Windows and run on Linux, (or MacOS) that's totally fine. You are not using the Windows binaries, but simply translating the APIs. Now, if you run a Windows container, then you have Windows binaries, which need to be licensed. Don't shoot the messenger 😊. |
|
For more information on licensing for Windows containers: https://learn.microsoft.com/en-us/virtualization/windowscontainers/images-eula |
That's for the windows base image. Not all windows containers use these base images, isn't that so ? |
|
Yes, they do. All Windows images use the Windows containers base images from MCR. You might have to track the original base image if you are using .Net for example, but they all do use the base images. If it is a Windows containers, there's no escape from that. |
This "different OS" isn't hackable (due lack of source code), which isn't something developers are typically interested in. So I don't see your suggestion as a viable solution. Even if what you say about the license is true, well… at least the amount of "non-debuggable opaque code" would be reduced as much as possible when there's no actual Windows host around the container. |
|
@vrapolinario If there's interest from Microsoft to champion the effort of Podman support for Windows containers on Windows then the relevant issue is #11809 and since it's locked, a new one would have to be created. The conclusion on that issue was that the Podman community needs Windows engineers to allow Podman to support this. Making yourself available to help with this is a good first step but I imagine it may not be enough unless some volunteer jumps in so MSFT may just need to help implement this themselves. Either way, a new issue mentioning #11809 would be a good way to move this forward since this issue is about a different (likely harder to implement and unlikely to be of Microsoft's interest) feature of supporting Windows containers on Linux (where IMO the main benefit is that it requires A LOT less resources than running a full Windows VM with a Docker/Podman daemon that you can connect to from the Linux host it's running on). |
|
I can't (and won't) debate that. I didn't mean to stear off the original conversation. Was just proposing an alternative which would help Windows apps be used with Podman. Honestly, I'd love to be able to tell Microsoft and Windows customers that they have Podman as an option to run Windows containers. In any case, I'm here if you have question on Windows containers and I can help. And please, keep in mind the licensing. It's not if it's true. If it is a Windows container it requires a Windows license. |
Sounds good. I'll take this internally, but even if there is interest, it won't happen so soon. |
|
I don’t remember my original intent in starting the question but our issue has actually become worse and/or changed but we run “atomic desktops” on development machines for a number of reasons which usually use has a development workflow of toolbox/distrobox for sandboxed dev then eventually on commit whether or not the build is ready to product gets sent to a build matrix where more or less it gets turned turned into a lightweight K8S container that way we’re building to prod always and release on git tags. We do bypass the kernel but not often usually for performance reasons. Docker has come a long way in running rootless and/or reasons we began using CRIO. So we use our host for OSX/Windows and ssh into a big SSH dev instance where we spend most our day using VSCode or VIM over SSH to big Debian boxes. WSL won’t work as we have auditing software (ZScalar) that prevents loopback and we do sometimes need to modify the Linux kernel for performance reasons very specific to us… but in any case 99% of our day is on a giant server and it’s much easier to have our host machine with Teams/etc and get good driver support for things like power management that only a few people will do the Debian laptop and VM into Windows thing for work communication. But the problem we have now might be different. While our use of multi cluster setups is to support internal business apps and deal with large data intake, workflow and light on-prem ML. The rest of the company isn’t probably isn’t even aware and just knows their data is in OneDrive and that Office 365 just works. We will plug into the graph API but that’s simply to update reports, etc. which can be done entirely in Linux. But that’s to just give you an overview. Our problem now is that it is trivial to output the results of 15tb of data process from multiple sources and drop it into essentially a certain teams project folder or watch when files change, etc. PowerPoint is presenting different challenges. This is definitely an edge use case but we can take the OpenXML spec , extract it have a generator that creates go code then “correctly” create guidelines, placeholders, etc from a designer who might’ve taken shortcuts. Then we have a content db where we store assets like logos and variables when we “know” certain things about the project. We found creating things from master templates/layouts/placeholders and not avoiding them even if we programmatically enter data avoids a lot of issues. We give then have a word document (user preference) where they enter data where we can’t fill out and essentially it creates a json schema we can validate and enter into text boxes etc. The point being is that PowerPoint does things like vertical-spacing poorly compared when you have text titles that might overflow to three lines to stay within a column layout and the description box will not adjust to 0.5” from the bottom. Same with grid lines where we can snap to but the user won’t or forget that 5 minutes before a meeting. We are trying to treat this essentially like a website with components that are aware of other components and adjust accordingly. Even worse is grouping or like adding a slide with project team members. Adding one at the last minute that can’t be validated by the server means we need a few simple plugins. Plus we also have the ability to have them mark items as basically “I want to not to stick to the design guide.” We’d really like the ability to test this like we do Linux microservices. Create reasonable build matrix of Windows versions and PowerPoint versions and create test slides. Our syntax is usually correct but typography on Windows and other display issues means we need real copies, create a few test slides and make sure PowerPoint or whatever is looking right. Then run through them and basically create a selenium/web screenshot type setup so we can make sure we aren’t doing any regression or if a designer gives us a new slide what worked on their machine also works on lower DPI screens, different versions etc. what we are doing is simple and our validating the schema is correct we’d like to boot up a lightweight container, open a test file go through the slides take screenshots and make sure we past tests. Accidentally moving a dynamically placed text box off grid isn’t the end of the world and we aren’t looking for pixel perfect. But right now we’d have to build ansible VMs of giant Windows images in HDR, several different resolutions last x versions and then multiply that by the last couple versions of PowerPoints. Containers let us build caches and we can often debloat them. We are a large >50k org and have whatever high level of partnership so licensing or whatever isn’t an issue I’m sure. It’s a pretty simple ask in the end “do our plugins or server side processing doing the same thing work” and while we do this with Linux microservices on every build there’s a huge difference between a 300 mb image with build matrix and cache is huge. Not to mention we use GitHub on-prem to push images up with the runner and some minor custom tooling and it “just works” with GitOps. We don’t need this to create an image though catching an error and rolling back the services and plugin package that power this would be nice. So oddly while we do a lot of complex things we can call the Office365 API to make sure more or less the API didn’t change (some traditional sysadmin or secops person didn’t change permissions randomly), there’s no visual or Windows element to it so far as needing to not boot Windows. What I’d rather not do is create some strange VM workflow maybe with snapshots to speed it up and create an entirely new runner, etc for one sort of minor thing. So really any container that could slim Windows down and/or containerize it would be helpful. Not to kiss ass but the open nature of Azure/Office 365 (even though we do our K8S on-prem means we build out our apps and call out to a simple API means we actually kept to Microsoft). Surely there’s a way internally manual apps are tested for graphical regression. I get that might not be available to us but the UI rendering in Windows differs so much we can’t reliably test it and I’m sure it’s due to years of bowl level crust built up over the years. Even if we don’t have a traditional UI interface and could only take screen shots (using ansible or some scripting to open a file, create a screenshot, move to next template, etc) we can create a Windows host cluster and deploy containers on them. Maybe calling DirectX (or whatever) but I’m not familiar enough with creating graphical apps to know how the rendering engine works. If we generate a UI it’ll be a website but we are mostly Rust/Golang/C++ programmers, so as long as we can do something to record output from the CLI that’s what we are looking for. Sorry for the wall of text and maybe getting off topic but essentially we’re looking to generate screenshots from a container (docker or otherwise) even if it means layering on packages not included in the Windows container, Podman or Docker. I’m sure internally you might literally have bare metal racks that not only have full Windows installs but also different physical graphics cards. But also your day to day devs probably have a slimmed down solution that works 90% of the time. Sorry for the wall of text but thinking this through a bit but I understand not wanting an interactive GUI for licensing issues on containers, but even if we have to call low level com interops to open up an app, open a file, move through slides, take screenshots that might actually be more automated and a better solution. Let me know if you have any ideas. Thanks! |
Had Windows Server Nano not become vaporware, it was probably intended to evolve for use cases such as these |
Not exactly. Nano Server was never intended to support legacy apps, so this would not be a supported scenario for that base image. With that said, Let me clarify that we have the following images: I know you all see this as a monstrous container in terms of size. Keep in mind the goal here is not to compare to any Linux image, but be an advantage versus running the same on a VM. A container with that size, but that can fit into a modern stack is still better than a VM you need to manage the old way. Now, the problem here is running Office apps (Word, PowerPoint) on Windows containers. Currently, they are not supported. I'm not sure if they work on the Server image, you might want to try it. The image you might want to try is: |
|
Hi all this issue opened very long back just want to know how to build windows based docker images using podman build command, is that podman will support? |
|
it would be so cool to see this feature in podman |
|
Well, when are we making progress? |
|
As soon as a contributor steps up to work on it. |
and others
I realise this might be out of scope but I thought I'd raise it so it can be considered.
Windows Containers are slightly modified docker containers, which contain windows executables and the windows registry. These file systems could be mounted with case-insensitive options, and then a very thin wine shim could run the actual binaries. From what I can see, all the necessary executables and dlls are included in the images (e.g. cmd.exe).
https://docs.microsoft.com/en-us/virtualization/windowscontainers/
This would be super-useful to be able to run e.g. windows builds on linux, see and example of a windows dockerfile here: https://github.com/hyperledger/solang/blob/792df33e712f1142da515b7b69f4a4fe7965b5e4/build/build-llvm-windows.dockerfile
The text was updated successfully, but these errors were encountered: