-
Notifications
You must be signed in to change notification settings - Fork 4.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Linux Packaging #2622
Comments
/cc @Petermarcu @piotrpMSFT @leecow @bleroy @gkhanna79
Let's take a moment and step back and look at the different pieces of the product. There are three major parts:
We haven't really gone back and done a look across all the packages to understand how we want to name them, but given the above, this would be my straw-man (note I haven't actually talked to anyone about this, so this isn't some official guidance, it's just what I've been thinking recently.)
As far as dependencies go, I imagine that
In the above world, the host and tooling version in place. Shared frameworks can be installed independently at the
A distro maintainer could always update whatever code ships in the shared framework, build a new copy of the shared framework and release an update. The issue above is only an issue for two cases:
There aren't great solutions for either cases. Since these dependencies are pulled down from nuget.org, there's no easy way for folks to do patches without having a revision shipped. The closest workaround would be to have a custom nuget server (or local source) and tell folks to reference that (and then publish packages to that feed which have you fixes). I believe things could be ordered in a way such that your version of the package would "win". For the first case, it's possible that the new Regardless, there's real engineering work to be done here. I'm interested in how distributions handle other package managers. E.g. if I'm using npm, when do I get distro maintained versions of packages and when do I get things from npm.org?
Bertrand plans to produce something, but as I said above, we don't have great guidance today, so if you can help us figure out what makes sense that would be great. |
Seen from the user's perspective, I would expect most developers will want to just install the latest SDK. Users who want to setup a server environment where the tooling is not required will prefer I was thinking the same thing about giving distro package maintainers possibilities to update through a private NuGet feed that wins over nuget.org. Looks like the right thing to do. About npm, as far as I can tell, things are a little different. The packages include npm, node, or both. Npm is just the tooling. Node does contain the standard library, but that pretty much comes as a monolithic thing, not packages. Everything else comes from the public npm repo, or from private npm repos. Those are relatively recent, and I haven't seen signs that they are used for distro-specific patching. I'm not even sure the scenario makes much sense for Node because of the monolithic nature of the SL. If you want, I could look into other platforms that would be closer in nature to .NET. Maybe Java or Go? Of course this will have to be settled more precisely and written down in documentation form. |
One thing I've liked is a package convention I've seen which has "platformname" for the execution runtime, and "platformname-dev" for the development assets for that package. Is that a convention we should follow? It's very intuitive in that people ask the right questions. If someone sees a dockerfile with a -dev package in it they know its for development and they also then look for one that doesn't have -dev in it for production. This is similar to the first proposal @bleroy gave. Java: Java is the most like .NET in that it has a "shared framework". Things like go are different because you get the runtime as an output of your build. Node is somewhat similar in that you can run on a shared node install but it is different in that its not a compiled language and its completely natural to expect a compiler and most of the SDK to be available at runtime. There is a lot of other content here that I didn't get to yet but I wanted to mention that. I'll try to read more of this tonight. |
Maybe:
|
I personally like that pattern |
A couple comments but I think @ellismg has really well articulated much of it. I could see there being a challenge when installing the Am I correct on that being the behavior of the package manager? Is there a way to express that a dependency should be brought in on install but not removed when the referring package changes? Based on this, there may be other ways we should think about this. One of the other options is to couple the sdk with a specific runtime and host to deliver only whole stacks.
update after 2.0 ships:
Hopefully this would leave dotnet-1.1-sdk alone if it were there and just install dotnet-2.0-sdk and its dependencies as well. This is where my package manager knowledge falls short so I'm definitely looking for insight from others :) My goal is that updating the SDK wouldn't remove a runtime that apps on the machine may be depending on. |
Hey everyone. Thanks for the great input and thoughts. @ellismg said:
Would it be fair to say that users will want:
Is this combination is working for 1.0 and 1.1? Is it expected to continue working for 2.0 and foreseeable future? If multiple installations of host are supported, is the
I can describe how some distributions handle what Maven does. Maven is a build tool for Java that does a bunch of things, but I am only going to talk about the pieces that are analogous to Fedora ships with a maven version that pretty much behaves like upstream. So if a user uses Fedora also ships with a variant of maven called We might want to do something similar: the default mode of @Petermarcu said:
Just an FYI, some distributions use I like the A
Most package managers are not that smart. In fact, I don't think anything is ever removed automatically. Both Debian and Fedora remove automatically installed packages on explicit uninstall. |
Good input. Sounds like "-sdk" is the way to go for the name. Sounds like we also don't need "-runtime" because thats what "dotnet" is when it doesn't have "-sdk".
dotnet/core-setup#1 and dotnet/core-setup#2 are correct. For dotnet/core-setup#3 it's unclear to me if over time users will want to be able to get multiple copies of "-sdk". That would depend on the SDK always being backwards compatible. If it ever broke, the fix would be to explicitly install some version of the "-sdk" package. To me, that says that |
To touch on a question in Matt's original post, the current Ubuntu debs are authored to resolve and download dependencies at install time. eg I would want to be very careful about automatically removing past versions. |
@ellismg Does this mean the new package will be
How would the user revert back to 1.0.4 if they ran into issues with 1.0.5?
Correct.
I think we want to have the packages be componentized and something like dotnet-1.0 is what would be equivalent to the Windows bundle we generate (whether, on RHEL, it is a bundle or triggers the chain is an implementation detail).
Is the convention for things like dotnet-sdk to be the version distro maintainer are chosen to be default or is it intended to be latest? I ask this because what I have seen (e.g. with |
There are two components to the package. The name of the package (in my proposal, that's This is in contrast to how I would say we should handle the shared framework. In that world we bake the major and minor number into the package name and bump the version of the RPM when patch releases come out. When a new major or minor version of the SF is released, packagers produce a new package (with a 1.0.0 version).
The same way they revert to an older version of any software that their package maintainer updates if there are issues. Sometimes the rpm's are archived and there's tooling to downgrade a package, sometimes you may need to build it yourself (from the source RPM).
My view on the situation (and I'm sure I'll be corrected by folks more in the know like @omajid) is that packages which need to be installable side by side bake version numbers into their package names. If we think it makes sense to have the ability to install SDKs side by side, then we probably want to include part of the version in the package identity. If we don't expect that's something we want to support (easily) because we believe the move to latest story is what we want everyone doing, then we should not include version information in the package name. |
Matt's entirely correct. A few points that I thought might be worth clarifying. A (distro) package is generally identified by two components: name and version. The version in turn often consists of two components: the upstream version and the distribution version. As an example for python on my system, the name is When it comes time to upgrade packages, the package manager looks at names to find "same" packages and versions to determine if it's an upgrade. For example, if it sees I have So if we call two packages as In other words, if we want to make two packages be installable side-by-side, we should give them different names. If we want to make one package be an upgrade for another, we should give them the same name. Often implicit in this entire convention is that the package stays compatible. A bump from one version to the next should not break anything, including the user experience and the command line interaction. This isn't true for distros with rolling releases, but they announce an upcoming break in advance. So if Given that we want I prefer Also, my understanding is that upgrading never removes a package. There's two notable exceptions. One, a package |
Sorry for the somewhat late reply, I had a bunch of PTO last week and didn't get to sit down long enough to go through this. @bleroy said:
Yes that is what I was thinking about (I don't like the current naming for debian branch...)
@omajid said:
Not even if its dependency no longer exists? I thought that it would remove said dependency. So to sum the whole discussion up and what we could agree on I hope. Please correct me if I mixed something up =) Packages:
This scenario has one right now relevant problem. Someone may want to stick with project.json for a while longer, but as soon as they run (Note that the above Lets please iterate on the above suggested conventions and lets come up with something that could be considered guidelines so we don't have complete chaos between different Linux distributions... Can anyone summon other people involved in packaging for other distros? |
The case that I am referring to is the following:
In this case the dependency on I just tried this out. I created 3 packages:
Then version bump
And to confirm, the package manager knows nothing requires
|
I would prefer if the .NET Core project made a recommendation (of course, distros can ignore it if they want). I personally prefer |
Yes, sdk would be preferable as far as we're concerned. Let's go for that. |
Updated to reflect your preferences, I'm also inclined to it, was just wondering how strong are the dev/devel conventions... Do we have anything else to add to this, or can we write it up somewhere in a better .md file perhaps, as guidelines? Packages:
|
On debian and derived distros, this is true, but only if you use |
Hey everyone, I'm currently maintaining .NET Core on Arch Linux (as AUR packages) and want to chime in on this discussion. Thanks @omajid for notifying me about it. First of all: Arch Linux is not an officially supported distribution by .NET Core, so I (and the maintainers before me) had to jump through some hoops to get it running. Specifically I haven't been able to build .NET Core from source because of the bootstrapping mechanism, but that's an entirely different topic. Currently the packages build the native components from source and replace the corresponding files from an official Ubuntu 16.04 build. (ugh...) There are 4 packages that I maintain:
Notes:
As far as I can see, this pretty much corresponds to what @RheaAyase wrote with the exception of dependencies which I'll adapt accordingly. |
Thanks all, this is very useful. We'll certainly add documentation with recommendations on package naming based on your input. @gegenschall can you give more details about some scenarios where someone would install |
@gegenschall: |
@bleroy If you were to deploy a server you wouldn't need the SDK, right? But this is a case you most likely wouldn't do with Arch + .NET Core as its officially unsupported, you're right. @RheaAyase You are absolutely right! I'll upload |
@gegenschall if you deploy to a server, you should only need the shared framework release, no? |
@bleroy that should depend on the host because that is what decides which version of framework to use for said application you want to run. You can have as many versions of framework as you wish. |
oooh, no, ok, sorry, so @gegenschall's |
I put a draft of the recommendation doc here: https://github.com/bleroy/core-docs/blob/dotnet/build-from-source/docs/core/building/distribution-packaging.md |
@bleroy Looks great.
|
@rakeshsinghranchi @vivmishra to make sure they see this and follow along on the proposed changes. |
@tmds we've discussed this for a month, I don't see why would we not have the |
Also @bleroy this document does not reflect what was discussed here at all. |
That would work, but it is not adding anything useful. Apps require a specific dotnet version. The sdk meta package points to a specific sdk. The specific sdk points to specific dotnet version. So, it's not useful as a dependency. As part of an update, you want to get the fixes for the versions you have installed (patch update). Updating to the latest (major/minor update), installs a version of dotnet you are not using. |
Yes, apps require a specific version, but that's just one scenario. There is value in being able to install "dotnet", and having updates that add the latest to the system. I agree it's not useful as a dependency, but it's useful by itself. I'm fine with making it optional, however. |
How did you reach that conclusion? I miss the rationale for the package when reading this thread. |
"There is value in being able to install "dotnet", and having updates that add the latest to the system". Makes it easier to maintain a runtime environment in a back compat way. "dotnet" without version number is also more discoverable. |
My opinion from my two points of view: As a user I see no reason to have always the latest major.minor on my server. When I update my server-side application's code, to use newer version, then I'll also manually update the framework - always keeping but one installed to save HDD space for economical reasons (I don't have freebie Azure VM) As a developer I would like my SDK to be the latest, and my framework to be -all of them- because I will probably work with more than one at a time with different projects. Based on these two points of view, I see no point in having the |
Yes, those are two important scenarios. There's also the scenario where as a hoster, I want to keep my servers ready to welcome apps written for all versions of .NET Core. This can be achieved in two ways: I can manually install versioned shared frameworks as they come out, or I can install Another is the one where as a a new user, I can easily discover the I do agree however that |
I like the idea of a
Now, what if we rename the
Thoughts? |
That's an interesting idea, especially as the host consists of a "dotnet" executable, but the host remains pretty useless on its own, which is why the package so far has been considered as a dependency only, not as a standalone thing. It also comes with strong constraints on how much it can do without an SDK, and what help messages we can put in there (and how they can be localized). We also can't make assumptions about the name of the If the new user experience is what we want to aim In the end, we seem to be stuck between the idea of having a |
True, I haven't thought of that, however in that case they could just install the sdk which will effectively do the same thing. The only in-production actual use of this that comes to my mind is an university, where every student has an account on some powerful server to run their heavy calculations or projects, etc... They won't cry for the extra 100MBs from having an sdk instead of just the framework...
I think so, and I would be inclined to this solution, if we are to have some Can I assume then, that these are our current go-to names or is there anything else?
|
Yes. I'll remove the |
//bump Is this published already? |
The PR has been submitted: dotnet/docs#2014. Publication date will depend on the doc team's schedule. I'll close this issue when that's done. |
Following the discussion that happened elsewhere on naming of .NET downloads, packages, and container images (https://github.com/dotnet/designs/issues/2), we'd like to make the following change to the package naming guidelines... Instead of just |
Fine by me.
|
Then, maybe we can change |
If you were a new user trying out .net then you would be directed by everyone to |
Any updates for availability on Arch Linux based distros? |
The generic linux-x64 binary for 2.0 (preview) should work on arch. Can you try that? We are also working to make it easier to build .NET Core form source. We have a Fedora package here that you could try to adapt and build on arch too: https://pagure.io/fedora-dotnet/tree/f26 |
I'm sorry, I never reported back. My apologies! Arch packages are using the recommended naming scheme since about 2.0.0-preview1. They are available here: One minor nitpick: |
Not sure how Arch normally does it, but on Fedora-land we have to make pre-release packages have a 0 "fedora release version" just to handle cases like this. Our Preview 2 packages are versioned as |
I just found out about https://github.com/dotnet/cli/issues/4014 which also touched on this. |
@RheaAyase can we close this with the recent update to the packaging doc? https://docs.microsoft.com/en-us/dotnet/core/build/distribution-packaging |
Sorry to dig up an old issue but I am struggling to see how one can install multiple versions through a package manager such as Yes I have read and re-read: https://docs.microsoft.com/en-us/dotnet/core/build/distribution-packaging My understanding is that I should be to install several specific versions of the SDK & Runtime side-by-side. Eg:
However at least for the Fedora repo, it would seem builds have stopped being published in this way.
Sure I can install a specific version like this Am I missing something obvious? Or is this just not possible? |
@brad-jones You should always use software packaged by the contributors to the Linux distribution in question, not by 3rd party or upstream ;) Our packages may contain Fedora specific tweaks and fixes, and you can most certainly install them side by side.
|
@RheaAyase thanks for the tip about the copr repo, I am up and running again. I guess my next question is, then why isn't the copr repo used in the official dotnet core installation instructions at https://www.microsoft.com/net/download/linux-package-manager/fedora28/sdk-2.1.402 |
@brad-jones that is a good question ;p It is something that I suggested many times without any answer at all. Maybe it will change finally, given the recent developments... |
A few things to make clear regarding packaging.
What are we packaging? A single package that contains everything, or two or even three separate packages? (host, framework...)
How do we call it? There are debian based packages with naming as dotnet-host, dotnet-hostfxr, dotnet-sharedframework. Now imagine
dnf install
orapt-get install
- what would the user install when, which takes me to my next question...What are the dependencies of these packages, which would depend on which one?
How do we version it? What would our packages be when it comes to versions? Using the above dotnet-host as example:
dotnet-host-1.0
And how can user combine versions/whatever at the same time? Say Bob has sdk 1.1 some time in the future, and wishes to be able to target framework 1.0 and 1.1. Our understanding is that Bob would have to install a package for said sdk, which as such would have only the latest framework as dependency (or not even that) and Bob would have to optionally add whatever other framework he would like to publish for.
Bonus question. Where does our (packagers) responsibility start? Say we will have to fix something our-distro-specific and pushing it upstream is not good enough. Say we need the fix ASAP. We can of course build it ourselves with the fix, and we would, but what about nuget packages that do not contain this fix, if someone would be targeting their application for our distro, from some other platform? Can we be responsible for this somehow, can we get the fix in faster than with next whatever update in a few months?
Last but not least, I would like to suggest that these could be compiled into nice packaging guidelines for all the various Linux distributions out there.
cc @Fedora-dotnet @omajid @tmds @nmilosev
(I hope that I got everything covered and didn't mix my notes up, please correct me if so...)
The text was updated successfully, but these errors were encountered: