Skip to content

Linkerd CLI Chocolatey Package#3921

Closed
drholmie wants to merge 2 commits intolinkerd:masterfrom
drholmie:master
Closed

Linkerd CLI Chocolatey Package#3921
drholmie wants to merge 2 commits intolinkerd:masterfrom
drholmie:master

Conversation

@drholmie
Copy link
Contributor

@drholmie drholmie commented Jan 14, 2020

This draft PR aims to fix #3063 by building a chocolatey package for linkerd2's Windows CLI
The package currently exists in the folder windowspkg

To build (on windows system):

  • Install chocolatey
  • cd into windowspkg/linkerd2-cli
  • Run choco pack
  • Run choco install linkerd2-cli -s . , make sure cmd is running as administrator
  • Run refreshenv to add linkerd2-cli path to system
  • To unistall run choco uninstall linkerd2-cli

The following is further work to be done:

  • Fill linkerd2-cli.nuspec appropriately
  • Fill LICENSE.txt and VERIFICATION.txt appropriately

Future work:

  • After approval future commits will add auto update feature for the package

Signed-off-by: Animesh Narayan Dangwal animesh.leo@gmail.com

This draft PR aims to fix linkerd#3063 by building a chocolatey package for linkerd2's Windows CLI
The package currently exists in the folder `windowspkg`

To build (on windows system):
- Install chocolatey
- cd into `windowspkg/linkerd2-cli`
- Run `choco pack`
- Run `choco install linkerd-cli -s . `, make sure cmd is running as administrator
- Run `refreshenv` to add linkerd2-cli path to system
- To unistall run `choco uninstall linkerd2-cli`

The following is further work to be done:
- Fill `linkerd2-cli.nuspec` appropriately
- Fill `LICENSE.txt` and `VERIFICATION.txt` appropriately

Future work:
- After approval future commits will add auto update feature for the package

Signed-off-by: Animesh Narayan Dangwal <animesh.leo@gmail.com>
@adleong
Copy link
Member

adleong commented Jan 14, 2020

Hi @drholmie! I'm not familiar with Chocolatey or windows package management in general so I'm hoping you can help answer some fairly basic questions.

Is the goal here simply to help with installation of the Linkerd CLI on windows (as opposed to solving distribution of the package)? In other words, the workflow would be to clone this repository and then run the above Chocolatey commands to install the CLI?

Pushing a Linkerd package to some online repository is out of scope here, I assume?

It looks like this package works by simply downloading the Linkerd CLI binary from the github releases page. This is a bit unintuitive for me because if I'm installing a tool from within a git repository, my default expectation is that it would be built from the current source. Is this standard practice for Chocolatey packages?

This is part of the draft PR to fix linkerd#3063 adds the `update.ps1` file
to showcase maintaining a chocolatey package for linkerd2's Windows CLI
The package currently exists in the folder `windowspkg`

To run auto update (on windows system):
- Install chocolatey
- Install auto update by running `cinst au`
- cd into `windowspkg/linkerd2-cli`
To showcase its working change the `linkerd2-cli.nuspec` file's `version`
tag to lower than `2.6.1` (Linkerd2's current latest stable release)
- Run `update`

The following is further work to be done:
- Fill `linkerd2-cli.nuspec` appropriately
- Fill `LICENSE.txt` and `VERIFICATION.txt` appropriately
- Add correct information to `update.ps1` to fill `LICENSE.txt` and
`VERIFICATION.txt` appropriately

Future work:
- If approved will start working towards maintaining it as an official package

Signed-off-by: Animesh Narayan Dangwal <animesh.leo@gmail.com>
@drholmie
Copy link
Contributor Author

Added the update.ps1 file to showcase maintaining a chocolatey package for linkerd2's Windows CLI
The package currently exists in the folder windowspkg

To run auto update (on windows system):

  • Install chocolatey
  • Install auto update by running cinst au
  • cd into windowspkg/linkerd2-cli
    To showcase its working change the linkerd2-cli.nuspec file's version tag to lower than 2.6.1 (Linkerd2's current latest stable release)
  • Run update
    (If facing issues in using update try running Set-ExecutionPolicy unrestricted )

The following is further work to be done:

  • Fill linkerd2-cli.nuspec appropriately
  • Fill LICENSE.txt and VERIFICATION.txt appropriately
  • Add correct information to update.ps1 to fill LICENSE.txt and
    VERIFICATION.txt appropriately

Future work:

  • If approved will start working towards maintaining it as an official package

@adleong
Copy link
Member

adleong commented Feb 5, 2020

Thanks, @drholmie. I'm still unclear on a lot of the details here, perhaps because I'm so unfamiliar with package management on windows. I've skimmed the AU docs, but there's a lot of detail there that I don't have context for so it would really help me if you could spell this out in as much detail as possible.

What are the exact steps (including the specific commands) that we need to perform to create the package? What exactly will happen when we run those commands?

What are the exact steps (including specific commends) that we would need to run during a Linkerd release in order to update the package?

And I'm still unclear how the hardcoded version numbers that are checked into this git repository get updated. I know AU is involved somehow but, who actually updates these files and how?

@drholmie
Copy link
Contributor Author

drholmie commented Feb 6, 2020

Thank you for reviewing this again!

What are the exact steps (including the specific commands) that we need to perform to create the package? What exactly will happen when we run those commands?

There isn't that much of a change, just that on top of Linkerd's regular release process (where the cli is added on the release page), we just need to run update to trigger the update.ps1 script, this reads from the release page itself as mentioned here (it only looks for stable releases)

The script then makes the appropriate changes to the chocolateyInstall.ps1 script here
We then push this to the external repo or chocolatey's repo and whenever a user runs choco upgrade linkerd2-cli it will look at the source (external or chocolatey repo) and get the updated version.

What are the exact steps (including specific commends) that we would need to run during a Linkerd release in order to update the package?

If this is added we just need to run update after the packages are added to the releases page. (again do note the current update.ps1 looks only for stable releases)

And I'm still unclear how the hardcoded version numbers that are checked into this git repository get updated. I know AU is involved somehow but, who actually updates these files and how?

the update.ps1 script does it. As mentioned here, this looks at the release page, finds the appropriate information and updates the chocolateyInstall.ps1 file in the specified fields (packageName, url, checksum of chocolateyInstall.ps1) respectively.

To know more about AU in general you can also check out a tutorial here here and example here

@drholmie
Copy link
Contributor Author

Hey there! @adleong just wanted to say one can test this out properly now since a new stable version of Linkerd was recently released. Just run the update.ps1 script and you can notice the changes made to the chocolateyInstall.ps1 file.
Hope this helps :) Do tell if any further work needs to be done on this.

@adleong
Copy link
Member

adleong commented Feb 20, 2020

Thanks, @drholmie! I think @ihcsim is going to take a look at this.

@drholmie
Copy link
Contributor Author

Awesome! Thanks for the help and the update @adleong :)

@ihcsim
Copy link
Contributor

ihcsim commented Feb 21, 2020

@drholmie I will reach out to you next week with some questions around incorporating this into our GH Actions workflow, and publishing the bits on the official Chocolatey repository.

@drholmie
Copy link
Contributor Author

Looking forward to it and will prepare accordingly. Thanks @ihcsim !

@ihcsim
Copy link
Contributor

ihcsim commented Feb 27, 2020

@drholmie I read up a bit on Chocolatey. The first thing that stood out to me is the rigorous moderation process around distributing packages in the community repo. My understanding is that everytime we want to publish a new release, we have to go through the same vetting process. Do you have experience working with the community repo maintainers? Is it a time-consuming process?

Regarding the Automatic Package Updater module, it hinges on being able to run the update.ps1 script in PowerShell, since it's a.... PowerShell module, right? What about if we update your script to point to https://run.linkerd.io/install, instead of the GitHub release page? (I assume that script only works in PowerShell, not Windows cmd.)

As for your other earlier questions:

@drholmie
Copy link
Contributor Author

@drholmie I read up a bit on Chocolatey. The first thing that stood out to me is the rigorous moderation process around distributing packages in the community repo. My understanding is that everytime we want to publish a new release, we have to go through the same vetting process. Do you have experience working with the community repo maintainers? Is it a time-consuming process?

Ah! that's a good catch @ihcsim . Thank you for bringing it up :) I unfortunately don't have experience working with the maintainers (this would be my first choco package), however Linkerd can maintain an external repo for the package, although that comes with its own advantages and disadvantages as explained here. I'll try getting in contact though on chocolatey's gitter channel for more detailed information on the moderation process (if that's alright).

Regarding the Automatic Package Updater module, it hinges on being able to run the update.ps1 script in PowerShell, since it's a.... PowerShell module, right? What about if we update your script to point to https://run.linkerd.io/install, instead of the GitHub release page? (I assume that script only works in PowerShell, not Windows cmd.)

Yup, the update.ps1 script will work in PowerShell, however choco install works both for cmd and PowerShell. The reason the script looks at the release page is because it's actually streaming the webpage and then using regex, figures out the new package information (code) so I don't think it can work by pointing it to https://run.linkerd.io/install.

An interesting thing to note is that even a .NET environment is good to test out these scripts as evident from the chocolatey github action (although to clarify Linkerd can't use the action as, if I understood the code correctly, the action seems to support only 32-bit systems)

As for your other earlier questions:

* LICENSE.txt - See https://github.com/linkerd/linkerd2/blob/master/LICENSE

* VERIFICATION.txt -  The script at https://run.linkerd.io/install has a `validate_checksum` function. We will need to document the Windows equivalence of it.

Awesome! Thank you for the links, will incorporate them if/ when its time to move forward with this PR.

@ihcsim
Copy link
Contributor

ihcsim commented Feb 28, 2020

however Linkerd can maintain an external repo for the package

Yeah, at this point, I prefer not to maintain a server just to hold one package. Looking at the non-commercial server options, it doesn't look like we can just store it on a S3 or GCS bucket like Helm.

(although to clarify Linkerd can't use the action as, if I understood the code correctly, the action seems to support only 32-bit systems)

Yeah, I was hopeful when I saw this action the other day. I'm afraid this will be a blocker, if we can't add this to our CI workflow. Any chance that you can try this out on your fork?

@drholmie
Copy link
Contributor Author

drholmie commented Mar 2, 2020

Thanks for the quick response @ihcsim! Yup will poke around with the action in the fork, although will take time on that unfortunately, hope that's okay.
Regarding the moderation process, reached out on chocolatey's gitter, hopefully will get an answer soon :)

}

Get-ChocolateyWebFile @packageArgs
Install-ChocolateyPath $packageArgs.fileFullPath 'Machine' No newline at end of file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this adds the tools\linkerd.exe path to the global $PATH, right? Can we use the User path type so that the script doesn't require UAC/admin privileges on the machine?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! that's a good catch. Thanks for pointing that out :) Will look into and modify the script to allow for use without admin privileges.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe if you use the Install-Chocolatey* family of commands, ShimGen should make a shim in the ProgramData/chocolatey/bin folder automatically, hence not requiring a modification of the path variable. It's for example how Helm does it:

$packageName = 'kubernetes-helm'
$toolsDir =  "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"
$tempDir = "$toolsDir\temp"

$packageArgs = @{
    PackageName    = $packageName
    Url64bit       = 'https://get.helm.sh/helm-v3.1.1-windows-amd64.zip'
    Checksum64     = '5efc1c7bdea42a17a1ea197a3a2e47c8f1b0a5464894b98ae73a4c0620163040'
    ChecksumType64 = 'sha256'
    UnzipLocation  = $toolsDir
}

# Download and unzip into a temp folder
Install-ChocolateyZipPackage @packageArgs

That zip is a file with an windows-amd64 folder in it with helm.exe, LICENCE and README.md in it. Rather straightforward.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! That's a interesting way to do it. Although currently Linkerd doesn't make a zip folder for the windows package. Will look into this to find analogous ways for how it currently is (if any). Thank you for suggesting this :)

$packageArgs = @{

packageName = 'linkerd'
fileFullPath = "$toolsPath\linkerd.exe"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to set this value from an env var so that user can decide where to download the .exe to?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, if required the script can also accept command line arguments for the path. Should I accommodate for that as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even better! Thanks!

@ihcsim
Copy link
Contributor

ihcsim commented Mar 3, 2020

@drholmie I managed to get the Chocolatey GitHub Actions to run on a separate repository at https://github.com/ihcsim/linkerd-chocolatey/actions/runs/48648586. The workflow generates and uploads the linkerd-stable-2.6.1.nupkg artifact. Can you test to see if the .nupkg file actually works? (I don't have a Windows machine.) What we really want is to update the upload step to push this to the Chocolatey community repository.

Regarding uninstallation, what does chocolateyUninstall.ps1 look like? This should only uninstall the Linkerd CLI from the user's machine, not the Linkerd control plane.

As for upgrade, I think there are two aspects to it. IIUC, the AU is meant to be run on the user's local machine to faciliate auto-upgrade of the Linkerd CLI. The other part lies in the Linkerd CI workflow, where we should be able to write some scripts to sed the release number and checksum in .Nuspec file.

@drholmie
Copy link
Contributor Author

drholmie commented Mar 5, 2020

@drholmie I managed to get the Chocolatey GitHub Actions to run on a separate repository at https://github.com/ihcsim/linkerd-chocolatey/actions/runs/48648586. The workflow generates and uploads the linkerd-stable-2.6.1.nupkg artifact. Can you test to see if the .nupkg file actually works? (I don't have a Windows machine.) What we really want is to update the upload step to push this to the Chocolatey community repository.

Awesome! Thank you for the help on this @ihcsim :) The .nupkg works! To upload it to the repo one can follow this. (We'd also have to make an account as well).

Regarding uninstallation, what does chocolateyUninstall.ps1 look like? This should only uninstall the Linkerd CLI from the user's machine, not the Linkerd control plane.

Chocolatey's uninstall is actually automatic now, one needn't make a separate script for it as before. It only uninstalls the cli from my experience of using it.

As for upgrade, I think there are two aspects to it. IIUC, the AU is meant to be run on the user's local machine to faciliate auto-upgrade of the Linkerd CLI. The other part lies in the Linkerd CI workflow, where we should be able to write some scripts to sed the release number and checksum in .Nuspec file.

Ah so the AU is actually meant for the maintainers of the package to automate the process of making the updated package. So it is the maintainer(s) or organisation that runs the update.ps1, after which they can simply push the package to the community repo (or external source). From the user's side they need to simply run (for example) choco upgrade linkerd2-cli which would then check if the source (community repo or external repo) has an updated Linkerd. If so it installs that, otherwise says the package is up-to date.

Also regarding the moderation process, before getting an answer from the gitter channel, did a little digging with a few packages release pages and their choco release history. The time taken is very varied, and could even extend to a little less than two months, which might be a problem since Linkerd releases stable versions fairly quickly. Would this be a blocking issue?

@ihcsim
Copy link
Contributor

ihcsim commented Mar 9, 2020

We might not need the update.ps1 after all. I added a step to my GH Actions workflow to sed the version defined in the .nuspec file. See here.

Also, I feel like we should just not bother with publishing the .nupkg file to the Chocolatey community repo. If we attach the file to Linkerd's GitHub releases, Windows users can just download it and run choco install linkerd.nupkg, right?

Meanwhile, there are 3 items that I am hoping you can help me with:

  1. Move the scripts to the bin folder. You can follow the folder layout, filenames and .nuspec metadata seen in my test repo
  2. Per previous conversation, update the .nuspec file to accept command line argument to override Linkerd CLI installation path
  3. Update the chocolateyInstall.ps1 to accept user input (env var or command line arg) to override the version number and the checksum value. If the checksum is absent, then the Get-ChocolateyWebFile command should skip the validation. The reason why this is needed is because AFAICT, CI doesn't know what the binaries checksums are

Feel free to add to this PR, or start a new one. Thanks a lot!

@EraYaN
Copy link

EraYaN commented Mar 11, 2020

I feel like having the package in the community repo would be very useful as a user, it stops us from having to do all kinds of manual things for specific packages. (Very much like how it's very neat to have most packages in main debian/ubuntu repos for example). If you pass the first automatic checks AND you don't change your package around too much, the process is similar to other moderated/opinionated package repositories and can be mostly automatic, although you might not want to release versions every day, so only stable ones, otherwise moderators might get fed up with you.

Most of the requirements are about stating license, authors and copyright attributions and they are fairly picky on those. And stuff like are all fields filled and is the package id sane. Plus not flagging VirusTotal. Self contained executables are very straightforward to package.

I ran the install for the nupkg:

The use of .nupkg or .nuspec in for package name or source is known to cause issues. Please use the package id from the nuspec `<id />` with `-s .` (for local folder where nupkg is found).
Chocolatey v0.10.15
Installing the following packages:
linkerd.2.6.1.nupkg
By installing you accept licenses for the packages.

linkerd v2.6.1
linkerd package files install completed. Performing other installation steps.
 The install of linkerd was successful.
  Software install location not explicitly set, could be in package or
  default install location if installer.

Chocolatey installed 1/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Did you know the proceeds of Pro (and some proceeds from other
 licensed editions) go into bettering the community infrastructure?
 Your support ensures an active community, keeps Chocolatey tip top,
 plus it nets you some awesome features!
 https://chocolatey.org/compare

This does not result in linkerd in the path. It also does not prompt to run the install script. It does not find the chocoInstall.ps1 script according to the logs and thus does not run it nor install linkerd.

Helm as example: (https://github.com/synax/chocolatey-packages/tree/master/automatic/kubernetes-helm)

2020-03-11 11:41:17,680 16656 [DEBUG] - Capturing package files in 'C:\ProgramData\chocolatey\lib\kubernetes-helm'
2020-03-11 11:41:17,684 16656 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\kubernetes-helm\helm-v3.1.1-windows-amd64.zip.txt'
  with checksum '43EDDB5E1207ACA9AD67A244B00F3E3F'
2020-03-11 11:41:17,685 16656 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\kubernetes-helm\kubernetes-helm.nupkg'
  with checksum '007ECBDD3B5B2C7C4B67D35248E7E76A'
2020-03-11 11:41:17,686 16656 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\kubernetes-helm\kubernetes-helm.nuspec'
  with checksum '92B8494DF71659253F359C7FF5DB30F2'
2020-03-11 11:41:17,687 16656 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\kubernetes-helm\tools\chocolateyInstall.ps1'
  with checksum '8F5A0D1F826C8FE6DDCBB7762384758E'
2020-03-11 11:41:18,629 16656 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\kubernetes-helm\tools\windows-amd64\helm.exe'
  with checksum 'ACE06BEF62D3F7ED0D6BD077E0BBA181'
2020-03-11 11:41:18,632 16656 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\kubernetes-helm\tools\windows-amd64\LICENSE'
  with checksum '6A18660411AF65E17F370BDBB50E6957'
2020-03-11 11:41:18,633 16656 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\kubernetes-helm\tools\windows-amd64\README.md'
  with checksum 'D19035E20DAA65E3E2B204A5D9AB6B71'
2020-03-11 11:41:18,651 16656 [DEBUG] - Calling command ['"C:\ProgramData\chocolatey\tools\shimgen.exe" --path="..\\lib\kubernetes-helm\tools\windows-amd64\helm.exe" --output="C:\ProgramData\chocolatey\bin\helm.exe"  --iconpath="C:\ProgramData\chocolatey\lib\kubernetes-helm\tools\windows-amd64\helm.exe"']
2020-03-11 11:41:19,246 16656 [DEBUG] -  [ShimGen] [WARN ] Could not extract icon from associated program. Using default. Error:
2020-03-11 11:41:19,248 16656 [DEBUG] -  [ShimGen]   Selected Icon is invalid
2020-03-11 11:41:19,460 16656 [DEBUG] -  [ShimGen] Microsoft (R) Visual C# Compiler version 4.8.3752.0
2020-03-11 11:41:19,462 16656 [DEBUG] -  [ShimGen] for C# 5
2020-03-11 11:41:19,463 16656 [DEBUG] -  [ShimGen] Copyright (C) Microsoft Corporation. All rights reserved.
2020-03-11 11:41:19,464 16656 [DEBUG] -  [ShimGen] This compiler is provided as part of the Microsoft (R) .NET Framework, but only supports language versions up to C# 5, which is no longer the latest version. For compilers that support newer versions of the C# programming language, see http://go.microsoft.com/fwlink/?LinkID=533240
2020-03-11 11:41:19,466 16656 [DEBUG] -  [ShimGen] ShimGen has successfully created 'C:\ProgramData\chocolatey\bin\helm.exe'
2020-03-11 11:41:19,484 16656 [DEBUG] - Command ['"C:\ProgramData\chocolatey\tools\shimgen.exe" --path="..\\lib\kubernetes-helm\tools\windows-amd64\helm.exe" --output="C:\ProgramData\chocolatey\bin\helm.exe"  --iconpath="C:\ProgramData\chocolatey\lib\kubernetes-helm\tools\windows-amd64\helm.exe"'] exited with '0'
2020-03-11 11:41:19,486 16656 [INFO ] -  ShimGen has successfully created a shim for helm.exe
2020-03-11 11:41:19,488 16656 [DEBUG] -   Created: C:\ProgramData\chocolatey\bin\helm.exe
  Targeting: C:\ProgramData\chocolatey\lib\kubernetes-helm\tools\windows-amd64\helm.exe
  IsGui:False

vs

2020-03-11 11:49:08,807 16552 [DEBUG] - Capturing package files in 'C:\ProgramData\chocolatey\lib\linkerd'
2020-03-11 11:49:08,810 16552 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\linkerd\linkerd.nupkg'
  with checksum 'E08D51FD0E9C3E70C3914620A6700E87'
2020-03-11 11:49:08,811 16552 [DEBUG] -  Found 'C:\ProgramData\chocolatey\lib\linkerd\linkerd.nuspec'
  with checksum 'BA376888E0C78AF84ADDA2870DA4052D'

@drholmie
Copy link
Contributor Author

We might not need the update.ps1 after all. I added a step to my GH Actions workflow to sed the version defined in the .nuspec file. See here.

Ah that's cool! Thank you for the help :)

Also, I feel like we should just not bother with publishing the .nupkg file to the Chocolatey community repo. If we attach the file to Linkerd's GitHub releases, Windows users can just download it and run choco install linkerd.nupkg, right?

I'm a little reluctant with this though, not just for the reasons @EraYaN mentioned, but also because its not really the way chocolatey is meant to be used (so to speak). Albeit if the waiting time is bad this is a good workaround it. @EraYaN do you have experience with publishing chocolatey packages? Could you elaborate on the usual amount of time taken with each release vs when a package becomes trusted?

Meanwhile, there are 3 items that I am hoping you can help me with:

1. Move the scripts to the `bin` folder. You can follow the folder layout, filenames and `.nuspec` metadata seen in my test [repo](https://github.com/ihcsim/linkerd-chocolatey)

2. Per previous conversation, update the `.nuspec` file to accept command line argument to override Linkerd CLI installation path

3. Update the `chocolateyInstall.ps1` to accept user input (env var or command line arg) to override the version number and the checksum value. If the checksum is absent, then the `Get-ChocolateyWebFile` command should skip the validation. The reason why this is needed is because AFAICT, CI doesn't know what the binaries checksums are

Feel free to add to this PR, or start a new one. Thanks a lot!

Awesome! Will work on this and add it to the current one, when approved will move along to a proper PR.

Thanks for the help everyone!

@EraYaN
Copy link

EraYaN commented Mar 11, 2020

I have no time frames, it's all human work. So "it varies" is probably close enough, it also does not really matter, it's expected as a repo user that there is some release lag.

The nuspec will need to include the chocolateyInstall.ps1 (most packages seem to do a tools/** glob) right now it does not seem to do so.

EDIT: if the exe is 64 bit the -Url64 argument should be used instead of the -Url argument.

@ihcsim
Copy link
Contributor

ihcsim commented Mar 13, 2020

@drholmie @EraYaN What I really like to see is folks like yourselves work with us to interface with the Chocolatey Windows community, on behalf of the rest of the Linkerd community, in all future releases. The Linkerd team can help you sort out Chocolatey's requirements around license, authors and copyright attributions etc.

In my imaginary world, I picture it as the Linkerd team packages and uploads the .nupkg file for a stable release to the GitHub release page (It's going to be part of our CI process anyway). Someone from the community creates an issue on the Linkerd repository with the title "Publish Linkerd 2.x Chocolatey Windows Package", get @grampelberg to assign the issue to the issue creater, and the issue creater takes ownership from there. We can even help set up a roster for future releases, if you can help recruit your Chocolatey peers to this task.

As an incentive 😉, I am sure @grampelberg and @wmorgan will be happy to send you folks some awesome Linkerd swags, share shoutouts and announcements on social media and maybe even get you to share about the Linkerd Windows release process in an upcoming Linkerd monthly community meeting (where I will nominate you as the next Linkerd heroes!).

@EraYaN
Copy link

EraYaN commented Mar 13, 2020

I mean it might be easier to do the push automatically in your release pipeline. Otherwise it'd just be a webhook to some other repo with some GitHub actions in it, which is fine but might be suboptimal (having the repo outside the main org that is). Although me or @drholmie can set it up I suppose and if you want to move it in-org at some point the repo can be transferred.

@drholmie
Copy link
Contributor Author

@EraYaN I believe it will be added to the release process itself, as discussed previously in the conversations above and since @ihcsim has made an action for it already here.

Also thanks for pointing out the install script issue! My apologies for not clarifying @ihcsim we'll need all the files in the /tools directory for installing the package.

Thanks @ihcsim for the offers and incentive approach!
Regarding recruiting chocolatey peers, we'll mostly require to first submit a few packages and build contacts from there. (at least this is from my side as I've never published a package on chocolatey before) @EraYaN do you have contacts in chocolatey already?

@EraYaN
Copy link

EraYaN commented Mar 13, 2020

I don't have any contacts, I just have submitted packages to similar community package repositories before and it's all not such a big deal. Everyone wants more (good) packages in the repo, especially since there is no support obligation for this one.

@ihcsim
Copy link
Contributor

ihcsim commented Mar 14, 2020

we'll need all the files in the /tools directory for installing the package.

@drholmie I think the value <file> just needs to be a relative path to the .nuspec file. It's more likely that the step run the choco command from the project root folder, causing it to read the content files relative to the root, not the .nuspec file.

Can you try again? https://github.com/ihcsim/linkerd-chocolatey/actions/runs/55697932

Here's my proposal for this PR:

  1. We will limit the scope of this PR to just adding the new scripts.
  2. Fix up all the items we previously discussed in Linkerd CLI Chocolatey Package #3921 (comment) (Feel free to open up a new PR.)
  3. We will open a 2nd PR to implement the CI task. The scope will be limited to packaging the .nupkg file, and uploading it to the GitHub releases page. This will be done by the Linkerd team, as you won't be able to test your changes to the release step
  4. Then we can handle the publication to the Chocolatey process separately. Once the vetting process is completed (few weeks, months?), we can submit a final PR to add a push step to the Linkerd CI

WDYT?

it's all human work

@EraYaN Automating the push is easy. That will happen in the Linkerd CI pipeline. I don't want to deal with the human works every couple weeks.

@EraYaN
Copy link

EraYaN commented Mar 16, 2020

If you don't mess with the moderators too much and steadily release good packages (eg. don't change the stuff and metadata around every submit) it's only human work on their end. Although you do need an email contact point for when shit goes wrong (for example the package no longer installs correctly) and they want you to resubmit.

@drholmie
Copy link
Contributor Author

we'll need all the files in the /tools directory for installing the package.

@drholmie I think the value <file> just needs to be a relative path to the .nuspec file. It's more likely that the step run the choco command from the project root folder, causing it to read the content files relative to the root, not the .nuspec file.

Can you try again? https://github.com/ihcsim/linkerd-chocolatey/actions/runs/55697932

Ah! I see, yup it works now. Thanks @ihcsim for clarifying :)

Here's my proposal for this PR:

1. We will limit the scope of this PR to just adding the new scripts.

2. Fix up all the items we previously discussed in [#3921 (comment)](https://github.com/linkerd/linkerd2/pull/3921#issuecomment-596336439) (Feel free to open up a new PR.)

3. We will open a 2nd PR to implement the CI task. The scope will be limited to packaging the `.nupkg` file, and uploading it to the GitHub releases page. This will be done by the Linkerd team, as you won't be able to test your changes to the release step

4. Then we can handle the publication to the Chocolatey process separately. Once the vetting process is completed (few weeks, months?), we can submit a final PR to add a `push` step to the Linkerd CI

WDYT?

Awesome! Sounds like a plan. Will work on this with that in mind.
Thanks again for all the help!

@ihcsim
Copy link
Contributor

ihcsim commented Mar 16, 2020

Per conversation with @drholmie, we are closing this issue, since it's has accumulated a number of research and testing related comments over the past 2 months and have drifted from master.

@drholmie will open up a new PR to address these items:

  1. Move the scripts to the bin folder, where all the Linkerd build scripts are. You can follow the folder layout, filenames and .nuspec metadata seen in my test repo
  2. Update the .nuspec file to accept command line argument to override Linkerd CLI installation path
  3. Update the chocolateyInstall.ps1 to accept user input (env var or command line arg) to override the checksum value. If the checksum is absent, then the Get-ChocolateyWebFile command should skip the validation. The reason why this is needed is because the CI jobs currently don't have any info on the checksum
  4. The Get-ChocolateyWebFile should use the -Url64 argument
  5. Tthe Install-ChocolateyPath command should use the User path type so that the installation can be run without super user privilege

The scope of the next PR will be limited to just introducing the new scripts to build the .nupkg file. A separate follow-up PR will be submitted to include the new scripts in the Linkerd CI workflow and update the release job to upload the .nupkg file to GitHub releases page.

@ihcsim ihcsim closed this Mar 16, 2020
drholmie added a commit to drholmie/linkerd2 that referenced this pull request Mar 27, 2020
This PR partially fixes linkerd#3063 by building a chocolatey package for Linkerd2's Windows CLI
It adds the build scripts for the Linkerd chocolatey package and based on discussions in
linkerd#3921

The package currently exists in the `bin/win` folder
The `.nuspec` script has been modified for easy integration with the workflow
done by @ihcsim here: https://github.com/ihcsim/linkerd-chocolatey

To build outside of workflow environment (on windows system):
   - Modify `linkerd.nuspec` version tag to an available stable version of Linkerd2's cli
     eg: stable-2.7.0
   - Install chocolatey
   - cd into `bin/win/`
   - Run `choco pack`
   - Run `choco install linkerd -s .`  (make sure cmd/ powershell is running as administrator)
   - Run refreshenv to add `linkerd.exe` path to system
   - To unistall run `choco uninstall linkerd`

Tackling requirements discussed in linkerd#3921
(specifically: linkerd#3921 (comment))

- Followed folder structure as described here: https://github.com/ihcsim/linkerd-chocolatey
- Modified package to accept version through the metadata available in `.nuspec` file
- Modified package to override install path with command line arg/ env variable
    To run with command line argument:
    - Run `choco install linkerd -s . --params="/path:<YOUR_PATH_HERE>"`
    To run with environment variable:
    - Run `$env:linkerdPath = "<YOUR_PATH_HERE>"` in powershell
    Uses default chocolatey install path if none of the above provided.
- Modified package to accept command line args/ env variable for checksum
    To run with command line argument:
    - Run `choco install linkerd -s . --params="/checksum:<VERSION_CHECKSUM>"`
    (Keep params space seperated when specifying multiple)
    To run with environment variable:
    - Run `$env:linkerdCheckSum = "<VERSION_CHECKSUM>"`
    If no checksum provided, default null value assigned and package ignores verification step
- This package is compatible in a non-administrative environment but one must install chocolatey
in a non-administrative path.
(Steps explained here: https://chocolatey.org/docs/installation#non-administrative-install)

Further plans:
This PR is part of a set of planned PRs based on the discussions in
linkerd#3921
(specifically: linkerd#3921 (comment))
drholmie added a commit to drholmie/linkerd2 that referenced this pull request Mar 27, 2020
This PR partially fixes linkerd#3063 by building a chocolatey package for Linkerd2's Windows CLI
It adds the build scripts for the Linkerd chocolatey package and based on discussions in
linkerd#3921

The package currently exists in the `bin/win` folder
The `.nuspec` script has been modified for easy integration with the workflow
done by @ihcsim here: https://github.com/ihcsim/linkerd-chocolatey

To build outside of workflow environment (on windows system):
   - Modify `linkerd.nuspec` version tag to an available stable version of Linkerd2's cli
     eg: stable-2.7.0
   - Install chocolatey
   - cd into `bin/win/`
   - Run `choco pack`
   - Run `choco install linkerd -s .`  (make sure cmd/ powershell is running as administrator)
   - Run refreshenv to add `linkerd.exe` path to system
   - To unistall run `choco uninstall linkerd`

Tackling requirements discussed in linkerd#3921
(specifically: linkerd#3921 (comment))

- Followed folder structure as described here: https://github.com/ihcsim/linkerd-chocolatey
- Modified package to accept version through the metadata available in `.nuspec` file
- Modified package to override install path with command line arg/ env variable
    To run with command line argument:
    - Run `choco install linkerd -s . --params="/path:<YOUR_PATH_HERE>"`
    To run with environment variable:
    - Run `$env:linkerdPath = "<YOUR_PATH_HERE>"` in powershell
    Uses default chocolatey install path if none of the above provided.
- Modified package to accept command line args/ env variable for checksum
    To run with command line argument:
    - Run `choco install linkerd -s . --params="/checksum:<VERSION_CHECKSUM>"`
    (Keep params space seperated when specifying multiple)
    To run with environment variable:
    - Run `$env:linkerdCheckSum = "<VERSION_CHECKSUM>"`
    If no checksum provided, default null value assigned and package ignores verification step
- This package is compatible in a non-administrative environment but one must install chocolatey
in a non-administrative path.
(Steps explained here: https://chocolatey.org/docs/installation#non-administrative-install)

Further plans:
This PR is part of a set of planned PRs based on the discussions in
linkerd#3921
(specifically: linkerd#3921 (comment))

Signed-off-by: Animesh Narayan Dangwal animesh.leo@gmail.com
drholmie added a commit to drholmie/linkerd2 that referenced this pull request Mar 27, 2020
This PR partially fixes linkerd#3063 by building a chocolatey package for Linkerd2's Windows CLI
It adds the build scripts for the Linkerd chocolatey package and based on discussions in
linkerd#3921

The package currently exists in the `bin/win` folder
The `.nuspec` script has been modified for easy integration with the workflow
done by @ihcsim here: https://github.com/ihcsim/linkerd-chocolatey

To build outside of workflow environment (on windows system):
   - Modify `linkerd.nuspec` version tag to an available stable version of Linkerd2's cli
     eg: stable-2.7.0
   - Install chocolatey
   - cd into `bin/win/`
   - Run `choco pack`
   - Run `choco install linkerd -s .`  (make sure cmd/ powershell is running as administrator)
   - Run refreshenv to add `linkerd.exe` path to system
   - To unistall run `choco uninstall linkerd`

Tackling requirements discussed in linkerd#3921
(specifically: linkerd#3921 (comment))

- Followed folder structure as described here: https://github.com/ihcsim/linkerd-chocolatey
- Modified package to accept version through the metadata available in `.nuspec` file
- Modified package to override install path with command line arg/ env variable
    To run with command line argument:
    - Run `choco install linkerd -s . --params="/path:<YOUR_PATH_HERE>"`
    To run with environment variable:
    - Run `$env:linkerdPath = "<YOUR_PATH_HERE>"` in powershell
    Uses default chocolatey install path if none of the above provided.
- Modified package to accept command line args/ env variable for checksum
    To run with command line argument:
    - Run `choco install linkerd -s . --params="/checksum:<VERSION_CHECKSUM>"`
    (Keep params space seperated when specifying multiple)
    To run with environment variable:
    - Run `$env:linkerdCheckSum = "<VERSION_CHECKSUM>"`
    If no checksum provided, default null value assigned and package ignores verification step
- This package is compatible in a non-administrative environment but one must install chocolatey
in a non-administrative path.
(Steps explained here: https://chocolatey.org/docs/installation#non-administrative-install)

Further plans:
This PR is part of a set of planned PRs based on the discussions in
linkerd#3921
(specifically: linkerd#3921 (comment))

Signed-off-by: Animesh Narayan Dangwal <animesh.leo@gmail.com>
cpretzer pushed a commit that referenced this pull request Apr 29, 2020
* Add Linkerd CLI Chocolatey Package

This PR partially fixes #3063 by building a chocolatey package for Linkerd2's Windows CLI
It adds the build scripts for the Linkerd chocolatey package and based on discussions in
#3921

Signed-off-by: Animesh Narayan Dangwal <animesh.leo@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create chocolatey package for Windows CLI releases

4 participants