Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GenerateReleaseNotes: Can it roll up release notes between deployments to specific environments #34

Closed
AHoran opened this issue Aug 15, 2016 · 32 comments

Comments

@AHoran
Copy link

AHoran commented Aug 15, 2016

Is it possible for the release notes task to roll up the release notes based on the builds that have been generated between deployments to different environments?
For example in my project, every release goes to dev environment, but not all releases make it to test, stage, prod etc.
I would love for the release notes for Prod environment deployment to reflect all the releases/builds that have occurred since last release to Prod.
See:
image

@rfennell
Copy link
Owner

Nice feature to ask for, the issue is walking the history of builds and releases to get the list of builds to include.

As the task currently works with a list of builds the work required will just be to generate the list, not sure how hard that will be. There is no single call to the REST API that will do it.

So it is on the backlog just not sure when I will have time to look into it

@AHoran
Copy link
Author

AHoran commented Aug 16, 2016

Are there any pointers anywhere for developing and testing these tasks? For example, if I wanted to look at this one myself. I can find my way through the powershell, it's the testing, packaging and debugging process that I'm not clear on.

@SeanWilliamson2004
Copy link

This would be good.
Something that allowed for that fact that there can be multiple builds between releases, and that releases are sometimes incomplete or abandoned. TFS 15 now has a facility to list work items for changes between the current release and a selected release.
To go one further would be to roll up details across multiple build sources. TFS 15 doesn't seem to do that.

@gregpakes
Copy link
Contributor

gregpakes commented Nov 17, 2016

I have noticed that the VSTS/TFS front end has this feature. Before you deploy to an environment it will list all the changesets/workitems between the previous release (on that environment) and the new one. Looking at dev tools the changesets URL looks like this:

https://tfs.domain.com/tfs/collection/%GUID%/%GUID%/_api/_versioncontrol/history?__v=5

I am unable to find any MS documentation on this endpoint and therefore assume it is private.

@gigeld
Copy link

gigeld commented Jan 3, 2017

Any news on this issue? Maybe would be easier to have an option to enter a previuos release ID manually (for example the release that was last deployed to PROD).
Thanks

@rfennell
Copy link
Owner

rfennell commented Jan 3, 2017

Sorry not had time to look at developing this feature, real world keeps getting in the way. Other projects have kept me busy. So no ETA for this feature, not even sure of the logic needed behind the scenes

Something like

  • Find previous release of same pipeline

    • add release to list
    • in that release check to see if environment deployed OK
    • if it did stop
    • if it did not keep going back finding and adding releases to the list until it deployed OK or their are no more release
  • Loop across the list of releases (as opposed to just the current release as I do at present) to generate the release notes for all the contained builds

Does that sound like it would work?

@gigeld
Copy link

gigeld commented Jan 4, 2017

If I understand correctly, this logic will work with references to the same environment.
In our project is helpful to know the difference between a release in Test and the last successful deployment in Stage, or even Prod.

Maybe with an (optional) extra filed to enter a previous release ID, the work items list can be generated using:
[...]/_apis/Release/releases/<NEW_RELEASE_ID>/workitems?baseReleaseId=<A_PREVIOUS_RELEASE_ID>

@rfennell
Copy link
Owner

rfennell commented Jan 4, 2017

OK, that makes sense, how does it sound if we by default look for the last successful deployment to the same environment stage, but allow a parameter based override with the name of the environment stage to check against to allow user use case?

@gigeld
Copy link

gigeld commented Jan 4, 2017

Oh yes, that would be a very elegant solution.

@rfennell
Copy link
Owner

rfennell commented Jan 5, 2017

This feature has been shipped in V2.0.4.

There are more notes on its usage in http://blogs.blackmarble.co.uk/blogs/rfennell/post/2017/01/05/Version-20x-of-my-Generate-Release-Notes-VSTS-Task-has-been-released-with-release-rollup-support

@rfennell rfennell closed this as completed Jan 5, 2017
@gigeld
Copy link

gigeld commented Jan 9, 2017

Thank you! I've managed to test it over the weekend and it works very well!

Probably the only comment that I have is the fact that if a release doesn't have any work items, it displays a 'None' string that can be annoying. Not sure how can be dealt with:

image

Any suggestions?

Thanks

@gigeld
Copy link

gigeld commented Jan 9, 2017

Sorry, I found a couple of issues:

  • It misses the last release. So if you have Release1, Release2, Release3 and Release4 where Release4 is the release of the last successful deployment on $stagename, the list of including releases is: Release1, Release2, Release3.

  • Even the Release Notes are generated, the log shows this error repeated a few times:

##[error]System.Management.Automation.MethodInvocationException: Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (404) Not Found." ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
at System.Net.WebClient.DownloadString(Uri address)
at CallSite.Target(Closure , CallSite , Object , Object )
--- End of inner exception stack trace ---
at System.Management.Automation.ExceptionHandlingOps.ConvertToMethodInvocationException(Exception exception, Type typeToThrow, String methodName, Int32 numArgs, MemberInfo memberInfo)
at CallSite.Target(Closure , CallSite , Object , Object )
at System.Management.Automation.Interpreter.DynamicInstruction`3.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)

@rfennell
Copy link
Owner

rfennell commented Jan 9, 2017

So in your example the release notes are triggered by release 4?

As you should be able to see in the logic I thought I always added the triggering release, then look for other relevant releases.

Have you tried running the release in verbose mode (set the global variable of system.debug to the value true). The logs should give more info, at present that error just means 'it failed'.

@rfennell rfennell reopened this Jan 9, 2017
@rfennell
Copy link
Owner

rfennell commented Jan 9, 2017

On the None,None issue. I suspect that these are coming from the actual PowerShell ln.411 of the script adds the word none if there is no data to expand.

I think the best option is to make the value a property, so you can choose to have a blank or any word you want (defaulting to None). I will get that done ASAP

@rfennell
Copy link
Owner

rfennell commented Jan 9, 2017

Just shipped 2.1.1 which allows you to change the text for the None message. Make it blank and that should fix that issue

@rfennell rfennell self-assigned this Jan 9, 2017
@gigeld
Copy link

gigeld commented Jan 10, 2017

Thank you for the fix - 'None' is gone :)

It seems that we are getting that error only on one pipeline so I'll investigate further.

Regarding the last release issue, my explanation was wrong - sorry! So if we have Release 4, 5, 6, 7 with Release 4 the release of the last successful deployment on $stagename, the $releases variable will contain only Release 5, 6, 7.
Here's an example from our pipeline where the highlight shows the release on the successful deployment on the highlighted environment, but the last release in $releases variable is 'Full-SystemTests-04.11.2016-3'
highlightrelease

@SeanWilliamson2004
Copy link

I've not tested the fixed version, but quick tests on the first V2 release suggest it doesn't roll up multiple builds that are rolled into a single release. I've been using release-listingrollups-template.md.
Is this functionality included?
The Deployment summary in TFS was updated to do this.

@rfennell
Copy link
Owner

@gigeld give your scenario of Release 4, 5, 6, 7, where:

  • 7 is the current release triggering the release notes generation
  • 4 is the last successful release to the stage we are monitoring

I would only expect the $release (and release notes files) to contain Releases 5, 6 and 7 i.e.

  • The current release 7
  • The two release since the last successful release 5 and 6

Release 4 should not be included as that was successful and its workitem/commits etc are all listed in its release notes.

So this behaviour is by design - if I understand you correctly

@rfennell
Copy link
Owner

@SeanWilliamson2004 Remember that historic releases are only included since the last successful release to the stage being monitored (by default the current stage).

This means if you last release succeeded the release notes will only contain information for the current release.

If you enable detail logging (set a global variable system.debug set to true) the log file will details out all the choice it made to get the data set

@SeanWilliamson2004
Copy link

SeanWilliamson2004 commented Jan 10, 2017

I'm not sure but I don't think that covers what I'm after.
We use CI for builds but we don't trigger releases from the builds. Releases are run manually periodically so there can be a number of builds between releases.
After I raised it with Microsoft, the TFS RM Deployment summary now shows commits and work items associated with all builds since the release being compared with. So for this issue to be shown, you only need two releases.
For example, below is a screen grab of an RM Deployment summary. In this case there is a release that is being skipped, but there have also been two builds since that release. The commit associated with UC 20 was pushed before the first build and the commit associated with UC 21 before the second build. RM picks up on both now:
image

Release notes produced output:
#Release notes for release BuildConsole1_V2
Release Number : Release-11
Release completed 10/01/17 13:40:51

Changes since last successful releases to ‘’
Including releases:
Release-11

##Builds
###BuildConsole1_V2
Build Number : 20170110.6
Build completed 10/01/17 13:40:28
Source Branch refs/heads/master

###Associated work items

User Story 26 [Assigned by: ] UC 21
###Associated change sets/commits

ID b7342b28434b178fd056423386e1deb90debbf58 Updated Module1.vb

@rfennell
Copy link
Owner

rfennell commented Jan 10, 2017

@SeanWilliamson2004 short answer is no - you are correct the task just does not work that way.

Looking at VSTS itself, on the summary page you highlight, it gets the list of work items that differ between the current release and the current one using the REST call request

https://<instance>.vsrm.visualstudio.com/GitHub/_apis/Release/releases/271/workitems?baseReleaseId=278&%24top=250 

where 271 and 278 are the release IDs, the important item is that the parameter baseReleaseId is not documented in the API, but the whole of RM REST API is only preview.

Similarly a POST request is done to

https://<imnstance>.visualstudio.com/_apis/git/repositories/<ID>/commitsBatch 

(with a suitable playload listing commit SHAs)

This task could be re-written to use this mechanism, but it would be a very serious rewrite. In fact I think it could be another extension as it does the job in a very different way.

All that said, at this time I would be reticent in making this change as the REST calls I need to make are not officially published

@gigeld
Copy link

gigeld commented Jan 10, 2017

@rfennell you're right - the release of the last successful deployment shouldn't be included in the release notes. We needed that information and without much thinking I thought to be the last release in the $releases variable. Sorry, it was my mistake.

@gregpakes
Copy link
Contributor

gregpakes commented Jan 11, 2017

I'm seeing the same 404 as another user above:

2017-01-11T01:21:21.0850965Z ##[debug]Getting associated changesets/commits for build [26949]
2017-01-11T01:21:21.0850965Z ##[debug]Using SystemVssConnection personal access token
2017-01-11T01:21:21.1474966Z ##[debug]Processing: ##vso[task.logissue type=error;]System.Management.Automation.MethodInvocationException: Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (404) Not Found." ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
2017-01-11T01:21:21.1474966Z ##[error]System.Management.Automation.MethodInvocationException: Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (404) Not Found." ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
2017-01-11T01:21:21.1474966Z    at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
2017-01-11T01:21:21.1474966Z    at System.Net.WebClient.DownloadString(Uri address)
2017-01-11T01:21:21.1474966Z    at CallSite.Target(Closure , CallSite , Object , Object )
2017-01-11T01:21:21.1474966Z    --- End of inner exception stack trace ---
2017-01-11T01:21:21.1474966Z    at System.Management.Automation.ExceptionHandlingOps.ConvertToMethodInvocationException(Exception exception, Type typeToThrow, String methodName, Int32 numArgs, MemberInfo memberInfo)
2017-01-11T01:21:21.1474966Z    at CallSite.Target(Closure , CallSite , Object , Object )
2017-01-11T01:21:21.1474966Z    at Invoke-GetCommand(Closure , FunctionContext )

@rfennell
Copy link
Owner

@gigeld great so that is one issue sorted - this thread is getting complex!

@rfennell
Copy link
Owner

rfennell commented Jan 11, 2017

@gregpakes Leaping back a long way in this thread to a, previous point you made, yes the VSTS UI seems to have its own magic way to get the differences, as I mentioned above to @SeanWilliamson2004 I am investigating with MS as to when these APIs might be added to the official API

Secondly on your last post, can you give some more context?

  • I assume that you are running within a release
  • hence there have been a load of previous API calls to get to this point (so the basic API access is OK)
  • The wi/changeset/commit changes for build 26949 are the issue
  • After the logging line the next rest call is to get the work items, so I assume this is the issue (staritng around Ln466)
  • So I would, in an authenticated browser, issue the REST call below to make sure you see data
https://<your instanance>.visualstudio.com/defaultcollection/<your tp>/_apis/build/builds/26949/workitems?api-version=2.0
  • If this returns data try the same call for the changesets and or commits

If this shows data (or the empty data set i.e. not the 4040 error) I think we need to add some more debug logging to show which rest calls are being made.

My gut feeling is I suspect it might be due to some TFVC/Git issue. ** which source control is the artifact using?**

In fact what are the details of the artifact behind this build?

I have modified the logging in 2.2.x to try to help pinpoint this issue

@gigeld
Copy link

gigeld commented Jan 12, 2017

@rfennell it's all good here. Thanks again for your prompt support! Man, you rock! :)

@gregpakes
Copy link
Contributor

@rfennell Thanks for the detailed reply. The 404 is no longer replicatable for me. I suspect the issue is that the builds it is trying to reference have been deleted. Our builds are very big, so we only keep a couple successful ones around. So I am guessing it was trying to get details from the build, but failed with a 404?

@rfennell
Copy link
Owner

That makes sense, maybe should trap that

@rfennell
Copy link
Owner

I am going to close this monster, multi person issue.

If anyone still has issues please open new issues that focus on just that issue, thanks

@gigeld
Copy link

gigeld commented Jan 27, 2017

@rfennell Any luck with trapping that 404 error? In our pipelines is thrown when some builds have been deleted (usually by the retention policy). The Generate Release Notes task will still list all the Work Items found in the available builds but because of the 404 errors,, the task will display in red, as 'Failed'.
Maybe just a warning message would be a better option than 404 error. Thanks!

@rfennell
Copy link
Owner

I have not looked, let me log a new issue #72

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants