/info: automated versioning by versioneer needs fixing

[yarikoptic]

#221 added versioning by versioneer but seems deployment is not entirely kosher with it since getting

{
  "schema_version": "0.2.0",
  "schema_url": "https://raw.githubusercontent.com/dandi/schema/master/releases/0.2.0/dandiset.json",
  "version": "0+unknown"
}

I guess it depends on how it gets deployed (may be fetch is happening without fetching the git tags or smth like that)

[dchiquito]

huh, I tried this when I deployed it and it worked fine. I will work on this.

[dchiquito]

The problem seems to be that when the project slug is deployed in Heroku, it doesn't have a VCS context, so versioneer can't infer the version. Unfortunately, Heroku uses setup.py develop rather than setup.py build, so versioneer can't rewrite the _version file when Heroku is "building" the app for deployment.

[yarikoptic]

most likely due to shallow clone in deployment. Don't do shallow or do sufficient depth to hit a release (100 commits? 1000? unlikely worthwhile to bother) and it might all start working

[yarikoptic]

an we are now at

$> git describe --tags
v0.0.1-348-gda32e0e

in dandi-api , but I bet (before RF to make staging work etc) we have progressed deployed instance to some state. What state is/was that? it must be tagged!

[yarikoptic]

FWIW, my findings -- deploy action

• builds is in develop mode

Running setup.py develop for dandiapi      

which I believe causes versioneer to not mint "static" version since in develop mode it wants to rely on current git information.

• pushes git to heroku only the HEAD and no tags

To https://git.heroku.com/dandi-api.git
86
   da32e0e..93faf5f  HEAD -> main

to test if having a tag on that remote heroku git would somehow help , I pushed v0.1.1 to that git repo ... no immediate fix for /info but may be after next release/restart something would show up?

but I think it is the develop build is what screwing it up.

[yarikoptic]

let's see what would happen whenever we redeploy some new release, now that I pushed v0.1.1 to that heroku git repo. -staging indeed just sucks code in directly from github and I guess doesn't bother to fetch tags. But I think the core issue is that it relies on -e . in the requirements.txt which then makes it a devel release, thus versioneer doesn't replace dandiapi/_version.py with a "finalized one", and since no git "in deployed" location -- it all doesn't work.
So may be trying to get away from -e . in requirements.txt would provide remedy

FTR: sources for the heroku buildpack having all details: https://github.com/heroku/heroku-buildpack-python.git

[yarikoptic]

ok, installation without -e didn't help

       Building wheels for collected packages: dandiapi, django-allauth, django-minio-storage, click-didyoumean, psycopg2, coreschema, wrapt
         Building wheel for dandiapi (PEP 517): started
         Building wheel for dandiapi (PEP 517): finished with status 'done'
         Created wheel for dandiapi: filename=dandiapi-0+unknown-py3-none-any.whl size=78463 

if only it was possible to augment execution of the builder so it could at least show more detail about the source location :-/

[waxlamp]

Because of how Heroku builds deployed code, there is no VCS context at all (as Daniel mentioned above), so versioneer simply cannot retrieve the correct version from its computations. To be clear: it's not a problem of a shallow clone or anything like that; it's more that the software setup Heroku receives is not a git repository.

I have an ugly solution in mind but it will probably "just work": in our GitHub action that does deployment, we simply run a script that generates a dandiapi/_version.py of our own, injecting the result of git describe --tag into it, then commit it, then let the Heroku deployment action run (which eventually pushes the code to a Heroku remote).

I'm interested to try this because I can't figure out another cleaner strategy, and surprisingly other people out in the world don't seem to care about this. If there is a more kosher way to do what I'm suggesting (it seems that perhaps there's a way involving versioneer itself), or if we figure out how others are maintaining API versions in deployed Heroku apps, I'd like to hear about those and then try the best option.

(attn @yarikoptic, @dchiquito, @brianhelba)

[yarikoptic]

I would advise to address #404 first by making https://github.com/dandi/dandi-api/blob/master/.github/workflows/production-deploy.yml deploy for both instances. This would be beneficial regardless of this issue. And then we get back to this one ;-)

[waxlamp]

I will do that, but I am almost certain it will have no bearing on Heroku's build process.

[dandibot]

🚀 Issue was released in v0.2.18 🚀

[yarikoptic]

I don't see any indication that it was fixed:

$> curl -s https://gui-staging.dandiarchive.org/server-info/ | jq . | grep version
  "schema_version": "0.6.2",
  "version": "0+unknown",
  "cli-minimal-version": "0.14.2",
  "cli-bad-versions": []

$> curl -s -X GET "https://api-staging.dandiarchive.org/api/info/" | jq . | grep version
  "schema_version": "0.6.2",
  "version": "0+unknown",
  "cli-minimal-version": "0.14.2",
  "cli-bad-versions": []

[jjnesbitt]

There was an issue with the deployment, which I'm working on now.

[jjnesbitt]

@yarikoptic if you check again you'll see the correct version is now present in both staging and prod.

[yarikoptic]

confirming -- thank you

$> curl -s https://gui-staging.dandiarchive.org/server-info/ | jq . | grep version
  "schema_version": "0.6.2",
  "version": "0.2.18.post3+g1aa8bb4f",
  "cli-minimal-version": "0.14.2",
  "cli-bad-versions": []
1 21794.....................................:Mon 02 May 2022 11:10:21 AM EDT:.
lena:~
$> curl -s -X GET "https://api-staging.dandiarchive.org/api/info/" | jq . | grep version
  "schema_version": "0.6.2",
  "version": "0.2.18.post3+g1aa8bb4f",
  "cli-minimal-version": "0.14.2",
  "cli-bad-versions": []
1 21795.....................................:Mon 02 May 2022 11:10:28 AM EDT:.
lena:~
$> curl -s -X GET "https://api.dandiarchive.org/api/info/" | jq . | grep version       
  "schema_version": "0.6.2",
  "version": "0.2.18",
  "cli-minimal-version": "0.14.2",
  "cli-bad-versions": []
1 21796.....................................:Mon 02 May 2022 11:10:34 AM EDT:.
lena:~
$> curl -s https://gui.dandiarchive.org/server-info/ | jq . | grep version       
parse error: Invalid numeric literal at line 1, column 12
1 21797 ->1.....................................:Mon 02 May 2022 11:10:44 AM EDT:.
lena:~
$> curl -s https://gui.dandiarchive.org/server-info/                      
Redirecting to https://dandiarchive.org/server-info/%                                               1 21798.....................................:Mon 02 May 2022 11:10:53 AM EDT:.
lena:~
$> curl -s https://dandiarchive.org/server-info/ | jq . | grep version    
  "schema_version": "0.6.2",
  "version": "0.2.18",
  "cli-minimal-version": "0.14.2",
  "cli-bad-versions": []