refactor all code that builds csources

[timotheecour]

• follows CIs: attempt to use csources_v1 #16282, refs CIs: attempt to use csources_v1 #16282 (comment)
• refactor all code that builds csources so that changing csources url/commit can be done in a single place (nimBuildCsourcesIfNeeded), without affecting scripts
• updating csources_v1 won't affect nim repo since this PR hardcodes the hash at which to build csources_v1
• sh build_all.sh now does "the right thing" transparently (caches bin/nim_csources_$hash and only rebuilds when needed)
• remove ci/* except ci/funs.sh, which by design has no top-level statement, only reusable bash functions that can be used after . ci/funs.sh
• simplify CI pipelines (as you can see, the diff removes code overall)
• fixes Lock down csources version #11457

design goals

• DRY: hide the way we build csources (bootstrap nim) behind a bash function call, nimBuildCsourcesIfNeeded
• reusable bash functions for CI + build_all (note we still prefer nim scripts but this is for bootstrapping)
• future proof design so that we can build nim at any revision even if csources url/hash is later updated (ie we tie a nim version to a corresponding csources version), allowing tooling eg:
  • git bisect workflows
  • running benchmarks across several nim versions
• we won't need a new git repo csources_v2 etc, instead we can reuse csources_v1 with different commits (or tags/branches if needed for clarity)

notes

• I haven't updated appveyor.yml.disabled, it's outdated and not used anymore
• I haven't updated build_all.bat, it was already outdated (used csources instead of csources_v1), and I don't know if it's still used by windows users (given we have choosenim and that windows users can use bash scripts IIRC, eg as does azure_pipelines.yml); if it still needs to be maintained, we can update it (or even auto-generate it to keep the urls/hash in sync)
• I've removed the caching logic in ci_docs.yml because it wasn't used in other pipelines, it doesn't save much CI time, and it simplifies the pipeline quite a bit; if needed i can revert that change

future work

• remove .travis.yml refs refactor all code that builds csources #17815 (comment) => move travis, appveyor, ci.yml.disabled to unmaintained/ #17828
• consider removing build_all.bat (as explained below) or auto-generating it
• ./koch buildnim --rev:rev to build nim at a specified revision, taking care of bootstrapping from correct csources version (analog of D's https://github.com/CyberShadow/Digger) => see nim-digger: program to build nim at any revision (including forks/PRs) (cf D's digger); useful for git bisect timotheecour/Nim#332
• ./koch bisect --bad:rev1 --good:rev2 --cmd:xxx to simplify running git bisect, and taking care of building nim at which ever revision needed (in particular taking care of using correct csources version as needed), reusing ./koch buildnim => ./koch bisect timotheecour/Nim#702
• along with code APIs that can be reused

links

• csources2 timotheecour/Nim#251
• address CIs: attempt to use csources_v1 #16282 (comment)

[RSDuck]

I haven't updated build_all.bat, it was already outdated (used csources instead of csources_v1), and I don't know if it's still used by windows users (given we have choosenim and that windows users can use bash scripts IIRC, eg as does azure_pipelines.yml); if it still needs to be maintained, we can update it (or even auto-generate it to keep the urls/hash in sync)

I did not too long ago, because it's mentioned readme, so I would assume other people do this as well.

[timotheecour]

well I've updated it in the meantime in a prior commit, please test build_all.bat from this PR

[RSDuck]

everything seems to work well. Is there anything in particular I should test?

[timotheecour]

everything seems to work well. Is there anything in particular I should test?

that's it, thanks!