Enhancement: Small Improvement for Interactively Overriding Local Testing Harness Env Vars

[tzaffi]

Should I keep these changes around and merge harness_overwrite.py? This tinkering work grew out of an investigation of the downstream effects of this go-algorand PR: algorand/go-algorand#4951

Arguments For and Against Merging the PR

• DO NOT MERGE
  • This violates the goal of keeping up.sh as similar as possible across all 4 SDK's
  • Even if we decide that this should exist in all 4 SDK's, it's written in Python, and we can't impose installing Python across the other SDK's
  • It's not useful enough to justify the future maintenance costs
• MERGE
  • It's okay to have small "violations" of the uniformity goal, if there is a clear upside
  • Anything that eases the investigation of downstream implications for changes in go-algorand -even a little- is worthwhile
  • Python is so ubiquitous for shell scripting; we could impose requiring it in all the SDK's - imagine how much easier life would be if we could dispense with shell scripting in favor of python scripting?

Q: What does harness_overwrite.py do?

It makes it slightly easier locally to mod environments that the test harness needs. In particular

1. set INTERACTIVE_TESTING_ENVIRONMENT=1 in .test-env
2. run make harness and follow the prompts.

here are what the prompts look like

Which of the following env vars do you want to modify?

VERBOSE_HARNESS,TYPE,ALGOD_CHANNEL,ALGOD_URL,ALGOD_BRANCH,ALGOD_SHA,NETWORK_TEMPLATE,NETWORK_NUM_ROUNDS,INDEXER_URL,NODE_ARCHIVAL,INDEXER_BRANCH,INDEXER_SHA,SANDBOX_URL,SANDBOX_BRANCH,LOCAL_SANDBOX_DIR,ALGOD_CONTAINER,KMD_PORT,ALGOD_PORT,INDEXER_CONTAINER,INDEXER_PORT,POSTGRES_CONTAINER,POSTGRES_PORT

    (skip for ALL, or provide comma separated)
    
    A typical choice is:
TYPE,ALGOD_URL,ALGOD_BRANCH

CHOICES:

Then I entered the "typical choice" and proceeded with the next prompts:

...

CHOICES:
TYPE,ALGOD_URL,ALGOD_BRANCH
__________________________________________________
Please provide env variable overrides (skip to keep defaults)
TYPE (default `"channel"`):source
ALGOD_URL (default `"https://github.com/algorand/go-algorand"`):https://github.com/tzaffi/go-algorand
ALGOD_BRANCH (default `"master"`):improve-some-goal-errors
__________________________________________________
OVERWRITING
new_env[TYPE]=source (PREVIOUS: env[TYPE]="channel")
new_env[ALGOD_URL]=https://github.com/tzaffi/go-algorand (PREVIOUS: env[ALGOD_URL]="https://github.com/algorand/go-algorand")
new_env[ALGOD_BRANCH]=improve-some-goal-errors (PREVIOUS: env[ALGOD_BRANCH]="master")
__________________________________________________
...

Q: What are the implications for harness_overwrite.py in C.I.?

As long as one remembers not to merge in INTERACTIVE_TESTING_ENVIRONMENT=1 in .test-env,
C.I. will continue running as before. In order to run the same override environment in C.I., you
need to take the following steps:

0. Follow the steps above to locally overwrite test-harness/.env
1. Copy the file portion in the resulting .env file starting from TYPE=... all the way to the end into .test-env
2. Make sure to set INTERACTIVE_TESTING_ENVIRONMENT=0 and OVERWRITE_TESTING_ENVIRONMENT=1
3. Commit these changes and push to github

C.I. should now run against the new harness env variables.

[tzaffi]

"manually" as opposed with using dot_env which I couldn't figure out how to use to read from only one env file and ignore all other env variables.

[tzaffi]

• THE FOLLOWING SHOULD BE DELETED BEFORE MERGING

[bbroder-algo]

zeph how do we feel about this one?

[tzaffi]

zeph how do we feel about this one?

I think it's kind of cool, but someone else should give a spin to make sure it's worthwhile. I list some pros/cons above, the big con being the future maintenance cost.

[algochoi]

I think it is a nice improvement for local testing, and could be worthwhile having it in a long lasting fork or a branch, and reference it in the docs.

I don't feel as strongly about merging this into develop though, for reasons Zeph outlined above. I think having a uniform CI process across all the SDKs seems favorable. If this was implemented/enforced across the four SDKs we manage, I'd be more inclined to merge it in.

[tzaffi]

I think it is a nice improvement for local testing, and could be worthwhile having it in a long lasting fork or a branch, and reference it in the docs.

I don't feel as strongly about merging this into develop though, for reasons Zeph outlined above. I think having a uniform CI process across all the SDKs seems favorable. If this was implemented/enforced across the four SDKs we manage, I'd be more inclined to merge it in.

I'm not inclined to pursue a strong counter-argument recognizing that much of my excitement is due to my personal investment in it. Unless I hear back from more people by the end of the week, I'll close this PR (but will keep a copy in my personal repo for future testing needs).

[tzaffi]

I'm closing this PR as there was insufficient interest in pursuing it further. However, I'm keeping a copy on my personal repo for future reference.