Epic: SDK v0.50

[SpicyLemon]

Summary

This epic is for identifying and coordinating the work for bumping Cosmos SDK to v0.50.x.

Overview

We will need to make changes across at least two repos, this one and our fork of the SDK. We'll need to do a few key things that then allow us to do some work in parallel. In general, we will be commenting a bunch of stuff out and re-adding it all back in.

As we disable stuff, mark things with todo comments that reference this issue, e.g. // TODO[1760]: Add the marker module back in. These comments should give us a nice list of work to do if we do something like grep -rF 'TODO[1760]' *

When you pick up a checklist item, make an issue from it and assign it to yourself (so that we know someone's working on it). Use the "convert to issue" thing (right side when you hover over a checklist item) for this. It's okay to have a PR that checks off multiple things.

The Plan

1. In this repo, use a main-v0.50 branch that we will create PRs to. We'll start it off by switching the SDK to v0.50.x and commenting out a lot of stuff, then we'll work on fixing everything and putting it all back together.
2. In our SDK fork, use a release-pio/v0.50.x branch that starts with a v0.50.x tag and re-applies our customizations.
3. We can still make changes to main, but those will eventually need to be backported to main-v0.50.
4. Once or SDK fork is ready, tag our version and switch us over to that in the main-v0.50 branch.
5. Once main-v0.50 is done, merge it back into main.

Provenance Repo Plan

These should be done in this order.

• 1. Identify and Finish any key changes needed that aren't related to v0.50.x.
• Create a main-v0.50 branch from our main branch. #1771
• Bump sdk to v0.50.1 in main-v0.50  #1772
• 4. From the resulting TODO[1760]s compile a general list of things to work on and add entries to the TODOs checklist.
• 5. Finish some TODOs in the checklist by making PRs to main-v0.50.
• 6. Backport any PRs in main that aren't yet in main-v0.50.
• 7. Merge main-v0.50 back into main.
• 8. Finish all TODOs in the checklist by making PRs to main.

Provenance Repo Pre-Work

These are the key changes that need to be done before we start the v0.50 work in this repo.

• Add IBC rate limiting protection system #1498

Provenance Repo TODOs

These can (mostly) be done in any order and can be grouped together for PRs as needed. An entry that starts with a tag has related TODOs in the repo.

Example command for finding TODO entries for a tag:

grep --color --recursive --include '*' --include '.*' --exclude-dir .git --exclude-dir vendor --exclude statik.go -F 'TODO[1760]: account'

• account: Add the account keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• auth: Add the auth keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• authz: Add the authz keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• bank: Add the bank keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• capability: Add the capability keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• crisis: Add the crisis keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• distr: Add the distribution keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• evidence: Add the evidence keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• feegrant: Add the feegrant keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• gov: Add the gov keeper stuff back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• ica-host: Add the icahost back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• mint: Add the mint module back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• slashing: Add the slashing keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• staking: Add the staking keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• upgrade: Add the upgrade keeper back to the app. PR [TODOs] sdk-v0.50 app.go #1844
• wasm: Add the wasm keeper back to the app, and fix a couple provwasm sim parts.
• Add the quarantine module back in. #1902
• Add the sanction module back in. #1901
• Get rid of support for now-unsupported db types: ClevelDB, BoltDB and BadgerDB. PR Remove unsupported databases for sdk v0.50. #1862
• Ensure we're creating/handling/getting the codecs correctly See also the "cosmossdk.io/simapp/params".EncodingConfig struct.
• Fix empty rewards events. Delete Legacy Amino stuff.
• Add PreBlocker thing: https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/UPGRADING.md#set-preblocker. PR Add PreBlocker to v0.50. #1859
• Decide if we want anything from the confix command: https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/UPGRADING.md#config-files (nope).
• Update CLI setup for auto-cli stuff: https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/UPGRADING.md#autocli
• Setup textual sign mode: https://github.com/cosmos/cosmos-sdk/blob/v0.50.1/UPGRADING.md#textual-sign-mode
• Migrate the legacy gov props to msgs (might be optionaln, not sure, but needs to be done at some point).
• Register the new gRPC base node queries: https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#grpc
• Remove imports of "cosmossdk.io/simapp": https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#simapp
• Remove emission of EventTypeMessage events (now done automatically).
• Add crisis store: https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#xcrisis. PR Adds store for crisis module #1861
• Add proposers to old proposals (in the upgrade): https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#new-proposalproposer-field Can only affect current, open proposals, and stuff needs to be hard-coded, so we probably don't care about this.
• app-module: Add assertions of new interface types with each module definition, Ensure the BasicModuleManager is correct, and get rid of the global ModuleBasics variable (in app/app.go).
• auto-cli: Ensure we still get the GetAccount and Block commands.
• broadcast: Fix unit tests that depend on submitting a tx and getting a result. https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#broadcast-mode
• cli: Grab ReadPageRequestWithPageKeyDecoded from our fork and switch back to it.
• finalize-block: Refactor a bunch of tests to use FinalizeBlock (was BeginBlock/EndBlock stuff). TODOs finalize-block, event-history, signing,add cosmos.msg.v1.signer to protos txs #1873
• genutil: Add some genutil commands back in.
• gov-cli: Grab GenerateOrBroadcastTxCLIAsGovProp and related stuff from our fork of the SDK and restore its uses.
• proto: Update protos that use sdk.Int and sdk.Dec (now sdkmath .Int and .LegacyDec). Make sure all tx requests have a signer option. Make sure any interface-related options are correct. https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#acceptsimplements_interface-proto-annotations PR [TODOs] sdk-v0.50 app.go #1844
• proto: Update proto depenency retrieval. Update proto generation and management. Use cosmos/gogoprotoinstead of gogo/protobuf: https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#protobuf Regenerate the protos.
• rosetta: See what's up with this now. Add it back in or clean up the TODOs.
• signers: Update MsgCreateTriggerRequest validation because ValidateBasic() can't get signers anymore.
• signing: Fix tests that do their own thing to sign Txs. TODOs finalize-block, event-history, signing,add cosmos.msg.v1.signer to protos txs #1873
• streaming: Ensure that the new streaming stuff is sufficient.
• sync-info: Figure out how to keep this route where it is.
• yaml: Decide if we care about a yaml-related test.
• attribute: Delete the NewHandler and migrate the tests to the keeper.
• marker: Delete the marker handler and migrate handler tests to the keeper.
• metadata: Delete the metadata handler and migrate handler tests to the keeper.
• name: Delete the name handler.
• reward: Delete the reward handler. Clean up some event attribute stuff.
• marker: Delete the marker querier.
• metadata: Delete the metadata querier.
• bank: Remove a couple entries from the marker's expected keeper.
• async-icq: Add this back once they have a v0.50 compatible version.
• bank: Figure out how to use the bank queries in the CLI tests.
• gov: Fix tests/sims that try to get the last proposal or update gov params.
• ibc: Switch back to our fork of ibc-go or clean up the TODO in go.mod.
• ibc-host: Figure out where the ibchost stuff moved and add the stuff back to the app.
• params: Add migration of consensus params to the upgrade. https://github.com/cosmos/cosmos-sdk/blob/v0.47.6/UPGRADING.md#xconsensus
• consensus: Add new module registration for consensus module to the upgrade handler (0.47.x migration)
• params: Create module migrations to move params to their own stores: metadata, marker, name, attribute, msgfees, reward, trigger, ibctransfer, ibc-hooks. Track Migration of Parameters for Metadata, Marker, Name, and MsgFees Modules #1935
• `params: See what's up with ibc-host, ica-host, and icq params. (these params are being migrated via the migrations pattern. We still need to initialize these stores and send them to their respective keepers)
• Identify new gov (and other) params and set them appropriately.
• wasm: Switch back to our fork (or delete the TODO in go.mod).
• sdk: Bump the SDK back to a version from our fork.
• count-authz: Fix some tests once this is back in the SDK.
• event-history: Replace the event history stuff once we have it back in the SDK. TODOs finalize-block, event-history, signing,add cosmos.msg.v1.signer to protos txs #1873
• fee-handler: Replace the fee handler stuff once we have it back in the SDK. PR Remove unsupported databases for sdk v0.50. #1862
• locked-coins: Replace the locked coins stuff once we have it back in the SDK.
• msg-service-router: Switch back to the PioMsgServiceRouter once IMsgServiceRouter is back in the SDK. Ensure ours is still in-line with what the SDK has (plus our alterations). PR [TODOs] sdk-v0.50 app.go #1844
• Call bankkeeper.MigrateParamsProv in the upgrade handlers.
• msg-service-router: registerHybridHandler was added in v0.50 we need to figure out if we need to add msg fee logic to it
• exchange: Restore commented functionality once the better InputOutputCoins is back in the SDK.
• Audit changes to config files and consider switching back to the SDK's client.config. Ensure the pre-upgrade command works as expected.
• Ensure all tests pass again (and aren't just commented out).
• Identify and fix uses of deprecated stuff.
• github: Restore some github-action related stuff.
• Make the linter pass again.
• Make sure the SDK's swagger stuff is up to date, pull it in and make sure all our proto and swagger stuff is up-to-date too.
• Assure all proto txs have option (cosmos.msg.v1.signer) = "<signer>" and remove the GetSigners methods. Also, make sure tx msg has been tested with sim or other method where it has been signed.
• Remove rewards module
• Backport changes made to main since splitting off main-v0.50.
• Review the stargate whitelist to make sure it's got everything we need.
• Create Getparams and SetParams endpoints for the modules that have params, but don't have those endpoints.
• Ensure there are no more TODO[1760] comments.
• Update Github branch protection rules to require the linter and tests to pass again.
• Review the spec docs to ensure that the proto links in the spec docs are correct.
• Add the circuit breaker module.

Cosmos-SDK Fork Plan

These should be done in this order.

• 1. Identify the differences we've made to our fork. Add entries to the TODOs checklist for each.
• 2. Create the release-pio/v0.50.x branch from the most recent v0.50.x tag.
• 3. Finish all the TODOs in the checklist by making PRs to release-pio/v0.50.x.
• 4. Tag a new version.
• 5. Make main-pio match release-pio/v0.50.x.

Cosmos-SDK Fork TODOs

• Start a fresh branch based on upstream's most recent v0.50 tag. -> https://github.com/provenance-io/cosmos-sdk/tree/release/v0.50.x
• Audit our differences for v0.46.13 and add anything missing to this list.
• EventManager: NewEventManagerWithHistory, GetABCIEventHistory, runTx (Fee events and event aggregator), baseapp.Simulate also returns a context, simulation.SimulateFromSeed also returns the last block time, Propagate event history in EndBlock.
• Fee Handler: Add FeeHandler back to baseapp, switch back to IMsgServiceRouter. Prov: Add msg based fee support cosmos-sdk#593
• Streaming: Double check that what's in the SDK is in line with what we have/need.
• CountAuthorization: Add it back in.
• Add the allow_list back to the SendAuthorization.
• Double check the SendEnabled stuff (make sure they didn't put the whole list back in a single params object).
• Put the UpdateDenomMetadata back into the bank module.
• Pruning cmd: get existing viper instead of a new one.
• Send Restrictions: Should be able to use what the SDK has now, but need to double check. If we have a different InputOutputCoins func signature, we'll want that to switch back to that and make our own copy of that func.
• Locked Coins: Add it back in.
• Make sure there's still a CLI command for getting spendable balances.
• In the rootmulti store's buildCommitInfo method, sort the keys by name first.
• Fix genesis account number handling (in the x/auth Genesis stuff).
• Have authz.MsgExec.ValidateBasic() also run ValidateBasic on all its msgs.
• Better handling for when no handler exists for a gov prop.
• Fix group sims' account picker to always pick different accounts.
• Update the bank module migrations since we added one that the SDK doesn't have.

[SpicyLemon]

If you find more thing that need to be done (that you're not just handling as you find it), feel free to update the checklists.

[SpicyLemon]

This command can be used to check compilation of all the unit tests:

go test -mod=readonly  -tags='cgo ledger test_ledger_mock norace' -run='ZYX_NOPE_NOPE_XYZ' ./...

This is an example command for finding specific TODOs:

grep --color --recursive --include '*' --include '.*' --exclude-dir .git --exclude-dir vendor --exclude statik.go -F 'TODO[1760]: account'

[SpicyLemon]

The work for this bump is done, so I'm closing this epic!