Support icechunk

[jbusecke]

Replacing #96 since we cannot deploy to sandbox from a fork.

Depends on #95 and developmentseed/titiler#1235

Tasks

• Updated zarr to v3
• Modified the test fixture scripts to write v2 and v3 zarr stores explicitly (I retriggered them and committed the new fixtures)
• Add scripts/fixtures for native icechunk
• Deploy and test on real-world native icechunk data
• Add scripts/fixtures for virtual icechunk
• Deploy and test on real-world virtual icechunk data
• Implement auth as pydantic setting
• End-to-end test with deployment
  • I want the following links to work:
    • MUR from native icechunk store
    • MUR from virtual icechunk store
    • NLDAS from virtual icechunk store
    • RASI (HISTORICAL) from virtual icechunk store
• Move @maxrjones comment on dynamically generating fixtures to separate issue and prioritize -> Dynamically build test fixtures #105
• Maybe: Write more focused test for the backend format guesser? -> I think this only becomes important in case we want to factor that functionality out specifically. Currently the code should be well covered in the integration tests here.

[jbusecke]

The deployment is failing due to git missing?.

[jbusecke]

I should probably take those out?

[jbusecke]

Suggested change

print(f"DEBUG: authorize_virtual_chunk_access = {authorize_virtual_chunk_access}")

[jbusecke]

@hrodmn this is ready for a review! Please ignore the tons of files in the test fixtures (sorry they clobber the diff quite a bit).

[hrodmn]

I think this is great - nice work @jbusecke! The only changes I request are to avoid using assertions for raising errors in the actual runtime code.

For context on the configuration and deployment, the value that we set for TITILER_MULTIDIM_AUTHORIZED_CHUNK_ACCESS will be added as a secret value in a SSM secret that gets shared by multiple applications in a given deploy environment, so we may only have to define it once per deployment environment even if multiple applications want to use the same values.

titiler-multidim/.github/actions/cdk-deploy/action.yml

Lines 50 to 61 in 31c20aa

	- name: Get relevant environment configuration from aws secrets
	if: inputs.env_aws_secret_name != ''
	shell: bash
	working-directory: ${{ inputs.dir }}
	env:
	AWS_DEFAULT_REGION: us-west-2
	run: |
	if [[ -z "${{ inputs.script_path }}" ]]; then
	./scripts/sync-env.sh ${{ inputs.env_aws_secret_name }}
	else
	python ${{ inputs.script_path }} --secret-id ${{ inputs.env_aws_secret_name }}
	fi

[jbusecke]

Thanks so much for the review @hrodmn. Ill see if I can get this cleaned up and ready for Monday!

[jbusecke]

I think i addressed all of @hrodmn comments (see inline responses). I did factor out the chunk auth settings into a little helper but that is the only code I added.
EDIT: Just tested all the links above with the newly deployed changes, and they all still work nicely!