Map module does not display in jupyter notebooks without cell magic #23
Description
Currently our map module creates polar map by embedding an iframe that references polar.html ; this html file imports and runs several scripts, and expects a three parameter config object to render the map. The config object sets the following:
- Hemisphere (
northorsouth) - Parquet files array (can be local or remote, multiple files or single)
- Initial zoom level (kinda a useless parameter, since we reset the zoom automatically based the data extent of the geometries within the parquet files we load).
What works
Loading the iframe from a jupyter notebook using the %%html cell magic, with the parquet files explicitly encoded in the iframe brackets.
%%html
<iframe
src="../_static/maps/polar.html"
width="100%"
height="600"
frameborder="0"
style="border: 1px solid #ccc; border-radius: 5px;"
onload="this.contentWindow.CONFIG = {pole: 'south', parquetFiles: ['https://storage.googleapis.com/opr_stac/testing/2010_Antarctica_DC8.parquet'], defaultZoom: 3}">
</iframe>
also, loading in myst .md files using our custom directive:
## Map Display -- this works now via custom directive
Below is an Antarctic map showing test data. The map loads GeoParquet files directly in the browser using WebAssembly.
:::{polar-map} ../_static/maps/polar.html
:width: 100%
:height: 600px
:pole: south
:dataPath: https://storage.googleapis.com/opr_test_dataset_1
:parquetFiles: ["test_antarctic_random_walk.parquet"]
:defaultZoom: 3
:::
What doesn't work
A few things that we really expect should work:
Loading in ipython notebook using python
from IPython.display import IFrame, HTML
import json
def create_polar_map(pole='south', parquet_files=None, height=600):
"""
Create an embedded polar map.
Parameters:
-----------
pole : str
'north' or 'south' for Arctic or Antarctic projection
parquet_files : list
List of parquet file paths to display
height : int
Height of the map in pixels
"""
if parquet_files is None:
# Use the correct GCS bucket path
parquet_files = ['https://storage.googleapis.com/opr_test_dataset_1/test_antarctic_random_walk.parquet']
config = {
'pole': pole,
'parquetFiles': parquet_files,
'defaultZoom': 3
}
# Create HTML with embedded configuration
html = f'''
<iframe
src="../_static/maps/polar.html"
width="100%"
height="{height}"
frameborder="0"
style="border: 1px solid #ccc; border-radius: 5px;"
onload="this.contentWindow.CONFIG = {json.dumps(config)}">
</iframe>
'''
return HTML(html)
# Display the map
create_polar_map(pole='south', height=500)This is effectively the exact same call for an iframe, but within the python environment, so it should work, but it doesn't. Short of an issue with the json.dumps(config) call being wonky, I can't tell why we can't display here.
Loading in Markdown, using an iframe
Again, literally the same call that we were using for the %%html cell magic, but trying to render from document.md instead of a document.ipynb. Doesn't work.
Loading in github
Will never work, as github has explicitly disabled embedding and rendering iframes!
Why do we care?
- Right now we're hard coding parquet file values by hand, so there's no programmatic access or way to automate updates to the polar data holding maps
- We can only use .ipynb's to display anything
- We can't call maps interactively in python as part of the workflow for user display of maps. We could display using a different workflow to render the parquet files... but it would be nice to reuse the same display pipeline if possible
What's going on?
We're having trouble passing the config object. There's not really a clean way to do this in markdown... we might be able to get it fixed in python with some more hacking. If we did get it working in python, that would solve 50 to 75% of our issues, as we could automate the variable interpolation and expansion for the parquet files (to automate updates), and would have a pipeline to user facing display of geometries using the same tech stack (if we wanted).
What's the fix?
There might be low hanging fruit with the python call to display iframes, but this isn't as full or robust of a solution as I would like. For one thing, it only works in notebooks-- as opposed to also working in markdown files, which can call python, but don't have access to cell magics.
What we should be doing is templating our html builds correctly. Sphinx, jinja, django, and most other static web builders have fixed this issue ages ago. Basically, you have something like this:
<body>
<div id="map"></div>
<div class="map-info" id="info">Click on features for details</div>
<div class="cors-status" id="corsStatus"></div>
<script type="module">
// Dynamic import path resolution - works with relative paths
const getBasePath = () => {
const pathname = window.location.pathname;
const pathParts = pathname.split('/');
pathParts.pop(); // Remove the HTML filename
return pathParts.join('/') || '.';
};
const basePath = getBasePath();
const wasmModulePath = basePath + '/parquet_wasm.js';
console.log('Loading WASM module from:', wasmModulePath);
const { default: init, readParquet } = await import(wasmModulePath);
import * as arrow from 'https://cdn.jsdelivr.net/npm/[email protected]/+esm';
// CONFIGURATION EMBEDDED AT BUILD TIME
// These values will be replaced by the build process
const CONFIG = {
pole: '{{POLE}}', // 'north' or 'south'
parquetFiles: {{PARQUET_FILES}}, // JSON array of file URLs
centerLon: {{CENTER_LON|default(0)}},
defaultZoom: {{DEFAULT_ZOOM|default(3)}}
};
console.log('Using build-time embedded CONFIG:', CONFIG);
// No need for waitForConfig or getConfig functions anymore!
// The configuration is directly embedded in the HTML at build time
// Setup projections
const PROJECTIONS = {
south: {
code: 'EPSG:3031',
def: '+proj=stere +lat_0=-90 +lat_ts=-71 +lon_0=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +type=crs',
extent: [-3369955, -3369955, 3369955, 3369955],
wmsUrl: 'https://gibs.earthdata.nasa.gov/wms/epsg3031/best/wms.cgi',
wmsLayer: 'BlueMarble_ShadedRelief'
},
north: {
code: 'EPSG:3413',
def: '+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +type=crs',
extent: [-4194304, -4194304, 4194304, 4194304],
wmsUrl: 'https://gibs.earthdata.nasa.gov/wms/epsg3413/best/wms.cgi',
wmsLayer: 'BlueMarble_ShadedRelief'
}
};
// Copy the rest of the functions from polar.html
// (parseWKB, smartFetch, loadParquetFile, etc.)
// ... [rest of the code would go here]
// For now, just show that the config is loaded
document.getElementById('info').innerHTML =
`Config loaded: ${CONFIG.pole} pole with ${CONFIG.parquetFiles.length} file(s)`;
</script>
</body> ...the part of that's doing the actual templating is the double bracket syntax here:
// CONFIGURATION EMBEDDED AT BUILD TIME
// These values will be replaced by the build process
const CONFIG = {
pole: '{{POLE}}', // 'north' or 'south'
parquetFiles: {{PARQUET_FILES}}, // JSON array of file URLs
centerLon: {{CENTER_LON|default(0)}},
defaultZoom: {{DEFAULT_ZOOM|default(3)}}
};
...or even simpler, where we delete the config object entirely and just write the files array directly into the calling function with the {{ files }} variable interpolation specified at build time.
This would make the markdown call look something like a normal .. directive, with a simple list of parameters to replace.
Why hasn't this already been done?
MyST is a fundamental rewrite of a bunch of different technologies-- things like notebook-convert , and sphinx. That means that unlike nb-convert, which actually called sphinx, myst is re-implemented it's own parser to replace sphinx and add additional functionality. This expands what it can do, but to do this it's effectively like a major version bump, in that it doesn't maintain full compatibility with how things were handled in the prior ecosystem. And since it's new... not all of that functionality is well documented yet, even if it's implemented.