Pyodide 0.28 blog post

content/posts/0.28-release.md

@@ -0,0 +1,266 @@
+---
+title: "Pyodide 0.28 Release"
+date: 2025-07-04
+draft: false
+tags: ["announcement"]
+author: ["Gyeongjae Choi", "Hood Chatham", "Agriya Khetarpal"] # multiple authors
+showToc: true
+TocOpen: false
+draft: false
+hidemeta: false
+comments: false
+# canonicalURL: "https://canonical.url/to/page"
+disableHLJS: true # to disable highlightjs
+disableShare: false
+hideSummary: false
+searchHidden: true
+ShowReadingTime: true
+ShowBreadCrumbs: true
+ShowPostNavLinks: true
+cover:
+    # image: "<image path/url>" # image path/url
+    # alt: "<alt text>" # alt text
+    # caption: "<text>" # display caption under cover
+    relative: false # when using page bundles set this to true
+    hidden: true # only hide on current single page
+---
+
+We are pleased to announce the Pyodide v0.28.0 release.
+
+This release focused on standardizing the Pyodide platform.
+
+## Defining the Pyodide ABI
+
+In October 2024, the CPython steering council [approved restoring Emscripten as
+a tier 3 target for
+CPython](https://github.com/python/steering-council/issues/256), starting from
+Python 3.14. We wrote
+[PEP 776 – Emscripten Runtime support](https://peps.python.org/pep-0776/)
+and
+[PEP 783 – Emscripten Packaging](https://peps.python.org/pep-0783/)
+in order to standardize the Emscripten target for CPython.
+
+PEP 783 aims to standardize the binary interfaces that Pyodide packages should
+follow, helping ensure compatibility with current and future versions of
+Pyodide. Our plan is to have one ABI per Python version. This means that
+packages built for a particular version of Pyodide will be compatible with all
+Pyodide versions that have the same version of Python.
+
+As part of this effort, we defined [the Pyodide ABI](https://pyodide.org/en/stable/development/abi.html). This should help
+people using their own build tooling to ensure that their packages are
+compatible with our ABI. Of course, most people will continue to
+use `pyodide-build`.
+
+This is a crucial step towards enabling the Pyodide ecosystem to develop with
+greater independence from Pyodide runtime releases. If PEP 783 is approved,
+we will be allowed to upload wheels with the
+`pyodide_${YEAR}_${PATCH}_wasm32` platform tag to PyPI.
+
+### Building binary packages compatible with the Pyodide ABI
+
+Building a package with `pyodide-build` or `cibuildwheel` will automatically
+produce ABI-compliant packages (or fail to build). If you use custom build
+toolchains, such as `maturin` for Rust projects, please consult the [Pyodide ABI
+documentation](https://pyodide.org/en/stable/development/abi.html#pyodide-2025-0-under-development)
+to ensure your packages meet the necessary compatibility standards.
+
+## Decoupling packages from the Pyodide runtime
+
+Pyodide is a Python distribution, which means it includes a set of pre-built
+packages that are deployed together with the Pyodide runtime. In Pyodide's early
+stages, this approach was practical due to the challenges of getting packages to
+run in the browser, ensuring that all components were built and tested together.
+
+However, as both the Pyodide and WebAssembly ecosystems have matured, this
+integrated approach has become less sustainable:
+
+From the user's perspective, accessing packages not included in the Pyodide
+distribution often meant waiting for the next Pyodide release, a process that
+could take several months.
+
+From the maintainers' perspective, every commit to the Pyodide runtime repository
+triggered a rebuild and retest of over 250 packages. Even a minor code change
+could result in a CI run exceeding four hours.
+
+In this release, we've taken a significant step by unvendoring packages from the
+Pyodide runtime repository.
+
+All packages are now built in the separate
+[pyodide/pyodide-recipes](https://github.com/pyodide/pyodide-recipes)
+repository. The main
+[Pyodide runtime repository](https://github.com/pyodide/pyodide) now contains
+only the packages that are essential for testing the runtime. This modification
+will enable us to release sets of packages separately and more frequently,
+independent of the Pyodide runtime's release schedule.
+
+In the future, we are hoping that PEP 783 will be approved so people can upload
+their Pyodide wheels to PyPI and use them from there.
+
+## Python 3.13 support and disabled packages
+
+Pyodide 0.28.0 is built with Python 3.13 and a new ABI based on Emscripten
+4.0.9.
+
+Some packages that were previously included in Pyodide 0.27.X are disabled in
+Pyodide 0.28.0. Most of these are disabled because the wheel we used for Pyodide
+0.27 was built externally to our tools by the package maintainers and we are
+waiting on them to build a new version.
+
+The following packages are disabled because we are waiting on their maintainers
+to build a version for the updated ABI:
+
+- arro3-compute
+- arro3-core
+- arro3-io
+- duckdb
+- osqp
+- polars
+- pyarrow
+
+The following packages are disabled for other reasons:
+
+- cartopy
+- gensim
+- geopandas
+- pygame-ce
+- pyproj
+- zarr
+
+These packages will be re-enabled when we can resolve the issues with them. We
+welcome contributions from the community to help us with this.
+
+## A new Matplotlib backend for Pyodide
+
+For years, Pyodide relied on a custom Matplotlib backend (`wasm_backend`) to
+render plots directly in your browser. This backend was developed by the creator
+of Pyodide, [Michael Droettboom](https://droettboom.com/), who was also a core
+developer of Matplotlib.
+
+Another backend, [the `html5_canvas` backend](https://blog.pyodide.org/posts/canvas-renderer-matplotlib-in-pyodide/), was developed by Madhur Tandon as a part of a [Google Summer of Code
+2019 project](https://summerofcode.withgoogle.com/archive/2019/projects/4683094261497856).
+
+However, these backends hadn't been maintained for a long time, and without
+dedicated expertise among our core developers, they became increasingly incompatible
+with newer Matplotlib versions. This made it tough to keep up with new features and
+critical bug fixes.
+
+In this release, we have deprecated these custom backends, and [replaced them](https://github.com/matplotlib/matplotlib/pull/29568) with
+a patched version of
+[the `WebAgg` backend](https://matplotlib.org/stable/api/backend_webagg_api.html),
+one of the official browser-based Matplotlib backends. This new backend provides
+a more stable and feature-rich experience for rendering Matplotlib plots in the browser.
+
+A huge thank you to [Ian Thomas](https://github.com/ianthomas23), a Matplotlib
+core developer and JupyterLite maintainer, who wrote and maintains this new
+backend.
+
+## Other improvements
+
+### Standardized package loading with runtime paths
+
+This release introduces support for runtime paths in Emscripten modules, which
+allows us to correctly locate shared library dependencies.
+
+If a Python binary extension `somebinmod.so` depends on a shared library,
+`libsomedep.so`, this information will be included in the dynamic loader section
+of `somebindmod.so`. The dynamic loader will search for `libsomedep.so` on the
+`LD_LIBRARY_PATH`.
+
+Python wheels vendor their shared libraries, so if `somebinmod.so` is contained
+in `somepackage` then `somepackage.whl` will include a folder called
+`somepackage.libs` with `libsomedep.so` inside. However, `somepackage.libs`
+cannot be added to the `LD_LIBRARY_PATH` because we only want to search that
+directory when opening shared libraries in `somepackage`.
+
+Instead, each dynamic library has its own dependency search path called the
+runtime path. This is information included in the dynamic loader section of a
+shared library indicating where its dependencies should be located. In this
+case, `somebinmod.so` would have an entry saying to look in `somepackage.libs`
+for its dependencies.
+
+Previously, WebAssembly shared libraries files did not support runtime paths, so
+we had to use a custom patch for the Emscripten dynamic loader to apply the rule
+that `somepackage.libs` should be searched when loading libraries from
+`somepackage`. This patch exposed us to extra bugs and prevented us from being
+able to upstream fixes to the Emscripten dynamic loader. It also forced us to
+load dynamic libraries eagerly rather than lazily.
+
+We added runtime path to the WebAssembly specification for the shared library
+format, to the llvm WebAssembly object parser and linker, and to the `emcc`
+linker. We also updated the dynamic loader to use this information. Finally, we
+updated `pyodide-build` to emit the new runtime path data and we removed our
+patch to the dynamic loader.
+
+### Increased adoption of JavaScript Promise Integration (JSPI)
+
+JavaScript Promise Integration (JSPI) officially became a Stage 4 finished
+proposal on April 8, 2025, and Chrome 137 (released May 27, 2025) now supports
+JSPI by default, without any experimental flags.
+
+Pyodide has been a long-time experimenter with JSPI. We turned on several key
+JSPI features by default in Pyodide 0.27.7. This means you can now use
+`asyncio.run()` and `loop.run_until_complete()` in Pyodide to execute Python
+code in the browser to block for asynchronous operations. Previously, this
+capability was gated by the `enableRunUntilComplete` flag in `loadPyodide()`.
+Now, if your browser supports JSPI, these features are enabled automatically.
+See [our blog post about JSPI in Pyodide](https://blog.pyodide.org/posts/jspi/)
+for more information.
+
+### Support for `null` in the Python/JavaScript Foreign Function Interface
+
+As much as possible, the Pyodide foreign function interface tries to ensure that
+values round trip: if a JavaScript value is passed to Python and then back to
+JavaScript, it should come back `===` to the original value. Previously, both
+`null` and `undefined` were converted to `None` and `None` was converted to
+`undefined` so `null` would not round trip. This made it impossible to use
+JavaScript APIs that treat `null` and `undefined` differently.
+
+Fixing this was more difficult than it would seem. When JavaScript values are
+passed into C, we represent them as a WebAssembly `externref`. We want to use a
+special value to indicate that an error occured. There is a special WebAssembly
+instruction called `ref.is_null` to check whether an `externref` is `null`. All
+other `externrefs` were opaque to WebAssembly, and to find out about their
+identity we needed to call out to JavaScript. Calling out to JavaScript to test
+for an error is slow (we measured a 2-3% performance hit for this) so we used
+`null` to signal an error internally.
+
+In order to prevent a `null` that came from the user from being misintepreted as
+an error signal, we converted all `null` to `undefined` before passing them into
+C. This made supporting `null` in the FFI impossible.
+
+However, the new WebAssembly Garbage Collection (wasm-gc) feature adds new
+instructions to create and check for special externrefs that are not `null`. We
+switched to using this whenever it is supported. If wasm-gc is not supported, we
+fall back to using a JavaScript function to test for an error and put up with
+the 2-3% performance hit. [Since December 2024, all major JavaScript runtimes
+support wasm-gc](https://webassembly.org/features/).
+
+## Community spotlight: Pyodide in the wild
+
+### SPy: statically typed Python
+
+- New collaborative work led to breakthroughs in early-stage in-browser,
+statically-typed Python for graphics or computationally heavy use cases, as part
+of an [SPy](https://github.com/spylang/spy) demo using Pyodide.
+
+[Read blog post by Łukasz Langa 🔗](https://lukasz.langa.pl/f37aa97a-9ea3-4aeb-b6a0-9daeea5a7505/)
+
+### Interactive documentation via CZI grant to Scientific Python
+
+Work wrapped up on Pyodide interoperability and in-browser interactive documentation
+via JupyterLite for Scientific Python libraries via the [2022 grant "Scientific Python Community
+& Communications Infrastructure"](https://blog.scientific-python.org/scientific-python/2022-czi-grant/) awarded
+by the Chan Zuckerberg Institute to [Scientific Python](https://scientific-python.org/).
+
+## Acknowledgements
+
+We appreciate the continued support from the Emscripten team, particularly from
+Sam Clegg; from the CPython team, particularly from Russell Keith-Magee and
+Łukasz Langa; and from the cibuildwheel team, particularly from Henry Schreiner
+and Joe Rickerby.
+
+Special thanks to Ian Thomas for his work on the new Matplotlib backend.
+
+Thanks to all the contributors who made this release possible:
+
+Agriya Khetarpal, airen1986, aiudirog, Alexey Ignatiev, Andrea Giammarchi, arctus-io, Artem Samokhin, Arturo Amor, Bart Broere, Christian Clauss, David, Dmitry Dygalo, Francesc Alted, Giuseppe Capasso, Greg Wilson, Gyeongjae Choi, Hood Chatham, Ian Thomas, Ikko Eltociear Ashimine, Isaac Brodsky, JHM Darbyshire, Joe Marshall, joellindegger, John Wason, Juniper Tyree, Kai Mühlbauer, Kaspar Emanuel, Loïc Estève, Lukas, Łukasz Langa, Marco Edward Gorelli, Michael Droettboom, mstrahov, Nicholas Bollweg, Oscar Gustafsson, Pascal Thomet, Pepijn de Vos, Samuel Colvin, Shaurya Bisht, Szabolcs Dombi, Teon L Brooks, Tom Dudley, Vineet Bansal