#6413·pyodide

loadDynlib() always uses RTLD_LOCAL, breaking dlopen() of extensions that need a package's own exported symbols (e.g. DuckDB)

Author: maxdemarziCreated Aug 4, 2026Updated Aug 5, 2026

Summary

src/js/dynload.ts's loadDynlib() -- used by both pyodide.loadPackage() and micropip.install() -- always calls Emscripten's loadDynamicLibrary() with local-only visibility (flags = 2, i.e. RTLD_NOW | RTLD_LOCAL). This means a loaded module's own exported symbols never reach Emscripten's shared/global symbol table, so a separate, later dlopen() call made from C/C++ code running inside that module (or any other already-loaded module) cannot resolve them -- even when the target module's WASM export section genuinely contains the symbol.

This is exactly the failure behind duckdb/duckdb-pyodide#2 ("extensions are not supported in the duckdb version of pyodide"), and I'd guess it affects any Pyodide package whose own compiled extension is designed to be a dlopen()-target for further, independently-loaded native code (plugin-style architectures), not just DuckDB.

Repro

  1. Install a real DuckDB Pyodide wheel (duckdb.__version__ confirmed working, import duckdb succeeds).
  2. dlopen() (via DuckDB's own internal extension-loading machinery, triggered by LOAD <extension>;) a separately-built, ABI/version-matched .duckdb_extension file that needs symbols from _duckdb's own compiled module (e.g. duckdb::ExtensionLoader::GetDatabaseInstance()).
  3. Fails: Dynamic linking error: cannot resolve symbol _ZN6duckdb15ExtensionLoader19GetDatabaseInstanceEv.

Confirmed directly, not guessed:

  • The symbol genuinely exists in _duckdb's own compiled .so, checked two ways: llvm-nm shows it as a defined (T) symbol, and WebAssembly.Module.exports() on the raw bytes lists it as a real, exported function.
  • pyodide._module.LDSO.loadedLibsByName["/lib/python.../site-packages/_duckdb..."].global is false after a normal import duckdb (whether via pyodide.loadPackage(["duckdb"]) or micropip.install(<duckdb wheel url>)).

Root cause (in Emscripten, surfaced via Pyodide's own loader)

emscripten-core/emscripten's src/lib/libdylink.js, loadDynamicLibrary(): whether a load is global is carried per-DSO as dso.global = flags.global, and only global-flagged exports get merged into the shared symbol table (mergeLibSymbols) -- otherwise they land in a call-scoped localScope, invisible to any other, later dlopen() call. Pyodide's own dynload.ts hardcodes flags = 2 (i.e. global: false) for every package it loads via loadDynlib(), with no way for a package (or its installer) to opt into global visibility.

Workaround found, no Pyodide/Emscripten rebuild needed

loadDynamicLibrary() already has an "already loaded, promote to global" branch:

javascript
var dso = LDSO.loadedLibsByName[libName];
if (dso) {
  if (!flags.global) { ... }
  else if (!dso.global) {
    dso.global = true;
    mergeLibSymbols(dso.exports, libName)   // promotes retroactively, no re-instantiation
  }
  ...
}

Calling pyodide._module.loadDynamicLibrary(key, {global: true, nodelete: true}) again, after the normal import duckdb, with the exact canonical key already present in LDSO.loadedLibsByName (to guarantee this branch is hit rather than a fresh, separate load), flips dso.global and merges the symbols in -- confirmed this resolves the dlopen() failure above completely, with a real end-to-end query then working.

This is reachable from pure Python too, not just JS glue on the host page -- pyodide_js (the Python-side FFI mirror of the JS pyodide controller object) exposes the same _module.loadDynamicLibrary/_module.LDSO, so a package's own Python code can self-heal this for its own dependencies without requiring anything from the embedding page.

Ask

Either of these would close the gap for anyone hitting this (not just DuckDB):

  1. A documented, supported way to request RTLD_GLOBAL-equivalent loading for a specific package via pyodide.loadPackage()/micropip.install() (an option, rather than the hardcoded flags = 2).
  2. At minimum, documentation of the workaround above (the pyodide_js._module.loadDynamicLibrary(key, {global: true}) re-promotion pattern) somewhere discoverable, since as far as I can tell this isn't written down anywhere today and duckdb-pyodide#2's own conclusion (linked above) reads as "not supported" rather than "here's how."

Happy to open a PR for either the doc or a global passthrough option if that's welcome -- wanted to raise it here first since I don't know this codebase's conventions for exposing new loader flags.