Skip to content

Modify the handing of Frameworks on Apple platforms #5606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
freakboy3742 wants to merge 11 commits into from
Closed

Modify the handing of Frameworks on Apple platforms #5606

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
cf315b3
Modify handling of Python Framework linking.
freakboy3742 Nov 6, 2025
9f0aac5
Add news fragment.
freakboy3742 Nov 6, 2025
5fc325b
Correct rustfmt issues.
freakboy3742 Nov 6, 2025
3820fa3
Correct some problems identified by clippy.
freakboy3742 Nov 6, 2025
ca120b7
Interpret an empty PYTHONFRAMEWORK as None not an empty string.
freakboy3742 Nov 6, 2025
55a1d95
Corrected handling of empty PYTHONFRAMEWORK definitions.
freakboy3742 Nov 6, 2025
4ed808d
Only add a framework linking path if there's a framework.
freakboy3742 Nov 6, 2025
4a77d15
Cleanup rustfmt problems.
freakboy3742 Nov 6, 2025
4bc523d
Merge branch 'main' into framework-linking
freakboy3742 Nov 12, 2025
20d4cb3
Document framework linking behavior.
freakboy3742 Nov 12, 2025
f2260c0
Convert empty framework prefixes to None.
freakboy3742 Nov 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions newsfragments/5606.changed.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
If the `sysconfigdata` defines `PYTHONFRAMEWORK`, and the build is configured to link the against Python library, the link will be performed against that framework, using `PYTHONFRAMEWORKPREFIX` as a framework search path.
52 changes: 47 additions & 5 deletions pyo3-build-config/src/impl_.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,11 @@ pub struct InterpreterConfig {
/// Serialized to `version`.
pub version: PythonVersion,

/// The name of the Python framework (if available)
///
/// Serialized to `framework`.
Comment on lines +101 to +107

@davidhewitt davidhewitt Nov 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should note that if this is set, this will be used for linking instead of shared and lib_name.

pub framework: Option<String>,

/// Whether link library is shared.
///
/// Serialized to `shared`.
Expand Down Expand Up @@ -242,6 +247,10 @@ def print_if_set(varname, value):
if value is not None:
print(varname, value)

def print_if_not_empty(varname, value):
if value:
print(varname, value)

# Windows always uses shared linking
WINDOWS = platform.system() == "Windows"

Expand All @@ -255,6 +264,7 @@ SHARED = bool(get_config_var("Py_ENABLE_SHARED"))
print("implementation", platform.python_implementation())
print("version_major", sys.version_info[0])
print("version_minor", sys.version_info[1])
print_if_not_empty("framework", get_config_var("PYTHONFRAMEWORK"))
print("shared", PYPY or GRAALPY or ANACONDA or WINDOWS or FRAMEWORK or SHARED)
print("python_framework_prefix", FRAMEWORK_PREFIX)
print_if_set("ld_version", get_config_var("LDVERSION"))
Expand Down Expand Up @@ -293,6 +303,7 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))
);
};

let framework = map.get("framework").cloned();
let shared = map["shared"].as_str() == "True";
let python_framework_prefix = map.get("python_framework_prefix").cloned();

Expand Down Expand Up @@ -360,6 +371,7 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))
Ok(InterpreterConfig {
version,
implementation,
framework,
shared,
abi3,
lib_name: Some(lib_name),
Expand Down Expand Up @@ -403,10 +415,10 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))
_ => bail!("expected a bool (1/true/True or 0/false/False) for Py_ENABLE_SHARED"),
};
// macOS framework packages use shared linking (PYTHONFRAMEWORK is the framework name, hence the empty check)
let framework = match sysconfigdata.get_value("PYTHONFRAMEWORK") {
Some(s) => !s.is_empty(),
_ => false,
};
let framework = get_key!(sysconfigdata, "PYTHONFRAMEWORK")
.ok()
.filter(|s| !s.is_empty())
.map(str::to_string);
let python_framework_prefix = sysconfigdata
.get_value("PYTHONFRAMEWORKPREFIX")
.map(str::to_string);

@davidhewitt davidhewitt Nov 11, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should do something similar to filter out cases where the prefix is an empty string? Maybe if that is semantically no different from None, we should consider removing the Option outer layer from these variables and just treat the empty string as unset?

@freakboy3742 freakboy3742 Nov 12, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd lean to filtering out empty PYTHONFRAMEWORKPREFIX examples; I can't speak with much Rust experience, but as a Python user, an explicit "None" makes more sense to me than ignoring an empty string. I'll push an update to that effect.

Expand All @@ -429,11 +441,13 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))
.map(|bytes_width: u32| bytes_width * 8)
.ok();
let build_flags = BuildFlags::from_sysconfigdata(sysconfigdata);
let shared = shared || framework.is_some();

Ok(InterpreterConfig {
implementation,
version,
shared: shared || framework,
framework,
shared,
abi3,
lib_dir,
lib_name,
Expand Down Expand Up @@ -510,6 +524,7 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))

let mut implementation = None;
let mut version = None;
let mut framework = None;
let mut shared = None;
let mut abi3 = None;
let mut lib_name = None;
Expand All @@ -535,6 +550,7 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))
match key {
"implementation" => parse_value!(implementation, value),
"version" => parse_value!(version, value),
"framework" => parse_value!(framework, value),
"shared" => parse_value!(shared, value),
"abi3" => parse_value!(abi3, value),
"lib_name" => parse_value!(lib_name, value),
Expand All @@ -561,6 +577,7 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))
Ok(InterpreterConfig {
implementation,
version,
framework,
shared: shared.unwrap_or(true),
abi3,
lib_name,
Expand Down Expand Up @@ -674,6 +691,7 @@ print("gil_disabled", get_config_var("Py_GIL_DISABLED"))

write_line!(implementation)?;
write_line!(version)?;
write_option_line!(framework)?;
write_line!(shared)?;
write_line!(abi3)?;
write_option_line!(lib_name)?;
Expand Down Expand Up @@ -1622,6 +1640,7 @@ fn default_cross_compile(cross_compile_config: &CrossCompileConfig) -> Result<In
Ok(InterpreterConfig {
implementation,
version,
framework: None,
shared: true,
abi3,
lib_name: Some(lib_name),
Expand Down Expand Up @@ -1665,6 +1684,7 @@ fn default_abi3_config(host: &Triple, version: PythonVersion) -> Result<Interpre
Ok(InterpreterConfig {
implementation,
version,
framework: None,
shared: true,
abi3,
lib_name,
Expand Down Expand Up @@ -2055,6 +2075,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_name: Some("lib_name".into()),
lib_dir: Some("lib_dir".into()),
framework: Some("Python".into()),
shared: true,
version: MINIMUM_SUPPORTED_VERSION,
suppress_build_script_link_lines: true,
Expand All @@ -2081,6 +2102,7 @@ mod tests {
implementation: PythonImplementation::PyPy,
lib_dir: None,
lib_name: None,
framework: None,
shared: true,
version: PythonVersion {
major: 3,
Expand All @@ -2106,6 +2128,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_name: Some("lib_name".into()),
lib_dir: Some("lib_dir\\n".into()),
framework: None,
shared: true,
version: MINIMUM_SUPPORTED_VERSION,
suppress_build_script_link_lines: true,
Expand All @@ -2128,6 +2151,7 @@ mod tests {
InterpreterConfig {
version: PythonVersion { major: 3, minor: 7 },
implementation: PythonImplementation::CPython,
framework: None,
shared: true,
abi3: false,
lib_name: None,
Expand All @@ -2151,6 +2175,7 @@ mod tests {
InterpreterConfig {
version: PythonVersion { major: 3, minor: 7 },
implementation: PythonImplementation::CPython,
framework: None,
shared: true,
abi3: false,
lib_name: None,
Expand Down Expand Up @@ -2259,6 +2284,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_dir: Some("/usr/lib".into()),
lib_name: Some("python3.7m".into()),
framework: None,
shared: true,
version: PythonVersion::PY37,
suppress_build_script_link_lines: false,
Expand Down Expand Up @@ -2289,6 +2315,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_dir: Some("/usr/lib".into()),
lib_name: Some("python3.7m".into()),
framework: Some("Python".into()),
shared: true,
version: PythonVersion::PY37,
suppress_build_script_link_lines: false,
Expand Down Expand Up @@ -2316,6 +2343,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_dir: Some("/usr/lib".into()),
lib_name: Some("python3.7m".into()),
framework: None,
shared: false,
version: PythonVersion::PY37,
suppress_build_script_link_lines: false,
Expand All @@ -2335,6 +2363,7 @@ mod tests {
InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 7 },
framework: None,
shared: true,
abi3: true,
lib_name: Some("python3".into()),
Expand All @@ -2359,6 +2388,7 @@ mod tests {
InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 9 },
framework: None,
shared: true,
abi3: true,
lib_name: None,
Expand Down Expand Up @@ -2394,6 +2424,7 @@ mod tests {
InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 7 },
framework: None,
shared: true,
abi3: false,
lib_name: Some("python37".into()),
Expand Down Expand Up @@ -2429,6 +2460,7 @@ mod tests {
InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 8 },
framework: None,
shared: true,
abi3: false,
lib_name: Some("python38".into()),
Expand Down Expand Up @@ -2464,6 +2496,7 @@ mod tests {
InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 9 },
framework: None,
shared: true,
abi3: false,
lib_name: Some("python3.9".into()),
Expand Down Expand Up @@ -2501,6 +2534,7 @@ mod tests {
major: 3,
minor: 11
},
framework: None,
shared: true,
abi3: false,
lib_name: Some("pypy3.11-c".into()),
Expand Down Expand Up @@ -2894,6 +2928,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_dir: None,
lib_name: None,
framework: None,
shared: true,
version: PythonVersion { major: 3, minor: 7 },
suppress_build_script_link_lines: false,
Expand All @@ -2917,6 +2952,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_dir: None,
lib_name: None,
framework: None,
shared: true,
version: PythonVersion { major: 3, minor: 7 },
suppress_build_script_link_lines: false,
Expand Down Expand Up @@ -2982,6 +3018,7 @@ mod tests {
implementation: PythonImplementation::CPython,
lib_dir: interpreter_config.lib_dir.to_owned(),
lib_name: interpreter_config.lib_name.to_owned(),
framework: None,
shared: true,
version: interpreter_config.version,
suppress_build_script_link_lines: false,
Expand Down Expand Up @@ -3112,6 +3149,7 @@ mod tests {
major: 3,
minor: 11,
},
framework: None,
shared: true,
abi3: false,
lib_name: Some("python3".into()),
Expand Down Expand Up @@ -3156,6 +3194,7 @@ mod tests {
let interpreter_config = InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 9 },
framework: None,
shared: true,
abi3: true,
lib_name: Some("python3".into()),
Expand Down Expand Up @@ -3204,6 +3243,7 @@ mod tests {
major: 3,
minor: 13,
},
framework: None,
shared: true,
abi3: false,
lib_name: Some("python3".into()),
Expand Down Expand Up @@ -3238,6 +3278,7 @@ mod tests {
let interpreter_config = InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 7 },
framework: None,
shared: true,
abi3: false,
lib_name: Some("python3".into()),
Expand Down Expand Up @@ -3293,6 +3334,7 @@ mod tests {
let mut config = InterpreterConfig {
implementation: PythonImplementation::CPython,
version: PythonVersion { major: 3, minor: 9 },
framework: None,
shared: true,
abi3: false,
lib_name: None,
Expand Down
2 changes: 2 additions & 0 deletions pyo3-build-config/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,7 @@ mod tests {
major: 3,
minor: 13,
},
framework: None,
shared: true,
abi3: false,
lib_name: None,
Expand Down Expand Up @@ -455,6 +456,7 @@ mod tests {
major: 3,
minor: 13,
},
framework: None,
shared: true,
abi3: false,
lib_name: None,
Expand Down
19 changes: 15 additions & 4 deletions pyo3-ffi/build.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,9 @@ fn emit_link_config(build_config: &BuildConfig) -> Result<()> {

println!(
"cargo:rustc-link-lib={link_model}{alias}{lib_name}",
link_model = if interpreter_config.shared {
link_model = if interpreter_config.framework.is_some() {
"framework="

@davidhewitt davidhewitt Nov 12, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the 3.11 CI failure has found a problem with this approach.

Seems like framework=Python does not specify the version, and nor does the search prefix which is just /Library/Frameworks (for the official distributions).

$ /Library/Frameworks/Python.framework/Versions/3.13/bin/python3 -m sysconfig | grep -i PYTHONFRAMEWORK
	PYTHONFRAMEWORK = "Python"
	PYTHONFRAMEWORKDIR = "Python.framework"
	PYTHONFRAMEWORKINSTALLDIR = "/Library/Frameworks/Python.framework"
	PYTHONFRAMEWORKINSTALLNAMEPREFIX = "/Library/Frameworks/Python.framework/Versions/3.13"
	PYTHONFRAMEWORKPREFIX = "/Library/Frameworks"

Not sure what implications this has for the iOS build either? Seems like it's fundamentally impossible to use frameworks and link the right version.

I wonder, maybe we don't need this PR? If cross-compile env has the correct lib_dir, potentially PyO3 main is now already doing the correct thing to link against iOS via the versioned .dylib directly with the other patches placed into PyO3 and maturin?

@freakboy3742 freakboy3742 Nov 13, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the 3.11 CI failure has found a problem with this approach.

Which failure is this? I'm seeing 11 skips, 32 successful tests in the most recent CI pass.

Seems like framework=Python does not specify the version, and nor does the search prefix which is just /Library/Frameworks (for the official distributions).

$ /Library/Frameworks/Python.framework/Versions/3.13/bin/python3 -m sysconfig | grep -i PYTHONFRAMEWORK
	PYTHONFRAMEWORK = "Python"
	PYTHONFRAMEWORKDIR = "Python.framework"
	PYTHONFRAMEWORKINSTALLDIR = "/Library/Frameworks/Python.framework"
	PYTHONFRAMEWORKINSTALLNAMEPREFIX = "/Library/Frameworks/Python.framework/Versions/3.13"
	PYTHONFRAMEWORKPREFIX = "/Library/Frameworks"

Not sure what implications this has for the iOS build either? Seems like it's fundamentally impossible to use frameworks and link the right version.

So - macOS frameworks and iOS frameworks (and every other non-macOS framework) are structured slightly differently.

macOS frameworks are versioned - there's a single top-level framework, with a "Versions" subfolder; each version of the framework is then contained in the top level framework. A Current symlink exists to point at one specific version, and the top level framework links the content of the current version so that /Library/Frameworks/Python.framework can be used for linking. However, that doesn't give you control over the specific version. If you want a specific version, you need to use -F /Library/Frameworks/Python.framework/Versions/3.13 as your framework search path.

I wonder, maybe we don't need this PR? If cross-compile env has the correct lib_dir, potentially PyO3 main is now already doing the correct thing to link against iOS with the other patches placed into PyO3 and maturin?

So... that doesn't currently work, because iOS doesn't ship a copy of libPython*.dylib in the lib folder. There's a range of reasons why that is the case... but with some other accomodations, that might indeed be an easier way to solve the problem. I'll experiment with this and report back.

@davidhewitt davidhewitt Nov 13, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the merge queue https://github.com/PyO3/pyo3/actions/runs/19310395011/job/55228919449

@davidhewitt davidhewitt Nov 13, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need to use -F /Library/Frameworks/Python.framework/Versions/3.13 as your framework search path.

I think I attempted this yesterday and had no luck. Possibly because Python.Framework only exists in /Library/Frameworks and not in any of the versioned subdirectories?

@davidhewitt davidhewitt Nov 13, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added CI-build-full label to this PR so any further pushes will run the full suite of versions we run in the merge queue.

} else if interpreter_config.shared {
""
} else {
"static="
Expand All @@ -160,11 +162,20 @@ fn emit_link_config(build_config: &BuildConfig) -> Result<()> {
} else {
""
},
lib_name = interpreter_config.lib_name.as_ref().ok_or(
"attempted to link to Python shared library but config does not contain lib_name"
)?,
lib_name = if let Some(framework) = &interpreter_config.framework {
framework
} else {
interpreter_config.lib_name.as_ref().ok_or(
"attempted to link to Python shared library but config does not contain lib_name",
)?
},
);

if interpreter_config.framework.is_some() {
if let Some(framework_prefix) = &interpreter_config.python_framework_prefix {
println!("cargo:rustc-link-search=framework={framework_prefix}");
}
}
if let Some(lib_dir) = &interpreter_config.lib_dir {
println!("cargo:rustc-link-search=native={lib_dir}");
} else if matches!(build_config.source, BuildConfigSource::CrossCompile) {
Expand Down
Loading