Skip to content

Commit 50bffbb

Browse files
Andrew Pullinfacebook-github-bot
authored andcommitted
Fix Neutron backend API compatibility and multiprocessing fallback
Summary: This fixes two issues preventing the Neutron backend from building in sandcastle/build environments: 1. **NeutronAtenPassManager API compatibility**: The `NeutronAtenPassManager` class was updated to require a `neutron_target_spec` parameter, but the call site wasn't updated. This caused a `TypeError` at runtime. Fixed by passing the already-available `neutron_target_spec` to the pass manager constructor. 2. **Multiprocessing fallback**: The Neutron converter uses multiprocessing for isolation when converting models, but this fails in restricted build environments (sandcastle) with an `EOFError` when trying to create a multiprocessing Manager. Added a try/except block to catch `EOFError` and `OSError` and fall back to direct execution of the converter when multiprocessing is unavailable. This maintains isolation in normal environments while allowing builds to succeed in restricted ones. Both issues were pre-existing on trunk and not caused by recent changes. Differential Revision: D87252895
1 parent f66b79f commit 50bffbb

File tree

1 file changed

+27
-15
lines changed

1 file changed

+27
-15
lines changed

backends/nxp/backend/neutron_converter_manager.py

Lines changed: 27 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -78,23 +78,35 @@ def convert(self, tflite_model: bytes, target: str) -> bytes:
7878
cctx.compilationOpts.minNumOpsPerGraph = 1
7979
cctx.compilationOpts.excludeGraphPasses = "MergeTranspose"
8080

81-
logger = multiprocessing.log_to_stderr()
82-
logger.setLevel(logging.WARNING)
83-
queue = multiprocessing.Manager().Queue()
81+
# Try to use multiprocessing for isolation, but fall back to direct execution
82+
# if the environment doesn't support it (e.g., in sandcastle/build environments)
83+
try:
84+
logger = multiprocessing.log_to_stderr()
85+
logger.setLevel(logging.WARNING)
86+
queue = multiprocessing.Manager().Queue()
87+
88+
process = multiprocessing.Process(
89+
target=convert_unsafe,
90+
args=(self.neutron_converter, tflite_model, cctx, queue),
91+
)
92+
process.start()
93+
process.join() # waits until the subprocess is complete
8494

85-
process = multiprocessing.Process(
86-
target=convert_unsafe,
87-
args=(self.neutron_converter, tflite_model, cctx, queue),
88-
)
89-
process.start()
90-
process.join() # waits until the subprocess is complete
95+
if queue.empty(): # signals the unsafe task did not run till the end
96+
raise RuntimeError(
97+
f"Neutron converter module terminated unexpectedly with exit code {process.exitcode}"
98+
)
9199

92-
if queue.empty(): # signals the unsafe task did not run till the end
93-
raise RuntimeError(
94-
f"Neutron converter module terminated unexpectedly with exit code {process.exitcode}"
100+
model_converted = queue.get()
101+
process.close()
102+
except (EOFError, OSError) as e:
103+
# Multiprocessing failed (likely due to environment restrictions)
104+
# Fall back to direct execution
105+
logging.warning(
106+
f"Multiprocessing not available ({e}), running neutron converter directly"
107+
)
108+
model_converted = self.neutron_converter.convertModel(
109+
list(tflite_model), cctx
95110
)
96111

97-
model_converted = queue.get()
98-
99-
process.close()
100112
return bytes(model_converted)

0 commit comments

Comments
 (0)