Adds the ability to build static archives in addition to shared objects #2179
Adds the ability to build static archives in addition to shared objects #2179
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an opt-in build path to produce static archives for the MHA forward/backward test libraries to avoid runtime symbol conflicts when multiple AITER versions are present.
Changes:
- Adds an
artool variable to the generated Ninja config (overridable viaAITER_AR_BIN). - When
AITER_BUILD_STATIC=1and the output islibmha_fwd.so/libmha_bwd.so, emits a correspondinglibmha_*wd.aarchive build rule and includes it in Ninja defaults.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| archive_rule = ["rule archive"] | ||
| archive_rule.append( | ||
| " command = rm -f $out && $ar rcs $out @$out.rsp\n rspfile = $out.rsp\n rspfile_content = $in" | ||
| ) | ||
|
|
There was a problem hiding this comment.
archive_rule is defined unconditionally here but then always either overwritten with the same contents in the emit_static_archive branch or replaced with [] in the else branch, so this initial definition is dead code. Consider defining archive_rule only inside the emit_static_archive branch (and otherwise leaving it empty) to avoid duplication and reduce confusion.
| archive_rule = ["rule archive"] | |
| archive_rule.append( | |
| " command = rm -f $out && $ar rcs $out @$out.rsp\n rspfile = $out.rsp\n rspfile_content = $in" | |
| ) |
| ldflags = ([] if is_standalone else [SHARED_FLAG]) + extra_ldflags | ||
|
|
||
| ext = EXEC_EXT if is_standalone else LIB_EXT | ||
| library_target = f"{name}{ext}" |
There was a problem hiding this comment.
Can we detect whether it's .a or .so based on the library target ext?
| emit_static_archive = ( | ||
| library_target is not None | ||
| and os.environ.get("AITER_BUILD_STATIC", "0") == "1" | ||
| and os.path.basename(library_target) in {"libmha_fwd.so", "libmha_bwd.so"} |
There was a problem hiding this comment.
Probably we can add some argument/option and pass the library_target down from compile.py in op_tests/cpp/mha as libmha_fwd/bwd.a instead of libmha_fwd/bwd.so?
Motivation
TE currently relies on
libmha_{b, f}wd.soto provide AITER symbols at runtime, however if other projects (e.g. pytorch) also build the AITER library, but of a different version, then loading the same symbols can create conflict. To avoid this, we at TE would like to be able to generate static archives as part of the build triggered throughaiter/op_tests/cpp/mha/compile.py.Technical Details
Gated by a new environment variable
AITER_BUILD_STATIC, there is now a new rule inserted which will buildlibmha_*wd.ausingAITER_AR_BIN(set toarby default). Backwards compatibility is ensured byAITER_BUILD_STATICbeing defaulted to0.Test Plan
Run
aiter/op_tests/cpp/mha/compile.pyand validate that the expected artifacts are produced.Test Result
Submission Checklist