AMMO toolkit with _GLIBCXX_USE_CXX11_ABI=0

[chenxu2048]

Hello everyone,

Our LLM service, built on TensorRT-LLM and several libraries, now requires integration with the AMMO toolkit for model quantization. However, we've run into a challenge with the toolkit being released with _GLIBCXX_USE_CXX11_ABI=1. Consequently, we need to rebuild all our used libraries with the same setting.

We're wondering if there's a version of the AMMO library compatible with prebuilt PyTorch, thus eliminating the dependency on _GLIBCXX_USE_CXX11_ABI.

Any insights or guidance on this matter would be greatly appreciated.

Thanks

[juney-nvidia]

@chenxu2048
Can you share the concrete context/steps of reproducing your issue?
We have prepared PIP wheel package(wo CXX11 ABI) which depends on the installation of AMMO package and it works. I want to learn more of your concrete issues here.

Thanks
June

[chenxu2048]

@juney-nvidia

Thanks for your answers! It was fixed now. We checked our CI and found that the ammo package was "polluted" by prebuilt torch with cxx11abi