Use IR in the optimizer #1701
Assignees
Labels
Comments
19 tasks
|
The ExternalTensor support in the IR is working only when code is launched at the same directory as the model file. Aka. we read the external data file using relative paths. This is because when a model is loaded as a proto, its path information is lost to the IR. We need to design a way (e.g. pass in base path when deserializing) to handle when the model file is somewhere else. This is relevant because I assume the optimizer will sometimes fold external tensors. Some examples may include |
|
Related: #1436 |
justinchuby
added a commit
that referenced
this issue
Aug 13, 2024
…perly (#1801) Implement efficient save/load and handle loading external data properly in the IR. Before this change, when a ModelProto containing external data is converted to IR, the external tensor objects will load the data from a path relative to the working directory, not the ONNX file. This is because we do not store the onnx file path and thus have no way to look for the external data file. With the change, a `base_dir` property is added to ExternalTensor that we can set, in a separate pass when the directory is available, so the object has full information to find the data file on disk. The base_dir is not serialized to the proto to maintain a relative path in the "location" field in TensorProto. #1701, #1792 Example: ``` >>> m.graph.initializers["model.model.decoder.layers.2.encoder_attn.v_proj.weight"].const_value.display() ExternalTensor<FLOAT,[512,512]>(path='model.onnx.data', name='model.model.decoder.layers.2.encoder_attn.v_proj.weight', offset=245864448, length=1048576, base_dir='/home/justinchu/dev/ONNXConverter/docker/dump_bash_bench/BlenderbotSmallForConditionalGeneration-torch -onnx-detailed-cpu-') Min: -0.08586505800485611, Max: 0.09103105217218399, NaN count: 0, Inf count: 0 Sparsity (abs<1e-06): 0.00 Histogram: 11504 ┼ 10226 ┤ ╭───────╮ 8948 ┤ ╭─╯ ╰─╮ 7670 ┤ ╭─╯ ╰─╮ 6392 ┤ ╭─╯ ╰─╮ 5113 ┤ ╭─╯ ╰─╮ 3835 ┤ ╭─╯ ╰─╮ 2557 ┤ ╭──╯ ╰─╮ 1279 ┤ ╭────╯ ╰────╮ 1 ┼────────────────╯ ╰─────────────────── -0.0859 -0.0682 -0.0505 -0.0306 -0.0129 0.0070 0.0225 0.0402 0.0557 0.0733 0.0910 ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
And improve robustness so it is reliable enough to be turned on by default.
The text was updated successfully, but these errors were encountered: