Skip to content

fix: support Float16/BFloat16/Float8 in TensorArray custom op #28335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tianleiwu merged 1 commit into from
Jun 4, 2026
Merged

fix: support Float16/BFloat16/Float8 in TensorArray custom op #28335

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions include/onnxruntime/core/session/onnxruntime_lite_custom_op.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,12 @@ struct TensorArray : public ArgBase {
case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT:
tensor = std::make_unique<Custom::Tensor<float>>(ctx, ith_input, true);
break;
case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16:
tensor = std::make_unique<Custom::Tensor<Ort::Float16_t>>(ctx, ith_input, true);
break;
case ONNX_TENSOR_ELEMENT_DATA_TYPE_BFLOAT16:
tensor = std::make_unique<Custom::Tensor<Ort::BFloat16_t>>(ctx, ith_input, true);
break;
case ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE:
tensor = std::make_unique<Custom::Tensor<double>>(ctx, ith_input, true);
break;
Expand Down Expand Up @@ -360,6 +366,18 @@ struct TensorArray : public ArgBase {
case ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING:
tensor = std::make_unique<Custom::Tensor<std::string>>(ctx, ith_input, true);
break;
case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FN:
tensor = std::make_unique<Custom::Tensor<Ort::Float8E4M3FN_t>>(ctx, ith_input, true);
break;
case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FNUZ:
tensor = std::make_unique<Custom::Tensor<Ort::Float8E4M3FNUZ_t>>(ctx, ith_input, true);
break;
case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2:
tensor = std::make_unique<Custom::Tensor<Ort::Float8E5M2_t>>(ctx, ith_input, true);
break;
case ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2FNUZ:
tensor = std::make_unique<Custom::Tensor<Ort::Float8E5M2FNUZ_t>>(ctx, ith_input, true);
break;
default:
ORT_CXX_API_THROW("unknown input type", ORT_RUNTIME_EXCEPTION);
break;
Expand Down
Loading