Sentence Embedding with SentenceTransformer and ONNX in C

This project demonstrates how to convert a SentenceTransformer model to ONNX format and use it to generate sentence embeddings in a C program. The goal is to compare the performance of generating embeddings in C versus Python.

Project Structure

• cprog: Contains the C program and the compiled executable.
  • main.c: The C program that loads the ONNX model, tokenizes input text, and generates embeddings.
  • onnx_inference: The compiled C program executable.
• python: Contains the Python script for converting the model to ONNX format.
  • convert_to_onnx.py: The script to convert the SentenceTransformer model to ONNX format.
  • sentence_transformer.onnx: The ONNX model file generated by the script.
• tests: Contains test scripts (currently empty).
• README.md: This readme file.
• pyproject.toml and poetry.lock: Poetry configuration files for managing Python dependencies.

Requirements

• Python 3.12
• Poetry
• ONNX Runtime
• GCC

Setup

Python Environment

1. Install Poetry if you haven't already:

   curl -sSL https://install.python-poetry.org | python3 -

2. Install the required Python dependencies:

   poetry install

3. Convert the SentenceTransformer model to ONNX format:

   poetry run python convert_to_onnx.py

C Environment

1. Ensure you have ONNX Runtime installed via Homebrew:

   brew install onnxruntime

2. Compile the C program:

   gcc -o cprog/onnx_inference cprog/main.c -I/opt/homebrew/Cellar/onnxruntime/1.17.1/include/onnxruntime -L/opt/homebrew/Cellar/onnxruntime/1.17.1/lib -lonnxruntime

Test

2. Run the C program to generate embeddings:

   ./onnx_inference "fox and dog"

Performance Comparison

(Add your performance comparison results here once you have them)

Acknowledgments

• SentenceTransformers library
• ONNX Runtime
• (Add any other libraries or resources you've used)