Consider migrating to Rust based Python package?

[haoxins]

General Question

More and more Python packages are built from the Rust code base. Such as:

1. https://github.com/pola-rs/polars
2. https://github.com/Qiskit/rustworkx
3. https://github.com/pydantic/pydantic-core

The associated tools are:

https://github.com/PyO3

The benefit is that we can build the logic from Rust once and publish it as Rust, Python, and even JavaScript (Node.js) packages.
This is especially the trending solution for the data/ML ecosystem.

When I start to build the services and SDKs based on the Nebula.
The first candidate language is Rust so that I can develop and deploy the services as Rust code, and build the Python SDKs (for data scientists) based on the Rust codebase.
This is the ideal solution for writing code only once.

But the Rust client for Nebula is not ready at last year.

Since we already have the first draft version of the Nebula Rust client, I think this is a good time to share this idea with you guys.

The other tools for more languages that I'm not used to are:
https://github.com/dtolnay/cxx for c++

cc @Nicole00 @wey-gu

[haoxins]

Another example from Temporal SDK core

[QingZ11]

It's really a great idea, thank you Haoxin.

[wey-gu]

General Question

More and more Python packages are built from the Rust code base. Such as:

1. https://github.com/pola-rs/polars

2. https://github.com/Qiskit/rustworkx

3. https://github.com/pydantic/pydantic-core

The associated tools are:

https://github.com/PyO3

The benefit is that we can build the logic from Rust once and publish it as Rust, Python, and even JavaScript (Node.js) packages.

This is especially the trending solution for the data/ML ecosystem.

When I start to build the services and SDKs based on the Nebula.

The first candidate language is Rust so that I can develop and deploy the services as Rust code, and build the Python SDKs (for data scientists) based on the Rust codebase.

This is the ideal solution for writing code only once.

But the Rust client for Nebula is not ready at last year.

Since we already have the first draft version of the Nebula Rust client, I think this is a good time to share this idea with you guys.

The other tools for more languages that I'm not used to are:

https://github.com/dtolnay/cxx for c++

cc @Nicole00 @wey-gu

Agreed, was considering pyo3 when facing perf issues in wey-gu/nebula-dgl#10

We could do so in free time!

[haoxins]

Also find this https://github.com/jni-rs/jni-rs for Java :)

[wey-gu]

Also, recently I had some chance to implement/build something with cython, later I could prepare a PR to add cython build of nebula-python and come with a benchmark.

[Abeautifulsnow]

👍Both Rust and Cython are excellent choices, but Rust (using PyO3) may be a more popular and convenient option when integrating with both Python and Node, especially if there already has a mature Rust-based repository here. Hope to see it soon.

[wey-gu]

https://github.com/nebula-contrib/rust-nebula

cc @BeautyyuYanli @PsiACE

[PsiACE]

https://github.com/nebula-contrib/rust-nebula

cc @BeautyyuYanli @PsiACE

Cool idea. We can start by simply checking that the core functionality is aligned.