Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Logging SDK speed regression #4093

Closed
emilk opened this issue Oct 30, 2023 · 1 comment · Fixed by #4094
Closed

Logging SDK speed regression #4093

emilk opened this issue Oct 30, 2023 · 1 comment · Fixed by #4094
Assignees
@jleibs
Labels
🌊 C++ API C/C++ API specific enhancement New feature or request 🚀 performance Optimization, memory use, etc
Milestone
0.10.1

Comments

@emilk
Copy link
Member

emilk commented Oct 30, 2023 •
edited
Loading

The C++ logging SDK is adding a lot of overhead, and we need to figure out why.

Reportedly 2ec5ce7 was much faster than 0.10.0, so something in 2ec5ce7...0.10.0 broke (TODO: verify).
Maybe c0521ee causes a Debug build of arrow-cpp to be built or something?

$ time RERUN=on ./build/rerun_vrs_example sample.vrs
real    0m15.105s
user    0m16.409s
sys     0m2.981s

$ time RERUN=off ./build/rerun_vrs_example sample.vrs
real    0m1.060s
user    0m1.043s
sys     0m0.017s

More info: https://rerunio.slack.com/archives/C04UBTMMGJK/p1698696318766679
@emilk emilk added enhancement New feature or request 👀 needs triage This issue needs to be triaged by the Rerun team 🚀 performance Optimization, memory use, etc 🌊 C++ API C/C++ API specific and removed 👀 needs triage This issue needs to be triaged by the Rerun team labels Oct 30, 2023
@emilk emilk added this to the 0.10.1 milestone Oct 30, 2023
@jleibs
Copy link
Member

jleibs commented Oct 30, 2023 •
edited
Loading

Using -DRERUN_DOWNLOAD_AND_BUILD_ARROW=off definitely seems to alleviate the problem.

With: -DRERUN_DOWNLOAD_AND_BUILD_ARROW=on

$ time RERUN=on ./build/rerun_vrs_example sample.vrs
real    0m15.105s
user    0m16.409s
sys     0m2.981s

With: -DRERUN_DOWNLOAD_AND_BUILD_ARROW=off

$ time RERUN=on ./build/rerun_vrs_example sample.vrs
real    0m3.643s
user    0m4.754s
sys     0m1.643s

Note, there's still a fair bit of overhead relative to having logging off all together:

$ time RERUN=off ./build/rerun_vrs_example sample.vrs
real    0m0.973s
user    0m0.952s
sys     0m0.022s

@jleibs jleibs mentioned this issue Oct 30, 2023
Merged
7 tasks
jleibs added a commit that referenced this issue Oct 31, 2023
@jleibs @Wumpf
Improve C++ SDK perf 5x by respecting CMAKE_BUILD_TYPE and enabling m…
a143e9d 
…imalloc (#4094)

### What
- Resolves: #4093

Arrow was default to a Debug build regardless of what CMAKE_BUILD_TYPE
was specified.

Baseline:
```
$ time RERUN=on ./build/rerun_vrs_example sample.vrs
real    0m15.105s
user    0m16.409s
sys     0m2.981s
```

`CMAKE_BUILD_TYPE=RelWithDebInfo`
```
$ time RERUN=on ./build/rerun_vrs_example sample.vrs
real    0m4.659s
user    0m5.259s
sys     0m2.257s
```

`-DARROW_MIMALLOC=ON`
```
$ time RERUN=on ./build/rerun_vrs_example sample.vrs
real    0m3.477s
user    0m5.015s
sys     0m1.291s
```

### Test
* [x] Linux
* [x] Mac
* [x] Windows, rerun build & test only

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/4094) (if
applicable)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG

- [PR Build Summary](https://build.rerun.io/pr/4094)
- [Docs
preview](https://rerun.io/preview/73f97330c4d4e8cbac82e7e0c0ea7e9d24fcd3c7/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/73f97330c4d4e8cbac82e7e0c0ea7e9d24fcd3c7/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://ref.rerun.io/dev/bench/)
- [Wasm size tracking](https://ref.rerun.io/dev/sizes/)

---------

Co-authored-by: Andreas Reich <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jleibs jleibs

Labels
🌊 C++ API C/C++ API specific enhancement New feature or request 🚀 performance Optimization, memory use, etc
Projects
None yet
Milestone
0.10.1
Development

Successfully merging a pull request may close this issue.

Improve C++ SDK perf 5x by respecting CMAKE_BUILD_TYPE and enabling mimalloc rerun-io/rerun
2 participants
@emilk @jleibs