Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace old host tree algorithm with new algorithm in JSON reader #17019

Merged
merged 3 commits into from
Oct 19, 2024

Conversation

karthikeyann
Copy link
Contributor

Description

This PR replaced old tree algorithm in JSON reader, with experimental algorithms and removed the experimental namespace.

Changes are old tree algorithm code removal, experimental namespace removal, code of scatter_offsets moved, always call new tree algorithm.
No functional change is made in this PR. All unit tests should pass with this change.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@karthikeyann karthikeyann added 3 - Ready for Review Ready for review by team libcudf Affects libcudf (C++/CUDA) code. cuIO cuIO issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 8, 2024
@karthikeyann karthikeyann requested a review from a team as a code owner October 8, 2024 19:40
@karthikeyann karthikeyann self-assigned this Oct 8, 2024
@karthikeyann karthikeyann requested a review from vuule October 9, 2024 17:27
@vuule
Copy link
Contributor

vuule commented Oct 10, 2024

what is the performance impact of the switch?

@karthikeyann
Copy link
Contributor Author

karthikeyann commented Oct 18, 2024

Benchmark comparison:

./benchmarks/JSON_READER_NVBENCH --benchmark json_read_io --benchmark nested_json_gpu_parser_depth --json "nested_json_gpu_parser_depth.json"
./benchmarks/JSON_READER_OPTION_NVBENCH --benchmark json_reader --json "reader_options.json"

branch-24.12 vs this PR

json_read_io

[0] NVIDIA RTX 5880 Ada Generation

io Ref Time Ref Noise Cmp Time Cmp Noise Diff %Diff Status
FILEPATH 337.697 ms 0.17% 333.087 ms 0.15% -4610.468 us -1.37% FAIL
HOST_BUFFER 243.569 ms 0.49% 242.734 ms 0.32% -834.790 us -0.34% FAIL
DEVICE_BUFFER 156.994 ms 0.49% 156.832 ms 0.18% -162.656 us -0.10% PASS

nested_json_gpu_parser_depth

[0] NVIDIA RTX 5880 Ada Generation

depth string_size Ref Time Ref Noise Cmp Time Cmp Noise Diff %Diff Status
2^1 2^20 1.643 ms 0.42% 1.621 ms 0.69% -21.960 us -1.34% FAIL
2^2 2^20 1.643 ms 0.76% 1.625 ms 0.50% -17.678 us -1.08% FAIL
2^3 2^20 5.249 ms 0.33% 5.149 ms 0.67% -100.062 us -1.91% FAIL
2^4 2^20 6.681 ms 0.50% 6.503 ms 0.50% -177.942 us -2.66% FAIL
2^1 2^22 1.900 ms 0.68% 1.887 ms 0.66% -12.583 us -0.66% PASS
2^2 2^22 1.901 ms 0.50% 1.884 ms 0.92% -16.896 us -0.89% FAIL
2^3 2^22 5.635 ms 0.54% 5.464 ms 0.50% -170.811 us -3.03% FAIL
2^4 2^22 7.072 ms 0.50% 6.907 ms 0.50% -164.793 us -2.33% FAIL
2^1 2^24 3.149 ms 0.58% 3.134 ms 0.58% -15.353 us -0.49% PASS
2^2 2^24 3.148 ms 0.49% 3.132 ms 0.50% -15.425 us -0.49% PASS
2^3 2^24 6.982 ms 0.50% 6.848 ms 0.50% -134.010 us -1.92% FAIL
2^4 2^24 8.746 ms 0.50% 8.577 ms 0.50% -168.694 us -1.93% FAIL
2^1 2^26 12.742 ms 0.24% 12.747 ms 0.21% 5.865 us 0.05% PASS
2^2 2^26 12.726 ms 0.25% 12.745 ms 0.23% 18.322 us 0.14% PASS
2^3 2^26 14.093 ms 0.17% 13.922 ms 0.17% -170.748 us -1.21% FAIL
2^4 2^26 17.765 ms 0.37% 17.595 ms 0.36% -170.727 us -0.96% FAIL
2^1 2^28 55.229 ms 0.23% 55.410 ms 0.28% 180.783 us 0.33% FAIL
2^2 2^28 55.290 ms 0.19% 55.228 ms 0.30% -61.586 us -0.11% PASS
2^3 2^28 52.710 ms 0.50% 52.806 ms 0.60% 96.309 us 0.18% PASS
2^4 2^28 71.135 ms 0.91% 71.166 ms 0.89% 31.242 us 0.04% PASS
2^1 2^30 224.089 ms 0.14% 224.347 ms 0.17% 257.710 us 0.12% PASS
2^2 2^30 224.063 ms 0.16% 224.314 ms 0.11% 251.106 us 0.11% FAIL
2^3 2^30 225.992 ms 0.26% 226.354 ms 0.22% 362.578 us 0.16% PASS
2^4 2^30 302.261 ms 0.64% 302.221 ms 0.70% -40.352 us -0.01% PASS

json_reader

[0] NVIDIA RTX 5880 Ada Generation

json_lines Ref Time Ref Noise Cmp Time Cmp Noise Diff %Diff Status
YES 236.325 ms 0.21% 236.223 ms 0.39% -101.778 us -0.04% PASS
NO 236.867 ms 0.41% 235.902 ms 0.31% -964.772 us -0.41% FAIL

@vuule Performance impact is minimal. Slightly faster on smaller size jsons.

@karthikeyann
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 1ce2526 into rapidsai:branch-24.12 Oct 19, 2024
102 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team cuIO cuIO issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

3 participants