Ensure compatibility with numpy 2.0.0

[KennethEnevoldsen]

Following the conversion guide, copy=False is no longer required and will result in an error: https://numpy.org/devdocs/numpy_2_0_migration_guide.html#adapting-to-changes-in-the-copy-keyword.

The following fix should resolve the issue.

error found during testing on the MTEB repository e.g. here

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[github-actions]

Show benchmarks

PyArrow==8.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.005361 / 0.011353 (-0.005992)	0.003983 / 0.011008 (-0.007025)	0.062865 / 0.038508 (0.024357)	0.029880 / 0.023109 (0.006771)	0.261465 / 0.275898 (-0.014433)	0.269791 / 0.323480 (-0.053689)	0.004198 / 0.007986 (-0.003788)	0.002942 / 0.004328 (-0.001387)	0.049002 / 0.004250 (0.044751)	0.043232 / 0.037052 (0.006180)	0.328774 / 0.258489 (0.070285)	0.297308 / 0.293841 (0.003467)	0.030552 / 0.128546 (-0.097994)	0.012632 / 0.075646 (-0.063015)	0.204156 / 0.419271 (-0.215116)	0.036014 / 0.043533 (-0.007519)	0.241224 / 0.255139 (-0.013915)	0.268358 / 0.283200 (-0.014842)	0.019227 / 0.141683 (-0.122456)	1.114515 / 1.452155 (-0.337639)	1.147029 / 1.492716 (-0.345688)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.094925 / 0.018006 (0.076919)	0.301548 / 0.000490 (0.301059)	0.000211 / 0.000200 (0.000011)	0.000043 / 0.000054 (-0.000011)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.018875 / 0.037411 (-0.018536)	0.062824 / 0.014526 (0.048298)	0.075657 / 0.176557 (-0.100900)	0.121926 / 0.737135 (-0.615209)	0.077102 / 0.296338 (-0.219236)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.286018 / 0.215209 (0.070808)	2.832222 / 2.077655 (0.754567)	1.462629 / 1.504120 (-0.041491)	1.354746 / 1.541195 (-0.186449)	1.339504 / 1.468490 (-0.128986)	0.718381 / 4.584777 (-3.866396)	2.401456 / 3.745712 (-1.344256)	3.013518 / 5.269862 (-2.256343)	1.944892 / 4.565676 (-2.620784)	0.078793 / 0.424275 (-0.345482)	0.005219 / 0.007607 (-0.002388)	0.349551 / 0.226044 (0.123507)	3.417844 / 2.268929 (1.148916)	1.830669 / 55.444624 (-53.613956)	1.502134 / 6.876477 (-5.374343)	1.529242 / 2.142072 (-0.612830)	0.793732 / 4.805227 (-4.011495)	0.133571 / 6.500664 (-6.367093)	0.042588 / 0.075469 (-0.032881)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	0.988167 / 1.841788 (-0.853620)	11.926728 / 8.074308 (3.852420)	9.806971 / 10.191392 (-0.384421)	0.173951 / 0.680424 (-0.506473)	0.015308 / 0.534201 (-0.518893)	0.310768 / 0.579283 (-0.268515)	0.268261 / 0.434364 (-0.166103)	0.342962 / 0.540337 (-0.197375)	0.431255 / 1.386936 (-0.955681)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.005680 / 0.011353 (-0.005673)	0.004231 / 0.011008 (-0.006778)	0.051009 / 0.038508 (0.012501)	0.031431 / 0.023109 (0.008322)	0.268582 / 0.275898 (-0.007316)	0.287942 / 0.323480 (-0.035538)	0.004442 / 0.007986 (-0.003543)	0.002818 / 0.004328 (-0.001511)	0.050241 / 0.004250 (0.045991)	0.039933 / 0.037052 (0.002881)	0.285814 / 0.258489 (0.027325)	0.316082 / 0.293841 (0.022241)	0.032416 / 0.128546 (-0.096130)	0.012398 / 0.075646 (-0.063248)	0.060779 / 0.419271 (-0.358493)	0.033706 / 0.043533 (-0.009827)	0.273915 / 0.255139 (0.018776)	0.289752 / 0.283200 (0.006553)	0.017859 / 0.141683 (-0.123824)	1.150224 / 1.452155 (-0.301930)	1.197467 / 1.492716 (-0.295250)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.093810 / 0.018006 (0.075803)	0.302529 / 0.000490 (0.302039)	0.000221 / 0.000200 (0.000021)	0.000047 / 0.000054 (-0.000008)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.022903 / 0.037411 (-0.014508)	0.077445 / 0.014526 (0.062919)	0.089335 / 0.176557 (-0.087222)	0.130848 / 0.737135 (-0.606287)	0.091106 / 0.296338 (-0.205232)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.294194 / 0.215209 (0.078985)	2.886983 / 2.077655 (0.809328)	1.557768 / 1.504120 (0.053648)	1.424467 / 1.541195 (-0.116727)	1.440625 / 1.468490 (-0.027865)	0.724793 / 4.584777 (-3.859984)	0.985216 / 3.745712 (-2.760496)	2.856826 / 5.269862 (-2.413036)	1.911638 / 4.565676 (-2.654039)	0.080350 / 0.424275 (-0.343925)	0.005616 / 0.007607 (-0.001991)	0.348713 / 0.226044 (0.122668)	3.414764 / 2.268929 (1.145835)	1.925056 / 55.444624 (-53.519568)	1.635752 / 6.876477 (-5.240725)	1.761117 / 2.142072 (-0.380955)	0.808309 / 4.805227 (-3.996918)	0.136893 / 6.500664 (-6.363771)	0.042116 / 0.075469 (-0.033354)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	1.004740 / 1.841788 (-0.837048)	12.495859 / 8.074308 (4.421550)	10.681233 / 10.191392 (0.489841)	0.133320 / 0.680424 (-0.547104)	0.015943 / 0.534201 (-0.518258)	0.304869 / 0.579283 (-0.274414)	0.128616 / 0.434364 (-0.305748)	0.345930 / 0.540337 (-0.194407)	0.457434 / 1.386936 (-0.929502)