maths: switch dot to faster for loop

[pulsejet]

Tiny optimization; the for loop warms up faster than the reduce.

import { dot } from "@xenova/transformers";

function dotNew(arr1, arr2) {
    let result = 0;
    for (let i = 0; i < arr1.length; i++) {
        result += arr1[i] * arr2[i];
    }
    return result;
}

const random1 = Array.from({ length: 10000 }, () => Array.from({ length: 512 }, () => Math.random()));
const random2 = Array.from({ length: 10000 }, () => Array.from({ length: 512 }, () => Math.random()));

// Replaced with dotNew for second run
console.time("dot");
for (let i = 0; i < 10000; i++) {
    dot(random1[i], random2[i]);
}
console.timeEnd("dot");

Output on node v20.5.1

dot: 43.757ms
dotNew: 11.982ms

Of course, eventually they're exactly the same. With 100k elements:

dot: 109.57ms
dotNew: 76.428ms

[xenova]

Oh interesting! Thanks for this! Could you do some benchmarks with typed arrays? Specifically, Float32Array?

[pulsejet]

Well the current dot function doesn't work at all on Float32Array (another reason to switch to the new one) because there's no reduce.

Typed arrays are extremely fast.

dot: 43.757ms
dotNew: 11.982ms
dotNew: 0.887ms  (Float32Array)
dotNew: 0.919ms (Float64Array)

With 100k elements

dot: 109.57ms
dotNew: 76.428ms
dotNew: 3.237ms (Float32Array)
dotNew: 3.167ms (Float64Array)

[xenova]

Just an update: This will be merged and included in the next release. 😇 I'm just finalizing a few other things first.

[pulsejet]

No worries, take your time.

Thanks for this awesome library 😎

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[xenova]

Just for the fun of it, I was doing some benchmarking myself (both in browser and node), and your update is consistently 5-10x faster (of course, depends on the array size)! Honestly, I'm quite surprised, since I would expect this to be a common pattern, and for reduce to be optimized for this case.

Thanks!