Skip to content

fix: orchestrator enqueue yield #21286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PhilWindle merged 1 commit into from
Mar 13, 2026
Merged

fix: orchestrator enqueue yield #21286

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions yarn-project/prover-client/src/orchestrator/orchestrator.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import { type Logger, type LoggerBindings, createLogger } from '@aztec/foundatio
import { promiseWithResolvers } from '@aztec/foundation/promise';
import { SerialQueue } from '@aztec/foundation/queue';
import { assertLength } from '@aztec/foundation/serialize';
import { sleep } from '@aztec/foundation/sleep';
import { pushTestData } from '@aztec/foundation/testing';
import { elapsed } from '@aztec/foundation/timer';
import type { TreeNodeLocation } from '@aztec/foundation/trees';
Expand Down Expand Up @@ -634,9 +635,11 @@ export class ProvingOrchestrator implements EpochProver {
}
};

// Enqueue onto the serial queue with limited workers to avoid starving the event loop.
// Workers yield between jobs via await, allowing I/O callbacks to process.
void this.deferredJobQueue.put(() => safeJob());
void this.deferredJobQueue.put(async () => {
void safeJob();
Copy link
Collaborator

@ludamad ludamad Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also do await safeJob() but I guess you'd have to fight the linter. (any await yields)

Copy link
Contributor Author

@alexghr alexghr Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awaiting here would introduce a limit on how many proofs we have in flight at a time. We want to use fire-and-forget here and use callbacks to continue up the rollup

// we yield here to the macro task queue such to give Nodejs a chance to run other operatoins in between enqueues
Copy link
Collaborator

@PhilWindle PhilWindle Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand this change. Doesn't this negate the purpose of having a queue and effectively take us back to setImmediate(() => void safeJob())?

await sleep(0);
});
}

private async updateL1ToL2MessageTree(l1ToL2Messages: Fr[], db: MerkleTreeWriteOperations) {
Expand Down
2 changes: 1 addition & 1 deletion yarn-project/stdlib/src/interfaces/prover-client.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ export const proverConfigMappings: ConfigMappingsType<ProverConfig> = {
enqueueConcurrency: {
env: 'PROVER_ENQUEUE_CONCURRENCY',
description: 'Max concurrent jobs the orchestrator serializes and enqueues to the broker.',
...numberConfigHelper(10),
...numberConfigHelper(50),
},
};

Expand Down
Loading