Real-time pitch adjustment

[svgeesus]

Split out from https://github.com/WebAudio/web-audio-api/issues/938 as a separate issue because this is orthogonal to on-the-fly decoding of compressed loop buffers.

[svgeesus]

Time Stretching And PitchShifting of Audio Signals

[svgeesus]

A Brief History of Pitch Correction in Music

[jas-ableton]

Yes, this would be very useful to have built into web audio sample playback. Being able to vary the playback speed while maintaining a constant pitch, or vary pitch while maintaining a constant playback speed would allow developers to implement lots of very useful workflows that are found in common DAWs (synchronizing recorded audio to a timeline, for example).

In a DAW like Ableton Live, this concept is referred to as "warping": https://www.ableton.com/en/manual/audio-clips-tempo-and-warping/

[padenot]

The variety of possible approaches makes this a good candidate to have (a) prototype(s) using AudioWorklet, to investigate the space in terms of algorithm (PSOLA/WSOLA/phase vocoder, a million others), and API.

This would probably be a new node to have maximal flexibility.

[rtoy]

I think we also agreed that it should be a processing node and not a source node. You feed it some audio, and it will pitch adjust the input.

[rtoy]

TPAC 2020:

@padenot said this has been implemented in an AudioWorklet. Next step is to get the code and see how it works and propose a new node and API for this.

[rtoy]

@padenot found the link: https://github.com/olvb/phaze/

[padenot]

AudioWG virtual F2F summary:

• This seems very useful as a basic building block
• Two things considered:
  • Add preservesPitch defaulting to false to AudioBufferSourceNode -- like its HTMLMediaElement counterpart (but with a default value that is the opposite for compatibility), this changes from resampling to time-stretching
  • Add a new node, PitchShiftNode (placeholder name), that changes the pitch of incoming audio. This might need to delay the incoming audio by a number of samples depending on various parameters (algorithm quality, pitch shifting ratio, etc.)