implement playht and elevenlabs

[supercrafter100]

No description provided.

[shiffman]

Wow @supercrafter100 this is incredible! Sadly I'm having some significant issues with the node-speaker package. It is not currently compatible with macos, I was able to install this patch TooTallNate/node-speaker#178 with the following:

npm install ItsJustMeChris/node-speaker#master

But the vscode workspace immediately closes after playing the audio each time. I can share more in Discord. Is there another way we might play the stream from elevenlabs? Maybe we try piping it into a file and use the existing play-sound package?

This means nothing to me but I was able to produce this error message in the console with the speaker package.

[1]    40711 trace trap  node src/experimental/tts-eleven.js

[shiffman]

I've added a test function that saves it to a file (which I know I can play) but while I see a file, I don't hear anything so I must have some setting wrong?

Does doing it this way defeat the whole purpose of streaming the audio anyway?

const elevenTTS = async (text, filePath) => {
  return new Promise(async (resolve, reject) => {
    const elevenLabs = new ElevenLabsClient({
      apiKey: config.elevenLabs.apiKey,
    });

    let audioData = [];

    const stream = await elevenLabs.generate({
      stream: true,
      voice: config.elevenLabs.voiceId,
      text: text,
      model_id: config.elevenLabs.model,
      output_format: config.elevenLabs.outputFormat,
    });

    stream.on('data', (chunk) => {
      audioData.push(chunk);
    });

    stream.on('end', () => {
      const audioBuffer = Buffer.concat(audioData);
      let wav = new WaveFile();
      wav.fromScratch(1, config.elevenLabs.sampleRate, '32', audioBuffer);
      fs.writeFileSync(filePath, wav.toBuffer());
      resolve();
    });
  });
};