Repository navigation
feat(cli): add audio playback support for TTS output #982
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,6 +39,7 @@ import { handleError } from "../errorHandler.js"; | |
| import { LoopSession } from "../loop/session.js"; | ||
| import { initializeCliParser } from "../parser.js"; | ||
| import { formatFileSize, saveAudioToFile } from "../utils/audioFileUtils.js"; | ||
| import { playAudio } from "../utils/audioPlayer.js"; | ||
| import { resolveFilePaths } from "../utils/pathResolver.js"; | ||
| import { animatedWrite } from "../utils/typewriter.js"; | ||
| import { createStreamAbortHandler } from "../utils/abortHandler.js"; | ||
|
|
@@ -925,16 +926,19 @@ export class CLICommandFactory { | |
| } | ||
|
|
||
| /** | ||
| * Helper method to handle TTS audio file output | ||
| * Helper method to handle TTS audio file output and playback | ||
| * Saves audio to file when --tts-output flag is provided | ||
| * Plays audio when --tts-play flag is provided | ||
| */ | ||
| private static async handleTTSOutput( | ||
| result: CliGenerateResult | unknown, | ||
| options: BaseCommandArgs & Record<string, unknown>, | ||
| ): Promise<void> { | ||
| // Check if --tts-output flag is provided | ||
| const ttsOutputPath = options.ttsOutput as string | undefined; | ||
| if (!ttsOutputPath) { | ||
| const shouldPlay = options.ttsPlay as boolean | undefined; | ||
|
|
||
| // Nothing to do if neither save nor play is requested | ||
| if (!ttsOutputPath && !shouldPlay) { | ||
| return; | ||
| } | ||
|
|
||
|
|
@@ -956,26 +960,48 @@ export class CLICommandFactory { | |
| return; | ||
| } | ||
|
|
||
| try { | ||
| // Save audio to file | ||
| const saveResult = await saveAudioToFile(audio, ttsOutputPath); | ||
| // Save audio to file if --tts-output is provided | ||
| if (ttsOutputPath) { | ||
| try { | ||
| const saveResult = await saveAudioToFile(audio, ttsOutputPath); | ||
|
|
||
| if (saveResult.success) { | ||
| if (!options.quiet) { | ||
| logger.always( | ||
| chalk.green( | ||
| `🔊 Audio saved to: ${saveResult.path} (${formatFileSize(saveResult.size)})`, | ||
| ), | ||
| if (saveResult.success) { | ||
| if (!options.quiet) { | ||
| logger.always( | ||
| chalk.green( | ||
| `🔊 Audio saved to: ${saveResult.path} (${formatFileSize(saveResult.size)})`, | ||
| ), | ||
| ); | ||
| } | ||
| } else { | ||
| handleError( | ||
| new Error(saveResult.error || "Failed to save audio file"), | ||
| "TTS Output", | ||
| ); | ||
| } | ||
| } else { | ||
| handleError( | ||
| new Error(saveResult.error || "Failed to save audio file"), | ||
| "TTS Output", | ||
| } catch (error) { | ||
| handleError(error as Error, "TTS Output"); | ||
| } | ||
| } | ||
|
|
||
| // Play audio if --tts-play is provided | ||
| if (shouldPlay) { | ||
| try { | ||
| if (!options.quiet) { | ||
| logger.always(chalk.blue("Playing audio...")); | ||
| } | ||
| await playAudio(audio.buffer, audio.format); | ||
| } catch (err) { | ||
| // Non-fatal: warn but don't crash | ||
| logger.always( | ||
| chalk.yellow(`Audio playback failed: ${(err as Error).message}`), | ||
| ); | ||
| logger.always( | ||
| chalk.yellow( | ||
| " Tip: Save the audio with --tts-output <file> and play manually.", | ||
| ), | ||
| ); | ||
| } | ||
| } catch (error) { | ||
| handleError(error as Error, "TTS Output"); | ||
| } | ||
|
Comment on lines
+963
to
1005
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Verify whether handleError exits the process or just logs
ast-grep --pattern $'export function handleError($$$) { $$$ }'
rg -nP -C2 '\bhandleError\b' src/cli/errorHandler.tsRepository: juspay/neurolink Length of output: 282 🏁 Script executed: cat -n src/cli/errorHandler.tsRepository: juspay/neurolink Length of output: 2640 🏁 Script executed: # Find the TTS handler context and session setup
rg -n "globalSession\|getCurrentSessionId" src/cli/factories/commandFactory.ts | head -20Repository: juspay/neurolink Length of output: 42 🏁 Script executed: # Check the broader context around the TTS code to see if we're in a session
sed -n '930,965p' src/cli/factories/commandFactory.tsRepository: juspay/neurolink Length of output: 1137 🏁 Script executed: # Find where handleTTSOutput is called
rg -n "handleTTSOutput" src/cli/factories/commandFactory.tsRepository: juspay/neurolink Length of output: 173 🏁 Script executed: # Check the broader command execution context to understand session lifecycle
rg -n "globalSession\|getCurrentSessionId" src/cli/factories/commandFactory.tsRepository: juspay/neurolink Length of output: 42 🏁 Script executed: # Check context around the handleTTSOutput call at line 2480
sed -n '2450,2490p' src/cli/factories/commandFactory.tsRepository: juspay/neurolink Length of output: 1513 🏁 Script executed: # Also check if there's any session management around the generate command execution
rg -n "async.*generate|async.*execute" src/cli/factories/commandFactory.ts | head -10Repository: juspay/neurolink Length of output: 710 🏁 Script executed: # Check the executeGenerate method to see session context
sed -n '2511,2600p' src/cli/factories/commandFactory.tsRepository: juspay/neurolink Length of output: 3272 🏁 Script executed: # Search for where globalSession is initialized/managed
rg -n "globalSession\|getCurrentSessionId" src/cli --include="*.ts" | head -20Repository: juspay/neurolink Length of output: 501 🏁 Script executed: # Fix the rg command syntax and search for globalSession initialization
rg -n "globalSession" src/cli --type ts | head -20Repository: juspay/neurolink Length of output: 1762 🏁 Script executed: # Check CLI entry point and main command handler
rg -n "globalSession\|executeGenerate\|executeStream" src/cli/index.tsRepository: juspay/neurolink Length of output: 42 🏁 Script executed: # Check parser.ts to understand session initialization
sed -n '1,100p' src/cli/parser.tsRepository: juspay/neurolink Length of output: 4039 🏁 Script executed: # Check where commands are dispatched from
rg -n "executeGenerate\|executeStream\|handleError" src/cli/parser.ts -A 3 -B 3Repository: juspay/neurolink Length of output: 42 🏁 Script executed: # Check if session ID is set anywhere for single commands vs loop
rg -n "setLoopSession\|setSessionId" src/lib/session/globalSessionState.tsRepository: juspay/neurolink Length of output: 42 🏁 Script executed: # Check globalSessionState to understand when session ID is set
sed -n '1,100p' src/lib/session/globalSessionState.tsRepository: juspay/neurolink Length of output: 3170 🏁 Script executed: # Check getCurrentSessionId implementation to confirm it returns null for single commands
rg -n "getCurrentSessionId" src/lib/session/globalSessionState.ts -A 5Repository: juspay/neurolink Length of output: 283 🏁 Script executed: # Verify the complete control flow: single command → executeGenerate → handleTTSOutput → handleError
sed -n '2568,2590p' src/cli/factories/commandFactory.tsRepository: juspay/neurolink Length of output: 1036 Save-failure short-circuits playback in single-command mode. When Suggested ordering- // Save audio to file if --tts-output is provided
- if (ttsOutputPath) {
- try {
- const saveResult = await saveAudioToFile(audio, ttsOutputPath);
- ...
- } catch (error) {
- handleError(error as Error, "TTS Output");
- }
- }
-
- // Play audio if --tts-play is provided
- if (shouldPlay) { ... }
+ // Play audio first so save failures cannot block playback
+ if (shouldPlay) {
+ try {
+ if (!options.quiet) logger.always(chalk.blue("Playing audio..."));
+ await playAudio(audio.buffer, audio.format);
+ } catch (err) {
+ logger.always(chalk.yellow(`Audio playback failed: ${(err as Error).message}`));
+ logger.always(chalk.yellow(" Tip: Save the audio with --tts-output <file> and play manually."));
+ }
+ }
+
+ if (ttsOutputPath) {
+ try {
+ const saveResult = await saveAudioToFile(audio, ttsOutputPath);
+ ...
+ } catch (error) {
+ handleError(error as Error, "TTS Output");
+ }
+ }🤖 Prompt for AI Agents |
||
| } | ||
|
|
||
|
|
@@ -3224,17 +3250,18 @@ export class CLICommandFactory { | |
| } | ||
| } | ||
|
|
||
| // Handle TTS audio output if --tts-output is provided | ||
| // Handle TTS audio output/playback if --tts-output or --tts-play is provided | ||
| // Note: For streaming, TTS audio is collected during the stream | ||
| // and saved at the end if available | ||
| const ttsOutputPath = options.ttsOutput as string | undefined; | ||
| if (ttsOutputPath) { | ||
| const shouldPlay = options.ttsPlay as boolean | undefined; | ||
| if (ttsOutputPath || shouldPlay) { | ||
| // For now, streaming TTS output is not yet available | ||
| // This will be enabled when the TTS streaming infrastructure is complete | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 💡 SUGGESTION: Emoji inconsistency in warning message The modified warning message removed the
This is inconsistent with the established codebase pattern where warning messages use Suggested fix: logger.always(
chalk.yellow(
"⚠️ TTS audio for streaming is not yet available. Use 'generate' command for TTS output.",
),
); |
||
| if (!options.quiet) { | ||
| logger.always( | ||
| chalk.yellow( | ||
| "⚠️ TTS audio output for streaming is not yet available. Use 'generate' command for TTS output.", | ||
| "TTS audio for streaming is not yet available. Use 'generate' command for TTS output.", | ||
| ), | ||
| ); | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| /** | ||
| * Audio playback utilities for CLI | ||
| * | ||
| * Provides functionality for playing TTS audio using platform-specific | ||
| * CLI tools with proper cleanup and error handling. | ||
| * | ||
| * @module cli/utils/audioPlayer | ||
| */ | ||
|
|
||
| import { execFile } from "node:child_process"; | ||
| import fs from "node:fs"; | ||
| import os from "node:os"; | ||
| import path from "node:path"; | ||
| import { promisify } from "node:util"; | ||
|
|
||
| import type { AudioFormat } from "../../lib/types/index.js"; | ||
|
|
||
| const execFileAsync = promisify(execFile); | ||
|
|
||
| /** | ||
| * Get the file extension for an audio format | ||
| * | ||
| * @param format - Audio format | ||
| * @returns File extension string (e.g., "mp3", "wav") | ||
| */ | ||
| export function getAudioExtension(format: AudioFormat): string { | ||
| switch (format) { | ||
| case "mp3": | ||
| return "mp3"; | ||
| case "wav": | ||
| return "wav"; | ||
| case "ogg": | ||
| return "ogg"; | ||
| case "opus": | ||
| return "opus"; | ||
| default: | ||
| return "mp3"; | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Get the platform-specific audio player command and arguments | ||
| * | ||
| * @param filePath - Path to the audio file | ||
| * @param format - Audio format | ||
| * @returns Object with command and args for execFile | ||
| */ | ||
| function getPlayerCommand( | ||
| filePath: string, | ||
| format: AudioFormat, | ||
| ): { command: string; args: string[] } { | ||
| const platform = process.platform; | ||
|
|
||
| switch (platform) { | ||
| case "darwin": | ||
| return { command: "afplay", args: [filePath] }; | ||
|
|
||
| case "linux": | ||
| if (format === "wav") { | ||
| return { command: "aplay", args: [filePath] }; | ||
| } | ||
| return { command: "paplay", args: [filePath] }; | ||
|
|
||
|
Comment on lines
+58
to
+63
|
||
| case "win32": | ||
| if (format === "wav") { | ||
| return { | ||
| command: "powershell", | ||
| args: [ | ||
| "-NoProfile", | ||
| "-Command", | ||
| `(New-Object System.Media.SoundPlayer '${filePath}').PlaySync()`, | ||
| ], | ||
|
Comment on lines
+67
to
+72
|
||
| }; | ||
| } | ||
| return { | ||
| command: "powershell", | ||
| args: [ | ||
| "-NoProfile", | ||
| "-Command", | ||
| `$player = New-Object -ComObject WMPlayer.OCX; $player.URL = '${filePath}'; $player.controls.play(); Start-Sleep -Seconds 1; while ($player.playState -eq 3) { Start-Sleep -Milliseconds 100 }; $player.close()`, | ||
| ], | ||
|
Comment on lines
+76
to
+81
|
||
| }; | ||
|
Comment on lines
+64
to
+82
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. PowerShell command uses single-quoted interpolation — breaks/injects if the temp path contains
Safer options: pass the path as a parameter/argument instead of interpolating, or escape Proposed fix (argument passing) case "win32":
+ // Pass path as a PowerShell argument to avoid quoting/injection issues
if (format === "wav") {
return {
command: "powershell",
args: [
"-NoProfile",
"-Command",
- `(New-Object System.Media.SoundPlayer '${filePath}').PlaySync()`,
+ "param($p) (New-Object System.Media.SoundPlayer $p).PlaySync()",
+ "-p",
+ filePath,
],
};
}
return {
command: "powershell",
args: [
"-NoProfile",
"-Command",
- `$player = New-Object -ComObject WMPlayer.OCX; $player.URL = '${filePath}'; $player.controls.play(); Start-Sleep -Seconds 1; while ($player.playState -eq 3) { Start-Sleep -Milliseconds 100 }; $player.close()`,
+ "param($p) $player = New-Object -ComObject WMPlayer.OCX; $player.URL = $p; $player.controls.play(); Start-Sleep -Seconds 1; while ($player.playState -eq 3) { Start-Sleep -Milliseconds 100 }; $player.close()",
+ "-p",
+ filePath,
],
};🤖 Prompt for AI Agents |
||
|
|
||
| default: | ||
| throw new Error( | ||
| `Unsupported platform: ${platform}. Audio playback is supported on macOS, Linux, and Windows.`, | ||
| ); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Play audio from a buffer using platform-specific CLI tools | ||
| * | ||
| * Writes the buffer to a temporary file, plays it using the appropriate | ||
| * system audio player, and cleans up the temp file afterward. | ||
| * | ||
| * Supported platforms: | ||
| * - macOS: uses `afplay` (built-in, supports mp3/wav/aac/flac) | ||
| * - Linux: uses `paplay` for non-wav, `aplay` for wav | ||
| * - Windows: uses PowerShell SoundPlayer (wav) or WMPlayer.OCX (mp3) | ||
| * | ||
| * @param buffer - Audio data buffer | ||
| * @param format - Audio format (mp3, wav, ogg, opus) | ||
| * @throws Error if playback fails or platform is unsupported | ||
| * | ||
| * @example | ||
| * ```typescript | ||
| * await playAudio(audioBuffer, "mp3"); | ||
| * ``` | ||
| */ | ||
| export async function playAudio( | ||
| buffer: Buffer, | ||
| format: AudioFormat, | ||
| ): Promise<void> { | ||
| const ext = getAudioExtension(format); | ||
| const tempFile = path.join(os.tmpdir(), `nl-tts-${Date.now()}.${ext}`); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Temp filename based only on Two near-simultaneous -import path from "node:path";
+import path from "node:path";
+import { randomUUID } from "node:crypto";
@@
- const tempFile = path.join(os.tmpdir(), `nl-tts-${Date.now()}.${ext}`);
+ const tempFile = path.join(
+ os.tmpdir(),
+ `nl-tts-${Date.now()}-${randomUUID()}.${ext}`,
+ );🤖 Prompt for AI Agents |
||
|
|
||
|
Comment on lines
+115
to
+117
|
||
| try { | ||
| // Write audio buffer to temp file | ||
| await fs.promises.writeFile(tempFile, buffer); | ||
|
|
||
| const { command, args } = getPlayerCommand(tempFile, format); | ||
|
|
||
| try { | ||
| await execFileAsync(command, args); | ||
| } catch (execError) { | ||
| const err = execError as NodeJS.ErrnoException; | ||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The Suggested fix: // Add timeout option to prevent indefinite hangs
await execFileAsync(command, args, { timeout: 60000 }); // 60s timeoutThis aligns with the project's timeout handling patterns seen in other CLI utilities. |
||
| // Handle binary not found | ||
| if (err.code === "ENOENT") { | ||
| if (process.platform === "linux" && command === "paplay") { | ||
| // Fallback to aplay on Linux | ||
| try { | ||
| await execFileAsync("aplay", [tempFile]); | ||
| return; | ||
| } catch (fallbackError) { | ||
| const fbErr = fallbackError as NodeJS.ErrnoException; | ||
| if (fbErr.code === "ENOENT") { | ||
| throw new Error( | ||
| "Neither paplay nor aplay found. Install PulseAudio (paplay) or ALSA (aplay) for audio playback.", | ||
| { cause: fallbackError }, | ||
| ); | ||
| } | ||
| throw fallbackError; | ||
| } | ||
| } | ||
|
|
||
| throw new Error( | ||
| `Audio player '${command}' not found. Ensure it is installed and available in PATH.`, | ||
| { cause: execError }, | ||
| ); | ||
| } | ||
|
Comment on lines
+130
to
+152
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Linux The fallback only triggers when the primary command is Proposed fix if (err.code === "ENOENT") {
- if (process.platform === "linux" && command === "paplay") {
- // Fallback to aplay on Linux
- try {
- await execFileAsync("aplay", [tempFile]);
- return;
- } catch (fallbackError) {
- const fbErr = fallbackError as NodeJS.ErrnoException;
- if (fbErr.code === "ENOENT") {
- throw new Error(
- "Neither paplay nor aplay found. Install PulseAudio (paplay) or ALSA (aplay) for audio playback.",
- { cause: fallbackError },
- );
- }
- throw fallbackError;
- }
- }
+ if (
+ process.platform === "linux" &&
+ command === "paplay" &&
+ format === "wav"
+ ) {
+ // aplay only decodes WAV/PCM; only safe to fall back for wav
+ try {
+ await execFileAsync("aplay", [tempFile]);
+ return;
+ } catch (fallbackError) {
+ const fbErr = fallbackError as NodeJS.ErrnoException;
+ if (fbErr.code === "ENOENT") {
+ throw new Error(
+ "Neither paplay nor aplay found. Install PulseAudio or ALSA for audio playback.",
+ { cause: fallbackError },
+ );
+ }
+ throw fallbackError;
+ }
+ }Note: the 🤖 Prompt for AI Agents |
||
|
|
||
| throw execError; | ||
| } | ||
| } finally { | ||
| // Always clean up temp file | ||
| try { | ||
| await fs.promises.unlink(tempFile); | ||
| } catch { | ||
| // Ignore cleanup errors | ||
| } | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
New
--tts-playbehavior is introduced here but doesn’t appear to be covered by the existing CLI TTS integration tests (e.g.,test/continuous-test-suite-tts.tscovers--tts-outputbut not--tts-play). Please add coverage that at least verifies the flag is recognized and that playback failures remain non-fatal (ideally by stubbing/guarding actual playback in CI).