-
Notifications
You must be signed in to change notification settings - Fork 38
feat: Voice Commands for Hands-Free Trail Navigation #1904
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
524aea3
Initial plan
Copilot 3f1e4e9
feat: add voice commands for hands-free trail navigation
Copilot 0a8837e
fix: address CodeRabbit review issues in voice commands feature
Copilot e84811c
fix: biome format/lint issues in voice commands files
Copilot 34baf21
fix: address Copilot inline review comments on voice commands feature
Copilot 26b404a
fix(voice): wire expo-av audio recording to mic button (Issue 1)
andrew-bierman 30d28e0
fix(voice): fix timer leaks, command ambiguity, and invalid i18n JSON
andrew-bierman e134c8f
fix(voice): fix TypeScript errors — icon name, AccessibilityState, te…
andrew-bierman c053750
fix(voice): fix biome errors in useVoiceCommands test and home index
andrew-bierman c16ef51
fix(ci): resolve all biome, circular dep, and catalog violations
andrew-bierman e5aa7a3
fix(types): use .at(0) guard instead of non-null assertion
andrew-bierman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| import { VoiceCommandScreen } from 'expo-app/features/voice/screens/VoiceCommandScreen'; | ||
| import { useColorScheme } from 'expo-app/lib/hooks/useColorScheme'; | ||
| import { StatusBar } from 'expo-status-bar'; | ||
| import { Platform } from 'react-native'; | ||
|
|
||
| export default function VoiceCommandsRoute() { | ||
| const { colorScheme } = useColorScheme(); | ||
|
|
||
| return ( | ||
| <> | ||
| <StatusBar | ||
| style={Platform.OS === 'ios' ? 'light' : colorScheme === 'dark' ? 'light' : 'dark'} | ||
| /> | ||
| <VoiceCommandScreen /> | ||
| </> | ||
| ); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| import type { WeightUnit } from 'expo-app/types'; | ||
|
|
||
| export interface PackItemInput { | ||
| name: string; | ||
| description?: string; | ||
| weight: number; | ||
| weightUnit: WeightUnit; | ||
| quantity: number; | ||
| category?: string; | ||
| consumable: boolean; | ||
| worn: boolean; | ||
| notes?: string; | ||
| image?: string | null; | ||
| catalogItemId?: number; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
144 changes: 144 additions & 0 deletions
144
apps/expo/features/voice/components/VoiceCommandPanel.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| import { Text } from '@packrat/ui/nativewindui'; | ||
| import { Icon, type MaterialIconName } from '@roninoss/icons'; | ||
| import { useTranslation } from 'expo-app/lib/hooks/useTranslation'; | ||
| import { Pressable, View } from 'react-native'; | ||
| import type { VoiceCommand, VoiceListeningState } from '../types'; | ||
|
|
||
| interface VoiceCommandPanelProps { | ||
| listeningState: VoiceListeningState; | ||
| lastTranscript: string; | ||
| lastCommand: string | null; | ||
| isTracking: boolean; | ||
| waypointCount: number; | ||
| availableCommands: VoiceCommand[]; | ||
| onStartListening: () => void; | ||
| onStopListening: () => void; | ||
| onTestCommand?: (transcript: string) => void; | ||
| } | ||
|
|
||
| const STATE_COLORS: Record<VoiceListeningState, string> = { | ||
| idle: 'bg-muted', | ||
| listening: 'bg-red-500', | ||
| processing: 'bg-amber-500', | ||
| error: 'bg-destructive', | ||
| }; | ||
|
|
||
| const STATE_ICONS: Record<VoiceListeningState, MaterialIconName> = { | ||
| idle: 'microphone-outline', | ||
| listening: 'microphone', | ||
| processing: 'dots-horizontal', | ||
| error: 'information', | ||
| }; | ||
|
|
||
| /** | ||
| * Main voice command control panel. | ||
| * Shows the microphone button, current status, last transcript, | ||
| * and a reference list of available commands. | ||
| */ | ||
| export function VoiceCommandPanel({ | ||
| listeningState, | ||
| lastTranscript, | ||
| lastCommand, | ||
| isTracking, | ||
| waypointCount, | ||
| availableCommands, | ||
| onStartListening, | ||
| onStopListening, | ||
| onTestCommand, | ||
| }: VoiceCommandPanelProps) { | ||
| const { t } = useTranslation(); | ||
|
|
||
| const isListening = listeningState === 'listening'; | ||
| const micBgClass = STATE_COLORS[listeningState]; | ||
| const micIcon = STATE_ICONS[listeningState]; | ||
|
|
||
| return ( | ||
| <View className="gap-6"> | ||
| {/* Status Row */} | ||
| <View className="flex-row items-center gap-4 rounded-2xl bg-card p-4 border border-border"> | ||
| <View | ||
| className={`h-3 w-3 rounded-full ${isTracking ? 'bg-green-500' : 'bg-muted-foreground'}`} | ||
| /> | ||
| <Text className="text-sm text-foreground font-medium"> | ||
| {isTracking ? t('voice.trackingActive') : t('voice.trackingInactive')} | ||
| </Text> | ||
| <View className="ml-auto flex-row items-center gap-1"> | ||
| <Icon name="map-marker-outline" size={14} color="#6b7280" /> | ||
| <Text className="text-sm text-muted-foreground"> | ||
| {t('voice.waypointsCount', { count: waypointCount })} | ||
| </Text> | ||
| </View> | ||
| </View> | ||
|
|
||
| {/* Microphone Button */} | ||
| <View className="items-center gap-3"> | ||
| <Pressable | ||
| onPressIn={onStartListening} | ||
| onPressOut={onStopListening} | ||
| className={`h-24 w-24 items-center justify-center rounded-full ${micBgClass}`} | ||
| accessibilityRole="button" | ||
| accessibilityLabel={isListening ? t('voice.stopListening') : t('voice.startListening')} | ||
| accessibilityState={{ selected: isListening }} | ||
| > | ||
| <Icon name={micIcon} size={40} color="white" /> | ||
| </Pressable> | ||
| <Text className="text-base font-semibold text-foreground"> | ||
| {listeningState === 'idle' && t('voice.holdToSpeak')} | ||
| {listeningState === 'listening' && t('voice.listening')} | ||
| {listeningState === 'processing' && t('voice.processing')} | ||
| {listeningState === 'error' && t('voice.error')} | ||
| </Text> | ||
| <Text className="text-xs text-muted-foreground text-center"> | ||
| {t('voice.holdMicDescription')} | ||
| </Text> | ||
| </View> | ||
|
|
||
| {/* Last Transcript */} | ||
| {lastTranscript ? ( | ||
| <View className="rounded-xl bg-card border border-border p-4 gap-1"> | ||
| <Text className="text-xs font-semibold uppercase tracking-wide text-muted-foreground"> | ||
| {t('voice.lastHeard')} | ||
| </Text> | ||
| <Text className="text-base text-foreground">"{lastTranscript}"</Text> | ||
| {lastCommand && ( | ||
| <Text className="text-xs text-primary mt-1"> | ||
| ✓ {t('voice.executedCommand', { command: lastCommand.replace(/_/g, ' ') })} | ||
| </Text> | ||
| )} | ||
| </View> | ||
| ) : null} | ||
|
|
||
| {/* Available Commands Reference */} | ||
| <View className="gap-2"> | ||
| <Text className="text-sm font-semibold uppercase tracking-wide text-muted-foreground px-1"> | ||
| {t('voice.availableCommands')} | ||
| </Text> | ||
| <View className="rounded-xl bg-card border border-border overflow-hidden"> | ||
| {availableCommands.map((cmd, index) => ( | ||
| <Pressable | ||
| key={cmd.name} | ||
| onPress={() => onTestCommand?.(cmd.patterns[0] ?? '')} | ||
| className={`p-3 flex-row items-center gap-3 ${ | ||
| index < availableCommands.length - 1 ? 'border-b border-border' : '' | ||
| }`} | ||
| accessibilityRole="button" | ||
| accessibilityLabel={`${t('voice.testCommand')} ${cmd.patterns[0]}`} | ||
| > | ||
| <View className="h-8 w-8 items-center justify-center rounded-full bg-violet-500/10"> | ||
| <Icon name="microphone-outline" size={14} color="#7c3aed" /> | ||
| </View> | ||
| <View className="flex-1"> | ||
| <Text className="text-sm font-medium text-foreground">"{cmd.patterns[0]}"</Text> | ||
| <Text className="text-xs text-muted-foreground">{cmd.description}</Text> | ||
| </View> | ||
| <Icon name="chevron-right" size={14} color="#9ca3af" /> | ||
| </Pressable> | ||
| ))} | ||
| </View> | ||
| <Text className="text-xs text-muted-foreground text-center px-4"> | ||
| {t('voice.tapCommandToTest')} | ||
| </Text> | ||
| </View> | ||
| </View> | ||
| ); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| import { ListItem } from '@packrat/ui/nativewindui'; | ||
| import { Icon } from '@roninoss/icons'; | ||
| import { featureFlags } from 'expo-app/config'; | ||
| import { useColorScheme } from 'expo-app/lib/hooks/useColorScheme'; | ||
| import { useTranslation } from 'expo-app/lib/hooks/useTranslation'; | ||
| import { useRouter } from 'expo-router'; | ||
| import { Platform, View } from 'react-native'; | ||
|
|
||
| /** | ||
| * Dashboard tile that navigates to the voice commands screen. | ||
| */ | ||
| export function VoiceCommandsTile() { | ||
| const router = useRouter(); | ||
| const { colors } = useColorScheme(); | ||
| const { t } = useTranslation(); | ||
|
|
||
| if (!featureFlags.enableVoiceCommands) return null; | ||
|
|
||
| return ( | ||
| <View> | ||
| <ListItem | ||
| className="ios:pl-0 pl-2" | ||
| titleClassName="text-lg" | ||
| leftView={ | ||
| <View className="px-3"> | ||
| <View className="h-6 w-6 items-center justify-center rounded-md bg-violet-500"> | ||
| <Icon name="microphone" size={15} color="white" /> | ||
| </View> | ||
| </View> | ||
| } | ||
| rightView={ | ||
| <View className="flex-1 flex-row items-center justify-center gap-2 px-4"> | ||
| <Icon name="chevron-right" size={17} color={colors.grey} /> | ||
| </View> | ||
| } | ||
| item={{ | ||
| title: t('voice.voiceCommands'), | ||
| subTitle: t('voice.handsFreeNavigation'), | ||
| }} | ||
| onPress={() => router.push('/voice-commands')} | ||
| target="Cell" | ||
| index={0} | ||
| removeSeparator={Platform.OS === 'ios'} | ||
| /> | ||
| </View> | ||
| ); | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
Textcomponent is imported from@packrat/ui/nativewinduibut is never used in this file. The component only usesListItem. This unused import should be removed.