Skip to content

Conversation

@spencrmartin
Copy link
Collaborator

@spencrmartin spencrmartin commented Oct 2, 2025

Intelligent API Key Testing System
Smart Provider Detection: Automatically detects provider from key format
sk-ant-* → Anthropic
sk-* → OpenAI
AIza* → Google
gsk_* → Groq

Real API Validation: Tests keys against actual provider APIs
Fallback Handling: Uses known working models when API responses are incomplete
Retry Logic: Multiple attempts with exponential backoff for reliability

New Layout Hierarchy:

🔑 Quick Setup - API Key Tester (primary option)
🚀 Automatic Setup - Tetrate & OpenRouter (OAuth flows)
⚙️ Manual Setup - "Other providers" link
Removed Redundancy: Eliminated separate OpenAI/Anthropic cards (handled by key tester)

Type of Change

  • Feature
  • Bug fix
  • Refactor / Code quality
  • Performance improvement
  • Documentation
  • Tests
  • Security fix
  • Build / Release
  • Other (specify below)

Testing

Related Issues

Relates to #ISSUE_ID
Discussion: LINK (if any)

Screenshots/Demos (for UX changes)

Before:
Screenshot 2025-10-02 at 9 24 38 AM

After:

Screen.Recording.2025-10-02.at.11.46.53.AM.mov

Email:

- Added OpenAI and Anthropic provider options to onboarding
- Created official provider icons (Tetrate, OpenAI, Claude/Anthropic)
- Implemented 2x2 grid layout for better organization
- Updated header to be left-aligned with original messaging
- Enhanced icon spacing and positioning for better visual hierarchy
- Added setup handlers for manual provider configuration
- Fixed icon export compatibility issues
- Maintained OAuth flows for OpenRouter/Tetrate
- Added manual configuration flows for OpenAI/Anthropic

Key changes:
- ProviderGuard.tsx: Complete redesign with 4 providers
- New icon components: OpenAI.tsx, Anthropic.tsx, Tetrate.tsx
- New setup utilities: openaiSetup.ts, anthropicSetup.ts
- Updated icons/index.ts for proper exports
- Enhanced spacing (mb-3) for more visual 'soul'

All icons use official branding and consistent 16px sizing.
- Created ApiKeyTester component that automatically tests API keys against multiple providers
- Integrated with existing providers() and getProviderModels() API endpoints
- Tests common providers: OpenAI, Anthropic, Google, Groq, Cohere, Mistral
- Automatically configures first working provider with GOOSE_PROVIDER and GOOSE_MODEL
- Added real-time validation with loading states and clear success/error feedback
- Positioned above 'Other providers' section for easy discovery
- Includes toast notifications and visual test results
- Handles authentication failures gracefully with specific error messages

Key features:
- Smart detection across multiple provider APIs
- One-click testing with progress indicators
- Automatic configuration on success
- Clear visual feedback for each provider tested
- Seamless integration with existing onboarding flow
- Fixed API key configuration using proper upsertConfig structure with is_secret: true
- Added delay after setting config to ensure it's applied before testing models
- Improved error handling with specific error messages from API responses
- Enhanced provider display names for better user experience
- Added Enter key support for quick testing workflow
- Enhanced dark mode support for result display colors
- Better error extraction from API responses for debugging
- Uses direct upsertConfig API calls for immediate effect

This should fix the issue where valid Anthropic API keys were showing as failed.
The component now properly:
1. Sets the API key as a secret configuration value
2. Waits for the config to be applied
3. Tests the provider's model endpoint
4. Shows specific error messages for debugging
- Fix ConfigProvider context hierarchy issue by moving ConfigProvider to app root level
- Add intelligent API key tester component with provider auto-detection
- Implement smart key format detection (OpenAI, Anthropic, Google, Groq)
- Add fallback model handling for providers with API response issues
- Reposition API key tester as primary onboarding option above provider cards
- Remove redundant OpenAI and Anthropic provider cards (handled by key tester)
- Add custom Key icon component matching design system
- Streamline onboarding flow: Quick Setup → Automatic Setup → Manual Setup
- Improve error handling and user feedback with detailed logging
- Enhance retry logic and configuration timing for better reliability
- Move 'Recommended' pill from Tetrate to API Key Tester as primary option
- Update provider cards to use consistent bg-transparent styling
- Replace 'Test Key' button text with ArrowRight icon for cleaner UI
- Add custom ArrowRight icon component matching design system
- Improve button height matching with input field using items-stretch
- Use design system Button component instead of custom blue styling
- Maintain blue focus ring on input while using standard button colors
- Create better visual hierarchy: API Key Tester (primary) → Provider cards (secondary)
@spencrmartin
Copy link
Collaborator Author

@michaelneale re onboarding conversations currently ongoing - I had to have it fall back to test the provider elements to known working models (haiku for anthropic) which probs isnt correct but the gist is there.

- Button now turns blue when user has entered an API key
- Uses 'default' variant (blue) when input is present and not loading
- Uses 'secondary' variant (gray) when input is empty or disabled
- Provides clear visual feedback to indicate when action is available
- Maintains proper disabled state for empty input and loading states
@michaelneale
Copy link
Collaborator

hrm interesting ... not sure if that is where we want things to go (can also detect environment as well if we wanted to assume they had a key) - we really want to steer people to something which is more one click where possible?

@DOsinga
Copy link
Collaborator

DOsinga commented Oct 21, 2025

closing because stale

@DOsinga DOsinga closed this Oct 21, 2025
spencrmartin added a commit that referenced this pull request Oct 30, 2025
- Add auto_detect.rs module for parallel provider testing
- Add /config/detect-provider API endpoint
- Add DetectProviderRequest and DetectProviderResponse structs
- Update OpenAPI specifications
- Backend compiles successfully

Part of PR #4950 + #5147 integration
spencrmartin added a commit that referenced this pull request Oct 30, 2025
- Add ApiKeyTester component calling /config/detect-provider
- Update ProviderGuard with ApiKeyTester in onboarding
- Add provider icons and regenerate API client
- Show progress and results during detection

Integration of PR #4950 + #5147
@spencrmartin spencrmartin mentioned this pull request Oct 30, 2025
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants