feat: Hardware Key Agent - consolidate globally shared PIV service variables by Joerger · Pull Request #53974 · gravitational/teleport

Joerger · 2025-04-12T00:45:00Z

This PR consolidates the globally shared PIV service variables into a globally shared PIV service. This in turn ensures that the prompt is also shared, which is required for hardware key PIN caching.

The end goal is to remove these global variables altogether in favor of ensuring the hardware key service for a process is only initialized once. This will be handled in a series of follow up PRs broken up from #53707.

espadolini · 2025-04-14T11:40:06Z

+	if yubiKeyService != nil {
+		// If a prompt is provided, prioritize it over the existing prompt value.
+		if customPrompt != nil {
+			yubiKeyService.prompt = customPrompt
+		}
+		return yubiKeyService
+	}


Can we share the same set of yubikeys but have different prompts? Is that something that we ever need?

It's not something we need currently, or should ever need. Right now we are primarily handling cases like:

// pre-action checks in profile/cert store (view keys, expiration, // active profile, etc.), no prompt expected piv.NewYubiKeyService(nil) // defaults to CLIPrompt, but no prompt expected (no signatures) // client and client store initialization in `client.NewClient`. // This store is/should be used for all key interactions. piv.NewYubiKeyService(customPrompt) // overwrite with custom prompt e.g. for Connect // adhoc `keys.ParsePrivateKey` calls which fall outside of the // client store's path (no hwKeyService to pass). piv.NewYubiKeyService(nil) // continue using custom prompt or default cli prompt

Or for non tctl/tsh/connect clients, it's likely they will just call keys.ParsePrivateKey adhoc and we don't have any custom prompt or hwkey service initialization logic preceding it.

espadolini · 2025-04-14T11:41:43Z

+	s.yubiKeysMux.Lock()
+	defer s.yubiKeysMux.Unlock()


Is this mux protecting s.yubiKeys, or is it protecting some global state used by FindYubiKey? Can you run FindYubiKey in parallel in two different *YubiKeyServices? Can you run FindYubiKey in parallel for the same serial number?

Ah I see how it could look like that, but no the mutex is just protecting the yubikey map. FindYubiKey is independent.

FYI this is exercised in TestGetYubiKeyPrivateKey_Interactive, though CI doesn't run it.

rosstimothy · 2025-04-14T20:51:35Z

+// TODO(Joerger): Ensure all clients initialize [NewYubiKeyService] only once so we can
+// remove this global variable.


What is the plan for this TODO?

This PR would address tsh, tctl, and Teleport Connect. For arbitrary API clients or tbot we can do the same type of initialization, or continue to provide a global service through a separate method.

Joerger · 2025-04-15T17:18:22Z

@espadolini @rosstimothy Friendly ping to re-review

…into a shared YubiKeyService, which will also share the prompt.

…riables (#53974) * Consolidate process-wide shared prompt mutex and yubikey connections into a shared YubiKeyService, which will also share the prompt. * Fix interactive piv service tests. * Fix wording on comment.

* feat: Hardware Key Agent - Add `api/utils/keys/hardwarekey` package (#53671) * Move hardware key files into new hardwarekey package. * Tidy up PIVSlot logic and remove its piv-go dependency. * Add godoc comments to cliPrompt methods; Update cliprompt.go license year. * Add godocs. * feat: Hardware Key Agent - Add `hardwarekey.Service` interface with adapted PIV implementation (#53674) * Add hardware key service interface; Add mock service for tests. * Add piv implementation of hardware key service. * Remove old yubikey parsing logic in favor of hardwarekey.PrivateKeyRef. * Remove HardwareSigner in favor of hardwarekey.PrivateKey. * Add test; Cleanup comment; Don't require pin/touch prompts in tests with MockHarwdareKeyService by default. * Fix tests. * Add TODOs. * Use unhashed digest for WarmupHardwareKey. * Address comments. * Fix race condition in test service. * Address comments. * feat: Hardware Key Agent - Add `api/harwdarekey/piv` package (#53677) * Remove YubiKeyPrivateKey * Remove keys package dependencies from PIV implementation. * Move PIV implementation into new piv package. * Move newPrivateKey into YubiKeyService.NewPrivateKey. * * Clean up separation of concerns between YubiKey and YubiKeyService * Replace hardwarekey.PrivateKey cache with YubiKey cache for proper support for multiple clusters or yubikeys * Add YubiKey version field for use in version specific signature edge cases * Change sharedPIVConnection connection from an embedded field. * Address comments. * Add getKeyRef helper method. * feat: Hardware Key Agent - Enrich the PEM encoded hardware private key file (#53675) * Enrich hardware key PEM file. * Add test. * Validate PrivateKeyRef before/after encode/decode. * Update api/utils/keys/hardwarekey/hardwarekey.go Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * Validate key ref in hardware key service implementations; Fix merge conflict. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * feat: Hardware Key Agent - set hardware key service in client store (#53563) * Replace prompt in keystore with hardware key service in client store. * Add client StoreConfig and StoreConfigOpt. * Add client store and key store tests. * Provide nil service for ProfileStatus's AppsForCluster and DatabasesForCluster methods. * Address comments. * Fix test. * Fix test with cmp.Diff. * feat: Hardware Key Agent - Propagate contextual key info from key store to hardware key prompts (#53703) * Propagate contextual key info from key store through to hardware key prompts. * Remove HardwareKeyPromptConstructor. * Cleanup; add tests. * feat: Hardware Key Agent - consolidate globally shared PIV service variables (#53974) * Consolidate process-wide shared prompt mutex and yubikey connections into a shared YubiKeyService, which will also share the prompt. * Fix interactive piv service tests. * Fix wording on comment. * Move RemoveProfile and ListProfileNames into ProfileStore. (#53781) * feat: Hardware Key PIN caching (#53976) * Add PinCachingPrompt. * Add PINCacheTimeout to auth preference proto message. * Report PIVPINCacheTimeout through ping; Store PIVPINCacheTimeout in profile. * Set PIV pin cache timeout for tsh and teleterm. * Add SetPrompt and GetPrompt to hardware key service interface. * Cleanup; Add test. * Address comments. * Rename to PINCacheTTL. * Apply suggestions. * Fix lint. * Simplify randPIN for test. * Use math/rand/v2. * Update terraform docs. * Revert pin caching change for connect to fix race condition. (#54140) * feat: Hardware Key Agent (#54026) * Add HardwareKeyAgent proto service. * Add hardware key agent client and server implementation. * Add tsh piv agent command. * Use hardware key agent service when available. * * Add config option to Teleport connect to start the hardware key agent server * Add flag to tsh daemon command to start the hardware key agent server * Fix hardware key agent client connection for Windows. * Add hardware key agent service; Add tests. * Minor cleanup. * Move hardware key agent proto to /api. * * Restructure packages based on dependencies. * Add hardware key agent test coverage for RSA, ECDSA, and ED25519 keys. * Add PIV test coverage for RSA keys. * Fix lint. * Fix lint; Fix test. * Address comments. * Address comments. * Fix merge conflict. * feat: Hardware Key Agent - require users to configure certificate (#54118) * * Hardware key agent checks whether a key is configured for Teleport clients before performing a signature. * Add detailed error message and docs for configuring PIV slot certificate. * Add additional checks for mismatched public key on PIV slot, which can occur when generating a new key on a PIV slot with an active login session. * Update api/utils/keys/piv/yubikey.go Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Apply suggestions from code review Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> --------- Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> * Remove MaxUint32 call to fix builds on 32-bit systems. (#54125) * feat: Hardware Key Agent - command hint (#54090) * Supply command for context on hardware key prompt. * * Include command in Teleport Connect hardware key prompts, excluding tshd commands * Fix proxy host context passed to Teleport connect hardware key prompts * Only use direct service for `tsh login` to avoid jumping between clients * Add new line before command. * Fix story. * Address comments. * Trim forward slash for windows. * Change proxy_host to proxy_hostname; Update comment. * feat: Hardware Key Agent w/ PIN caching - fix cross-cluster support (#54144) * Style hardware key prompt with command in Connect (#54258) * Style hardware key prompt with command in Connect. * Move `CliCommand.tsx` to `components` * Extend stories with a command * Use `CliCommand` component, improve spacing * Wrap command in the dialog * End sentences with a dot * Add a gap between `CliCommand` and errors in `OnlineDocumentGateway` --------- Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> * feat: Hardware Key Agent - fix socket replacement on Windows (#54126) * Prevent windows from trying to reuse the addr and getting a bind error. * Replace unresponsive windows unix sockets. * Explicitly check for error message instead of os.Stat. * Move windowsBindErrMessage. * Add cross-platform ErrAddrInUse constant. * Move error constants to hardwarekey package. * feat: Hardware Key Agent - initialize hardware key service at start of `tsh daemon` (#54226) * * Initialize shared hardware key service and client store for tshd * Replace CustomHardwareKeyPrompt with SetPrompt * Add lazy loaded tshd event service client and use it to initialize hardware key prompt early. * Address comments; Set TshdEventsClient in daemon service. * Fix test. * Fix potential race condition on global yubikey service prompt. * Add test. * Require ClientStore in `client.Config` (#54227) * Invert config.EnableEscapeSequences into config.DisableEscapeSequences so that the default value (false) results in the desired default behavior. * * Replace MakeDefaultClientConfig with CheckAndSetDefaults * Require ClientStore to be provided in config * Remove CustomHardwareKeyPrompt from config * Set client store in tests that were missing it. * Ensure tsh only initializes client store once. * Client config uses its own client store. * Replace uses of KeysDir with ClientStore. * Remove unused home apth for vnet process. * Remove unnecessary profile re-load and helper function. * Replace sync.Once with atomic; add get/setClientStore. * Fix uncaught merge conflict. * Fix test; Add comment for why we initialize the client store atomically. * Return error when using MemKeyStore for tsh puttyconfig. * Fix merge conflict. * feat: PIV PIN Caching - add file config option (#54328) * Add pin_cache_ttl as file config option. * Add test. * Update lib/config/fileconf.go Co-authored-by: Bernard Kim <bernard@goteleport.com> --------- Co-authored-by: Bernard Kim <bernard@goteleport.com> * Restore namespace in client config. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> Co-authored-by: Bernard Kim <bernard@goteleport.com>

* feat: Hardware Key Agent - Add `api/utils/keys/hardwarekey` package (#53671) * Move hardware key files into new hardwarekey package. * Tidy up PIVSlot logic and remove its piv-go dependency. * Add godoc comments to cliPrompt methods; Update cliprompt.go license year. * Add godocs. * feat: Hardware Key Agent - Add `hardwarekey.Service` interface with adapted PIV implementation (#53674) * Add hardware key service interface; Add mock service for tests. * Add piv implementation of hardware key service. * Remove old yubikey parsing logic in favor of hardwarekey.PrivateKeyRef. * Remove HardwareSigner in favor of hardwarekey.PrivateKey. * Add test; Cleanup comment; Don't require pin/touch prompts in tests with MockHarwdareKeyService by default. * Fix tests. * Add TODOs. * Use unhashed digest for WarmupHardwareKey. * Address comments. * Fix race condition in test service. * Address comments. * feat: Hardware Key Agent - Add `api/harwdarekey/piv` package (#53677) * Remove YubiKeyPrivateKey * Remove keys package dependencies from PIV implementation. * Move PIV implementation into new piv package. * Move newPrivateKey into YubiKeyService.NewPrivateKey. * * Clean up separation of concerns between YubiKey and YubiKeyService * Replace hardwarekey.PrivateKey cache with YubiKey cache for proper support for multiple clusters or yubikeys * Add YubiKey version field for use in version specific signature edge cases * Change sharedPIVConnection connection from an embedded field. * Address comments. * Add getKeyRef helper method. * Rename NewSoftwarePrivateKey to NewPrivateKey (#53598) * feat: Hardware Key Agent - Enrich the PEM encoded hardware private key file (#53675) * Enrich hardware key PEM file. * Add test. * Validate PrivateKeyRef before/after encode/decode. * Update api/utils/keys/hardwarekey/hardwarekey.go Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * Validate key ref in hardware key service implementations; Fix merge conflict. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * feat: Hardware Key Agent - set hardware key service in client store (#53563) * Replace prompt in keystore with hardware key service in client store. * Add client StoreConfig and StoreConfigOpt. * Add client store and key store tests. * Provide nil service for ProfileStatus's AppsForCluster and DatabasesForCluster methods. * Address comments. * Fix test. * Fix test with cmp.Diff. * feat: Hardware Key Agent - Propagate contextual key info from key store to hardware key prompts (#53703) * Propagate contextual key info from key store through to hardware key prompts. * Remove HardwareKeyPromptConstructor. * Cleanup; add tests. * feat: Hardware Key Agent - consolidate globally shared PIV service variables (#53974) * Consolidate process-wide shared prompt mutex and yubikey connections into a shared YubiKeyService, which will also share the prompt. * Fix interactive piv service tests. * Fix wording on comment. * Move RemoveProfile and ListProfileNames into ProfileStore. (#53781) * feat: Hardware Key PIN caching (#53976) * Add PinCachingPrompt. * Add PINCacheTimeout to auth preference proto message. * Report PIVPINCacheTimeout through ping; Store PIVPINCacheTimeout in profile. * Set PIV pin cache timeout for tsh and teleterm. * Add SetPrompt and GetPrompt to hardware key service interface. * Cleanup; Add test. * Address comments. * Rename to PINCacheTTL. * Apply suggestions. * Fix lint. * Simplify randPIN for test. * Use math/rand/v2. * Update terraform docs. * Revert pin caching change for connect to fix race condition. (#54140) * feat: Hardware Key Agent (#54026) * Add HardwareKeyAgent proto service. * Add hardware key agent client and server implementation. * Add tsh piv agent command. * Use hardware key agent service when available. * * Add config option to Teleport connect to start the hardware key agent server * Add flag to tsh daemon command to start the hardware key agent server * Fix hardware key agent client connection for Windows. * Add hardware key agent service; Add tests. * Minor cleanup. * Move hardware key agent proto to /api. * * Restructure packages based on dependencies. * Add hardware key agent test coverage for RSA, ECDSA, and ED25519 keys. * Add PIV test coverage for RSA keys. * Fix lint. * Fix lint; Fix test. * Address comments. * Address comments. * Fix merge conflict. * feat: Hardware Key Agent - require users to configure certificate (#54118) * * Hardware key agent checks whether a key is configured for Teleport clients before performing a signature. * Add detailed error message and docs for configuring PIV slot certificate. * Add additional checks for mismatched public key on PIV slot, which can occur when generating a new key on a PIV slot with an active login session. * Update api/utils/keys/piv/yubikey.go Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Apply suggestions from code review Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> --------- Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> * Remove MaxUint32 call to fix builds on 32-bit systems. (#54125) * feat: Hardware Key Agent - command hint (#54090) * Supply command for context on hardware key prompt. * * Include command in Teleport Connect hardware key prompts, excluding tshd commands * Fix proxy host context passed to Teleport connect hardware key prompts * Only use direct service for `tsh login` to avoid jumping between clients * Add new line before command. * Fix story. * Address comments. * Trim forward slash for windows. * Change proxy_host to proxy_hostname; Update comment. * feat: Hardware Key Agent w/ PIN caching - fix cross-cluster support (#54144) * Style hardware key prompt with command in Connect (#54258) * Style hardware key prompt with command in Connect. * Move `CliCommand.tsx` to `components` * Extend stories with a command * Use `CliCommand` component, improve spacing * Wrap command in the dialog * End sentences with a dot * Add a gap between `CliCommand` and errors in `OnlineDocumentGateway` --------- Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> * feat: Hardware Key Agent - fix socket replacement on Windows (#54126) * Prevent windows from trying to reuse the addr and getting a bind error. * Replace unresponsive windows unix sockets. * Explicitly check for error message instead of os.Stat. * Move windowsBindErrMessage. * Add cross-platform ErrAddrInUse constant. * Move error constants to hardwarekey package. * feat: Hardware Key Agent - initialize hardware key service at start of `tsh daemon` (#54226) * * Initialize shared hardware key service and client store for tshd * Replace CustomHardwareKeyPrompt with SetPrompt * Add lazy loaded tshd event service client and use it to initialize hardware key prompt early. * Address comments; Set TshdEventsClient in daemon service. * Fix test. * Fix potential race condition on global yubikey service prompt. * Add test. * Require ClientStore in `client.Config` (#54227) * Invert config.EnableEscapeSequences into config.DisableEscapeSequences so that the default value (false) results in the desired default behavior. * * Replace MakeDefaultClientConfig with CheckAndSetDefaults * Require ClientStore to be provided in config * Remove CustomHardwareKeyPrompt from config * Set client store in tests that were missing it. * Ensure tsh only initializes client store once. * Client config uses its own client store. * Replace uses of KeysDir with ClientStore. * Remove unused home apth for vnet process. * Remove unnecessary profile re-load and helper function. * Replace sync.Once with atomic; add get/setClientStore. * Fix uncaught merge conflict. * Fix test; Add comment for why we initialize the client store atomically. * Return error when using MemKeyStore for tsh puttyconfig. * Fix merge conflict. * feat: PIV PIN Caching - add file config option (#54328) * Add pin_cache_ttl as file config option. * Add test. * Update lib/config/fileconf.go Co-authored-by: Bernard Kim <bernard@goteleport.com> --------- Co-authored-by: Bernard Kim <bernard@goteleport.com> * Restore namespace in client config. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> Co-authored-by: Bernard Kim <bernard@goteleport.com>

github-actions Bot added the size/sm label Apr 12, 2025

github-actions Bot requested review from atburke and espadolini April 12, 2025 00:45

Joerger added the no-changelog Indicates that a PR does not require a changelog entry label Apr 12, 2025

Joerger mentioned this pull request Apr 12, 2025

feat: Hardware Key PIN caching #53976

Merged

espadolini reviewed Apr 14, 2025

View reviewed changes

Joerger mentioned this pull request Apr 14, 2025

Hardware Key Agent w/ PIN caching tracker #53704

Closed

Joerger requested a review from espadolini April 14, 2025 17:31

rosstimothy reviewed Apr 14, 2025

View reviewed changes

Joerger requested a review from rosstimothy April 14, 2025 21:43

Joerger mentioned this pull request Apr 15, 2025

feat: Hardware Key Agent #54026

Merged

Joerger requested a review from zmb3 April 15, 2025 17:18

Joerger added 3 commits April 15, 2025 10:37

Consolidate process-wide shared prompt mutex and yubikey connections …

84ff27f

…into a shared YubiKeyService, which will also share the prompt.

Fix interactive piv service tests.

77c414a

Fix wording on comment.

0ed5a16

Joerger force-pushed the joerger/shared-piv-service branch from 54a4bd0 to 0ed5a16 Compare April 15, 2025 17:37

rosstimothy approved these changes Apr 15, 2025

View reviewed changes

espadolini approved these changes Apr 15, 2025

View reviewed changes

public-teleport-github-review-bot Bot removed request for atburke and zmb3 April 15, 2025 22:00

Joerger added this pull request to the merge queue Apr 15, 2025

Merged via the queue into master with commit c8f7d97 Apr 15, 2025
41 checks passed

Joerger deleted the joerger/shared-piv-service branch April 15, 2025 23:45

Joerger mentioned this pull request Apr 23, 2025

Require ClientStore in client.Config #54227

Merged

Joerger mentioned this pull request Apr 25, 2025

[v17] Hardware Key Agent w/ PIN caching #54297

Merged

Joerger mentioned this pull request Apr 25, 2025

[v16] Hardware Key Agent w/ PIN caching #54298

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Hardware Key Agent - consolidate globally shared PIV service variables#53974

feat: Hardware Key Agent - consolidate globally shared PIV service variables#53974
Joerger merged 3 commits intomasterfrom
joerger/shared-piv-service

Joerger commented Apr 12, 2025

Uh oh!

espadolini Apr 14, 2025

Uh oh!

Joerger Apr 14, 2025 •

edited

Loading

Uh oh!

espadolini Apr 14, 2025

Uh oh!

Joerger Apr 14, 2025

Uh oh!

Joerger Apr 14, 2025

Uh oh!

Uh oh!

rosstimothy Apr 14, 2025

Uh oh!

Joerger Apr 14, 2025

Uh oh!

Joerger commented Apr 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		// TODO(Joerger): Ensure all clients initialize [NewYubiKeyService] only once so we can
		// remove this global variable.

Conversation

Joerger commented Apr 12, 2025

Uh oh!

espadolini Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

espadolini Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rosstimothy Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger commented Apr 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Joerger Apr 14, 2025 •

edited

Loading