feat: Hardware Key Agent by Joerger · Pull Request #54026 · gravitational/teleport

Joerger · 2025-04-15T03:32:57Z

Implements the Hardware Key agent for Teleport Connect and tsh piv agent (hidden command).

By default, tsh and tctl will look for an open agent to connect to route PIV connections / prompts. If no agent is found, these clients will connect to the direct PIV service as normal. If the agent server closes in the middle of a client call, the client will fallback to the direct service after a signature failure.

Depends on #53974

rosstimothy · 2025-04-16T19:34:31Z

+	// Check for an existing hardware key agent server and destroy it if it's unresponsive.
+	if err := checkAndDestroyExistingServer(ctx, keyAgentDir); err != nil {
+		return nil, trace.Wrap(err)
+	}
+
+	if err := os.MkdirAll(keyAgentDir, 0o700); err != nil {
+		return nil, trace.Wrap(err)
+	}


Should this be attempted a few times? What happens if the existing directory is recreated between checkAndDestroyExistingServer and os.MkdirAll?

Yeah, I tried to simplify things but it made it more racy. I'll revert to attempting to listening on the socket and handling syscall.EADDRINUSE as a pseudo-lock.

rosstimothy · 2025-04-16T21:00:07Z

+		return nil, trace.Wrap(err)
+	}
+
+	return newAgentListener(ctx, keyAgentDir)


Should we enforce a maximum number of attempts to prevent infinite recursion?

I'll just remove the recursion

We should still try a few times though right?

I don't think it needs to retry multiple times, just once with the potential to shut down an unresponsive agent. If the user is racing two different hardware key agents and that results in failing to open the listener for one, it'll just return the syscall.EADDRINUSE error on first retry. This isn't really an edge case that a user should ever run into.

zmb3

Just a few stylistic suggestions.

zmb3 · 2025-04-16T21:02:28Z

+// Signature is a private key signature.
+message Signature {
+  // For an RSA key, signature should be either a PKCS #1 v1.5 or PSS signature,
+  // depending on the hash and salt chosen. For an (EC)DSA key, it should be a


Would there be any benefit in also including the type of key in the message so that it's easier for the client to know how to parse it?

Not with go's crypto library in mind - the two types of signatures are handled interchangeably and handled by the caller. This comment is actually paraphrased from the crypto.Signer interface:

// Sign signs digest with the private key, possibly using entropy from // rand. For an RSA key, the resulting signature should be either a // PKCS #1 v1.5 or PSS signature (as indicated by opts). For an (EC)DSA // key, it should be a DER-serialised, ASN.1 signature structure. // ... Sign(rand io.Reader, digest []byte, opts SignerOpts) (signature []byte, err error)

* Add HardwareKeyAgent proto service. * Add hardware key agent client and server implementation. * Add tsh piv agent command. * Use hardware key agent service when available. * * Add config option to Teleport connect to start the hardware key agent server * Add flag to tsh daemon command to start the hardware key agent server * Fix hardware key agent client connection for Windows. * Add hardware key agent service; Add tests. * Minor cleanup. * Move hardware key agent proto to /api. * * Restructure packages based on dependencies. * Add hardware key agent test coverage for RSA, ECDSA, and ED25519 keys. * Add PIV test coverage for RSA keys. * Fix lint. * Fix lint; Fix test. * Address comments. * Address comments. * Fix merge conflict.

* feat: Hardware Key Agent - Add `api/utils/keys/hardwarekey` package (#53671) * Move hardware key files into new hardwarekey package. * Tidy up PIVSlot logic and remove its piv-go dependency. * Add godoc comments to cliPrompt methods; Update cliprompt.go license year. * Add godocs. * feat: Hardware Key Agent - Add `hardwarekey.Service` interface with adapted PIV implementation (#53674) * Add hardware key service interface; Add mock service for tests. * Add piv implementation of hardware key service. * Remove old yubikey parsing logic in favor of hardwarekey.PrivateKeyRef. * Remove HardwareSigner in favor of hardwarekey.PrivateKey. * Add test; Cleanup comment; Don't require pin/touch prompts in tests with MockHarwdareKeyService by default. * Fix tests. * Add TODOs. * Use unhashed digest for WarmupHardwareKey. * Address comments. * Fix race condition in test service. * Address comments. * feat: Hardware Key Agent - Add `api/harwdarekey/piv` package (#53677) * Remove YubiKeyPrivateKey * Remove keys package dependencies from PIV implementation. * Move PIV implementation into new piv package. * Move newPrivateKey into YubiKeyService.NewPrivateKey. * * Clean up separation of concerns between YubiKey and YubiKeyService * Replace hardwarekey.PrivateKey cache with YubiKey cache for proper support for multiple clusters or yubikeys * Add YubiKey version field for use in version specific signature edge cases * Change sharedPIVConnection connection from an embedded field. * Address comments. * Add getKeyRef helper method. * feat: Hardware Key Agent - Enrich the PEM encoded hardware private key file (#53675) * Enrich hardware key PEM file. * Add test. * Validate PrivateKeyRef before/after encode/decode. * Update api/utils/keys/hardwarekey/hardwarekey.go Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * Validate key ref in hardware key service implementations; Fix merge conflict. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * feat: Hardware Key Agent - set hardware key service in client store (#53563) * Replace prompt in keystore with hardware key service in client store. * Add client StoreConfig and StoreConfigOpt. * Add client store and key store tests. * Provide nil service for ProfileStatus's AppsForCluster and DatabasesForCluster methods. * Address comments. * Fix test. * Fix test with cmp.Diff. * feat: Hardware Key Agent - Propagate contextual key info from key store to hardware key prompts (#53703) * Propagate contextual key info from key store through to hardware key prompts. * Remove HardwareKeyPromptConstructor. * Cleanup; add tests. * feat: Hardware Key Agent - consolidate globally shared PIV service variables (#53974) * Consolidate process-wide shared prompt mutex and yubikey connections into a shared YubiKeyService, which will also share the prompt. * Fix interactive piv service tests. * Fix wording on comment. * Move RemoveProfile and ListProfileNames into ProfileStore. (#53781) * feat: Hardware Key PIN caching (#53976) * Add PinCachingPrompt. * Add PINCacheTimeout to auth preference proto message. * Report PIVPINCacheTimeout through ping; Store PIVPINCacheTimeout in profile. * Set PIV pin cache timeout for tsh and teleterm. * Add SetPrompt and GetPrompt to hardware key service interface. * Cleanup; Add test. * Address comments. * Rename to PINCacheTTL. * Apply suggestions. * Fix lint. * Simplify randPIN for test. * Use math/rand/v2. * Update terraform docs. * Revert pin caching change for connect to fix race condition. (#54140) * feat: Hardware Key Agent (#54026) * Add HardwareKeyAgent proto service. * Add hardware key agent client and server implementation. * Add tsh piv agent command. * Use hardware key agent service when available. * * Add config option to Teleport connect to start the hardware key agent server * Add flag to tsh daemon command to start the hardware key agent server * Fix hardware key agent client connection for Windows. * Add hardware key agent service; Add tests. * Minor cleanup. * Move hardware key agent proto to /api. * * Restructure packages based on dependencies. * Add hardware key agent test coverage for RSA, ECDSA, and ED25519 keys. * Add PIV test coverage for RSA keys. * Fix lint. * Fix lint; Fix test. * Address comments. * Address comments. * Fix merge conflict. * feat: Hardware Key Agent - require users to configure certificate (#54118) * * Hardware key agent checks whether a key is configured for Teleport clients before performing a signature. * Add detailed error message and docs for configuring PIV slot certificate. * Add additional checks for mismatched public key on PIV slot, which can occur when generating a new key on a PIV slot with an active login session. * Update api/utils/keys/piv/yubikey.go Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Apply suggestions from code review Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> --------- Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> * Remove MaxUint32 call to fix builds on 32-bit systems. (#54125) * feat: Hardware Key Agent - command hint (#54090) * Supply command for context on hardware key prompt. * * Include command in Teleport Connect hardware key prompts, excluding tshd commands * Fix proxy host context passed to Teleport connect hardware key prompts * Only use direct service for `tsh login` to avoid jumping between clients * Add new line before command. * Fix story. * Address comments. * Trim forward slash for windows. * Change proxy_host to proxy_hostname; Update comment. * feat: Hardware Key Agent w/ PIN caching - fix cross-cluster support (#54144) * Style hardware key prompt with command in Connect (#54258) * Style hardware key prompt with command in Connect. * Move `CliCommand.tsx` to `components` * Extend stories with a command * Use `CliCommand` component, improve spacing * Wrap command in the dialog * End sentences with a dot * Add a gap between `CliCommand` and errors in `OnlineDocumentGateway` --------- Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> * feat: Hardware Key Agent - fix socket replacement on Windows (#54126) * Prevent windows from trying to reuse the addr and getting a bind error. * Replace unresponsive windows unix sockets. * Explicitly check for error message instead of os.Stat. * Move windowsBindErrMessage. * Add cross-platform ErrAddrInUse constant. * Move error constants to hardwarekey package. * feat: Hardware Key Agent - initialize hardware key service at start of `tsh daemon` (#54226) * * Initialize shared hardware key service and client store for tshd * Replace CustomHardwareKeyPrompt with SetPrompt * Add lazy loaded tshd event service client and use it to initialize hardware key prompt early. * Address comments; Set TshdEventsClient in daemon service. * Fix test. * Fix potential race condition on global yubikey service prompt. * Add test. * Require ClientStore in `client.Config` (#54227) * Invert config.EnableEscapeSequences into config.DisableEscapeSequences so that the default value (false) results in the desired default behavior. * * Replace MakeDefaultClientConfig with CheckAndSetDefaults * Require ClientStore to be provided in config * Remove CustomHardwareKeyPrompt from config * Set client store in tests that were missing it. * Ensure tsh only initializes client store once. * Client config uses its own client store. * Replace uses of KeysDir with ClientStore. * Remove unused home apth for vnet process. * Remove unnecessary profile re-load and helper function. * Replace sync.Once with atomic; add get/setClientStore. * Fix uncaught merge conflict. * Fix test; Add comment for why we initialize the client store atomically. * Return error when using MemKeyStore for tsh puttyconfig. * Fix merge conflict. * feat: PIV PIN Caching - add file config option (#54328) * Add pin_cache_ttl as file config option. * Add test. * Update lib/config/fileconf.go Co-authored-by: Bernard Kim <bernard@goteleport.com> --------- Co-authored-by: Bernard Kim <bernard@goteleport.com> * Restore namespace in client config. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> Co-authored-by: Bernard Kim <bernard@goteleport.com>

* feat: Hardware Key Agent - Add `api/utils/keys/hardwarekey` package (#53671) * Move hardware key files into new hardwarekey package. * Tidy up PIVSlot logic and remove its piv-go dependency. * Add godoc comments to cliPrompt methods; Update cliprompt.go license year. * Add godocs. * feat: Hardware Key Agent - Add `hardwarekey.Service` interface with adapted PIV implementation (#53674) * Add hardware key service interface; Add mock service for tests. * Add piv implementation of hardware key service. * Remove old yubikey parsing logic in favor of hardwarekey.PrivateKeyRef. * Remove HardwareSigner in favor of hardwarekey.PrivateKey. * Add test; Cleanup comment; Don't require pin/touch prompts in tests with MockHarwdareKeyService by default. * Fix tests. * Add TODOs. * Use unhashed digest for WarmupHardwareKey. * Address comments. * Fix race condition in test service. * Address comments. * feat: Hardware Key Agent - Add `api/harwdarekey/piv` package (#53677) * Remove YubiKeyPrivateKey * Remove keys package dependencies from PIV implementation. * Move PIV implementation into new piv package. * Move newPrivateKey into YubiKeyService.NewPrivateKey. * * Clean up separation of concerns between YubiKey and YubiKeyService * Replace hardwarekey.PrivateKey cache with YubiKey cache for proper support for multiple clusters or yubikeys * Add YubiKey version field for use in version specific signature edge cases * Change sharedPIVConnection connection from an embedded field. * Address comments. * Add getKeyRef helper method. * Rename NewSoftwarePrivateKey to NewPrivateKey (#53598) * feat: Hardware Key Agent - Enrich the PEM encoded hardware private key file (#53675) * Enrich hardware key PEM file. * Add test. * Validate PrivateKeyRef before/after encode/decode. * Update api/utils/keys/hardwarekey/hardwarekey.go Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * Validate key ref in hardware key service implementations; Fix merge conflict. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> * feat: Hardware Key Agent - set hardware key service in client store (#53563) * Replace prompt in keystore with hardware key service in client store. * Add client StoreConfig and StoreConfigOpt. * Add client store and key store tests. * Provide nil service for ProfileStatus's AppsForCluster and DatabasesForCluster methods. * Address comments. * Fix test. * Fix test with cmp.Diff. * feat: Hardware Key Agent - Propagate contextual key info from key store to hardware key prompts (#53703) * Propagate contextual key info from key store through to hardware key prompts. * Remove HardwareKeyPromptConstructor. * Cleanup; add tests. * feat: Hardware Key Agent - consolidate globally shared PIV service variables (#53974) * Consolidate process-wide shared prompt mutex and yubikey connections into a shared YubiKeyService, which will also share the prompt. * Fix interactive piv service tests. * Fix wording on comment. * Move RemoveProfile and ListProfileNames into ProfileStore. (#53781) * feat: Hardware Key PIN caching (#53976) * Add PinCachingPrompt. * Add PINCacheTimeout to auth preference proto message. * Report PIVPINCacheTimeout through ping; Store PIVPINCacheTimeout in profile. * Set PIV pin cache timeout for tsh and teleterm. * Add SetPrompt and GetPrompt to hardware key service interface. * Cleanup; Add test. * Address comments. * Rename to PINCacheTTL. * Apply suggestions. * Fix lint. * Simplify randPIN for test. * Use math/rand/v2. * Update terraform docs. * Revert pin caching change for connect to fix race condition. (#54140) * feat: Hardware Key Agent (#54026) * Add HardwareKeyAgent proto service. * Add hardware key agent client and server implementation. * Add tsh piv agent command. * Use hardware key agent service when available. * * Add config option to Teleport connect to start the hardware key agent server * Add flag to tsh daemon command to start the hardware key agent server * Fix hardware key agent client connection for Windows. * Add hardware key agent service; Add tests. * Minor cleanup. * Move hardware key agent proto to /api. * * Restructure packages based on dependencies. * Add hardware key agent test coverage for RSA, ECDSA, and ED25519 keys. * Add PIV test coverage for RSA keys. * Fix lint. * Fix lint; Fix test. * Address comments. * Address comments. * Fix merge conflict. * feat: Hardware Key Agent - require users to configure certificate (#54118) * * Hardware key agent checks whether a key is configured for Teleport clients before performing a signature. * Add detailed error message and docs for configuring PIV slot certificate. * Add additional checks for mismatched public key on PIV slot, which can occur when generating a new key on a PIV slot with an active login session. * Update api/utils/keys/piv/yubikey.go Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> * Apply suggestions from code review Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> --------- Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> * Remove MaxUint32 call to fix builds on 32-bit systems. (#54125) * feat: Hardware Key Agent - command hint (#54090) * Supply command for context on hardware key prompt. * * Include command in Teleport Connect hardware key prompts, excluding tshd commands * Fix proxy host context passed to Teleport connect hardware key prompts * Only use direct service for `tsh login` to avoid jumping between clients * Add new line before command. * Fix story. * Address comments. * Trim forward slash for windows. * Change proxy_host to proxy_hostname; Update comment. * feat: Hardware Key Agent w/ PIN caching - fix cross-cluster support (#54144) * Style hardware key prompt with command in Connect (#54258) * Style hardware key prompt with command in Connect. * Move `CliCommand.tsx` to `components` * Extend stories with a command * Use `CliCommand` component, improve spacing * Wrap command in the dialog * End sentences with a dot * Add a gap between `CliCommand` and errors in `OnlineDocumentGateway` --------- Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> * feat: Hardware Key Agent - fix socket replacement on Windows (#54126) * Prevent windows from trying to reuse the addr and getting a bind error. * Replace unresponsive windows unix sockets. * Explicitly check for error message instead of os.Stat. * Move windowsBindErrMessage. * Add cross-platform ErrAddrInUse constant. * Move error constants to hardwarekey package. * feat: Hardware Key Agent - initialize hardware key service at start of `tsh daemon` (#54226) * * Initialize shared hardware key service and client store for tshd * Replace CustomHardwareKeyPrompt with SetPrompt * Add lazy loaded tshd event service client and use it to initialize hardware key prompt early. * Address comments; Set TshdEventsClient in daemon service. * Fix test. * Fix potential race condition on global yubikey service prompt. * Add test. * Require ClientStore in `client.Config` (#54227) * Invert config.EnableEscapeSequences into config.DisableEscapeSequences so that the default value (false) results in the desired default behavior. * * Replace MakeDefaultClientConfig with CheckAndSetDefaults * Require ClientStore to be provided in config * Remove CustomHardwareKeyPrompt from config * Set client store in tests that were missing it. * Ensure tsh only initializes client store once. * Client config uses its own client store. * Replace uses of KeysDir with ClientStore. * Remove unused home apth for vnet process. * Remove unnecessary profile re-load and helper function. * Replace sync.Once with atomic; add get/setClientStore. * Fix uncaught merge conflict. * Fix test; Add comment for why we initialize the client store atomically. * Return error when using MemKeyStore for tsh puttyconfig. * Fix merge conflict. * feat: PIV PIN Caching - add file config option (#54328) * Add pin_cache_ttl as file config option. * Add test. * Update lib/config/fileconf.go Co-authored-by: Bernard Kim <bernard@goteleport.com> --------- Co-authored-by: Bernard Kim <bernard@goteleport.com> * Restore namespace in client config. --------- Co-authored-by: STeve (Xin) Huang <xin.huang@goteleport.com> Co-authored-by: Zac Bergquist <zac.bergquist@goteleport.com> Co-authored-by: Paul Gottschling <paul.gottschling@goteleport.com> Co-authored-by: Grzegorz Zdunek <grzegorz.zdunek@goteleport.com> Co-authored-by: Bernard Kim <bernard@goteleport.com>

Joerger force-pushed the joerger/shared-piv-service branch from 54a4bd0 to 0ed5a16 Compare April 15, 2025 17:37

Joerger force-pushed the joerger/piv-key-agent-service branch from eed0bc2 to 02a9d0f Compare April 15, 2025 17:56

Joerger marked this pull request as ready for review April 15, 2025 17:56

Joerger requested review from rosstimothy and zmb3 April 15, 2025 17:56

github-actions Bot added size/lg tctl tctl - Teleport admin tool tsh tsh - Teleport's command line tool for logging into nodes running Teleport. ui labels Apr 15, 2025

github-actions Bot requested review from flyinghermit and ryanclark April 15, 2025 17:57

Joerger mentioned this pull request Apr 15, 2025

Hardware Key Agent w/ PIN caching tracker #53704

Closed

Joerger marked this pull request as draft April 15, 2025 19:15

Joerger removed request for flyinghermit and ryanclark April 15, 2025 19:15

Base automatically changed from joerger/shared-piv-service to master April 15, 2025 23:45

Joerger force-pushed the joerger/piv-key-agent-service branch from fddaf65 to 71ba814 Compare April 16, 2025 02:19

Joerger marked this pull request as ready for review April 16, 2025 02:20

github-actions Bot requested review from kimlisa and kiosion April 16, 2025 02:21

Joerger force-pushed the joerger/piv-key-agent-service branch from 71ba814 to d5b381a Compare April 16, 2025 03:36

Joerger added the no-changelog Indicates that a PR does not require a changelog entry label Apr 16, 2025

rosstimothy reviewed Apr 16, 2025

View reviewed changes

Joerger requested a review from rosstimothy April 16, 2025 20:30

Joerger mentioned this pull request Apr 16, 2025

feat: Hardware Key Agent - command hint #54090

Merged

rosstimothy reviewed Apr 16, 2025

View reviewed changes

zmb3 approved these changes Apr 16, 2025

View reviewed changes

Joerger requested a review from rosstimothy April 16, 2025 21:36

rosstimothy approved these changes Apr 16, 2025

View reviewed changes

public-teleport-github-review-bot Bot removed the request for review from kimlisa April 16, 2025 21:42

Joerger added 2 commits April 16, 2025 16:24

Address comments.

50127d5

Address comments.

c87ef8f

Joerger force-pushed the joerger/piv-key-agent-service branch from 1df2acb to c87ef8f Compare April 16, 2025 23:24

Fix merge conflict.

befa615

Joerger enabled auto-merge April 16, 2025 23:57

Joerger added this pull request to the merge queue Apr 17, 2025

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 17, 2025

Joerger added this pull request to the merge queue Apr 17, 2025

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 17, 2025

Joerger added this pull request to the merge queue Apr 17, 2025

Merged via the queue into master with commit 18cc4a1 Apr 17, 2025
45 checks passed

Joerger deleted the joerger/piv-key-agent-service branch April 17, 2025 01:35

Joerger mentioned this pull request Apr 25, 2025

[v17] Hardware Key Agent w/ PIN caching #54297

Merged

Joerger mentioned this pull request Apr 25, 2025

[v16] Hardware Key Agent w/ PIN caching #54298

Merged

Conversation

Joerger commented Apr 15, 2025

Uh oh!

Uh oh!

Uh oh!

rosstimothy Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rosstimothy Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

rosstimothy Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zmb3 left a comment

Choose a reason for hiding this comment

Uh oh!

zmb3 Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

Joerger Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Joerger Apr 16, 2025 •

edited

Loading