Skip to content

Store access token missing#15089

Merged
ph merged 3 commits intoelastic:fleetfrom
ph:store-access-token-missing
Dec 16, 2019
Merged

Store access token missing#15089
ph merged 3 commits intoelastic:fleetfrom
ph:store-access-token-missing

Conversation

@ph
Copy link
Copy Markdown
Contributor

@ph ph commented Dec 12, 2019

This PR add a few things

- It takes the keystore encryption implementation and create an
  io.Reader and io.Writer for it, we currently work with variable lenght
  blocks and we are not compatible for now with the one in libbeat.

- It implements a few storage mechanism and wrapper:
  - DiskStore: Save a io.reader directly to disk, the content is saved
    in a temporary file and the target is replace in an atomic
    operation.
  - EncryptedDiskStore: Same as Keystore but use the encrypted io.Reader
    and io.Writer.

- A fleet configuration is created in _meta/agent.fleet.yml, the content
  of this file is packed in the binary. When a user enroll the agent,
  we do a backup of the current agent.yml and the content is replaced
  with the agent.fleet.yml

- The enrollment information is saved into a "fleet.yml"

- The managed mode now read the content of the fleet.yml and creates a
  Kibana API client.

- A Separates Fleet Config struct is created.

- Enroll will now ask for confirmation before replacing the user
  configuration.

ref: #14951

ph added 2 commits December 12, 2019 14:09
    This PR add a few things

    - It takes the keystore encryption implementation and create an
      io.Reader and io.Writer for it, we currently work with variable lenght
      blocks and we are not compatible for now with the one in libbeat.

    - It implements a few storage mechanism and wrapper:
      - DiskStore: Save a io.reader directly to disk, the content is saved
        in a temporary file and the target is replace in an atomic
        operation.
      - EncryptedDiskStore: Same as Keystore but use the encrypted io.Reader
        and io.Writer.

    - A fleet configuration is created in _meta/agent.fleet.yml, the content
      of this file is packed in the binary. When a user enroll the agent,
      we do a backup of the current agent.yml and the content is replaced
      with the agent.fleet.yml

    - The enrollment information is saved into a "fleet.yml"

    - The managed mode now read the content of the fleet.yml and creates a
      Kibana API client.

    - A Separates Fleet Config struct is created.

    - Enroll will now ask for confirmation before replacing the user
      configuration.
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/ingest (Project:fleet)

@ph
Copy link
Copy Markdown
Contributor Author

ph commented Dec 13, 2019

@michalpristas

Have you seen this error before CI?

-- FAIL: TestShortRun (0.00s)

    operator_test.go:36: operation-start: chown /home/travis/gopath/src/github.com/elastic/beats/x-pack/agent/pkg/agent/operation/tests/scripts/short--1.0.yml: operation not permitted

=== RUN   TestShortRunInvalid

2019-12-12T20:55:33Z INFO	operation_fetch.go:80	operation 'operation-fetch' downloaded bumblebee. into 

2019-12-12T20:55:33Z INFO	operator.go:199	operation 'operation-verify' skipped for bumblebee.

2019-12-12T20:55:33Z INFO	process.go:72	address assigned to the process '/bin/bumblebee/bumblebee': '127.0.0.1:10000'

--- PASS: TestShortRunInvalid (1.00s)

=== RUN   TestLongRunWithStop

2019-12-12T20:55:34Z INFO	operation_fetch.go:80	operation 'operation-fetch' downloaded long.1.0 into 

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-verify' skipped for long.1.0

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-install' skipped for long.1.0

2019-12-12T20:55:34Z INFO	process.go:72	address assigned to the process '/bin/sh': '127.0.0.1:10000'

--- FAIL: TestLongRunWithStop (0.00s)

    operator_test.go:71: operation-start: failed to start '/bin/sh': fork/exec /bin/sh: operation not permitted

=== RUN   TestLongRunWithCrash

2019-12-12T20:55:34Z INFO	operation_fetch.go:80	operation 'operation-fetch' downloaded long.1.0 into 

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-verify' skipped for long.1.0

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-install' skipped for long.1.0

2019-12-12T20:55:34Z INFO	process.go:72	address assigned to the process '/bin/sh': '127.0.0.1:10000'

--- FAIL: TestLongRunWithCrash (0.01s)

    operator_test.go:112: operation-start: failed to start '/bin/sh': fork/exec /bin/sh: operation not permitted

=== RUN   TestTwoProcesses

2019-12-12T20:55:34Z INFO	operation_fetch.go:80	operation 'operation-fetch' downloaded long.1.0 into 

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-verify' skipped for long.1.0

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-install' skipped for long.1.0

2019-12-12T20:55:34Z INFO	process.go:72	address assigned to the process '/bin/sh': '127.0.0.1:10000'

--- FAIL: TestTwoProcesses (0.00s)

    operator_test.go:160: operation-start: failed to start '/bin/sh': fork/exec /bin/sh: operation not permitted

=== RUN   TestConfigurableRun

2019-12-12T20:55:34Z INFO	operation_fetch.go:80	operation 'operation-fetch' downloaded configurable.1.0 into 

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-verify' skipped for configurable.1.0

2019-12-12T20:55:34Z INFO	operator.go:199	operation 'operation-install' skipped for configurable.1.0

2019-12-12T20:55:36Z INFO	process.go:72	address assigned to the process '/home/travis/gopath/src/github.com/elastic/beats/x-pack/agent/pkg/agent/operation/tests/scripts/configurable-1.0-darwin-x86/configurable': '127.0.0.1:10000'

--- FAIL: TestConfigurableRun (2.51s)

    operator_test.go:209: found file /home/travis/gopath/src/github.com/elastic/beats/x-pack/agent/pkg/agent/operation/tests/scripts/configurable-1.0-darwin-x86/configurable

    operator_test.go:214: operation-start: failed to start '/home/travis/gopath/src/github.com/elastic/beats/x-pack/agent/pkg/agent/operation/tests/scripts/configurable-1.0-darwin-x86/configurable': fork/exec /home/travis/gopath/src/github.com/elastic/beats/x-pack/agent/pkg/agent/operation/tests/scripts/configurable-1.0-darwin-x86/configurable: operation not permitted

=== RUN   TestConfigurableByFileRun

2019-12-12T20:55:36Z INFO	operation_fetch.go:80	operation 'operation-fetch' downloaded configurablebyfile.1.0 into 

2019-12-12T20:55:36Z INFO	operator.go:199	operation 'operation-verify' skipped for configurablebyfile.1.0

2019-12-12T20:55:36Z INFO	operator.go:199	operation 'operation-install' skipped for configurablebyfile.1.0

--- FAIL: TestConfigurableByFileRun (0.00s)

    operator_test.go:294: found file /home/travis/gopath/src/github.com/elastic/beats/x-pack/agent/pkg/agent/operation/tests/scripts/configurablebyfile-1.0-darwin-x86/configurablebyfile

    operator_test.go:299: operation-start: chown /home/travis/gopath/src/github.com/elastic/beats/x-pack/agent/pkg/agent/operation/tests/scripts/configurablebyfile--1.0.yml: operation not permitted

@michalpristas
Copy link
Copy Markdown
Contributor

michalpristas commented Dec 16, 2019

seems like a permission problem. I've seen this happening in an incorrectly setup VM, root privileges are missing so you can chown or spawn a new process bin/sh

Copy link
Copy Markdown
Contributor

@michalpristas michalpristas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nits. this includes this missing PR right?

Comment thread x-pack/agent/dev-tools/cmd/buildfleetcfg/buildfleetcfg.go
Comment thread x-pack/agent/dev-tools/cmd/buildfleetcfg/buildfleetcfg.go Outdated
Comment thread x-pack/agent/dev-tools/cmd/buildfleetcfg/buildfleetcfg.go Outdated
Comment thread x-pack/agent/dev-tools/cmd/buildfleetcfg/buildfleetcfg.go Outdated
Comment thread x-pack/agent/pkg/core/plugin/app/process_cred.go Outdated
Comment thread x-pack/agent/pkg/core/plugin/app/process_cred_other.go Outdated
@ph
Copy link
Copy Markdown
Contributor Author

ph commented Dec 16, 2019

Weird, I can reproduce locally it in a freshly reset environment

@michalpristas
Copy link
Copy Markdown
Contributor

check if you are the owner of a directory

@michalpristas
Copy link
Copy Markdown
Contributor

michalpristas commented Dec 16, 2019

@ph if you fix // +build directives as suggested build will pass

@ph
Copy link
Copy Markdown
Contributor Author

ph commented Dec 16, 2019

@michalpristas Just pushed the commits for the previous comments, let's wait for the CI. 🤞

@ph ph merged commit 9394639 into elastic:fleet Dec 16, 2019
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
*  Feature: Allow to persist AccessToken and other connection data on disk
    This PR add a few things

    - It takes the keystore encryption implementation and create an
      io.Reader and io.Writer for it, we currently work with variable lenght
      blocks and we are not compatible for now with the one in libbeat.

    - It implements a few storage mechanism and wrapper:
      - DiskStore: Save a io.reader directly to disk, the content is saved
        in a temporary file and the target is replace in an atomic
        operation.
      - EncryptedDiskStore: Same as Keystore but use the encrypted io.Reader
        and io.Writer.

    - A fleet configuration is created in _meta/agent.fleet.yml, the content
      of this file is packed in the binary. When a user enroll the agent,
      we do a backup of the current agent.yml and the content is replaced
      with the agent.fleet.yml

    - The enrollment information is saved into a "fleet.yml"

    - The managed mode now read the content of the fleet.yml and creates a
      Kibana API client.

    - A Separates Fleet Config struct is created.

    - Enroll will now ask for confirmation before replacing the user
      configuration.

* allow to use the obfuscated store.

* fix build
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants