Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve handling of errors when binary store handles bad data #1289

Merged
merged 1 commit into from
Sep 21, 2023
Merged

Improve handling of errors when binary store handles bad data #1289

merged 1 commit into from
Sep 21, 2023

Conversation

felixfontein
Copy link
Contributor

When the binary store tries to write data and data is not there, or data is not a string, or zero or multiple tree branches have been provided, there's now a nice error also explaining about --output-format. (The only error handled so far was data not being there in the first tree branch, but without that --output-format hint.)

Fixes #596.

@felixfontein felixfontein mentioned this pull request Sep 16, 2023
Closed
@felixfontein felixfontein mentioned this pull request Sep 16, 2023
Open
@felixfontein felixfontein requested a review from a team September 16, 2023 13:25
hiddeco
hiddeco reviewed Sep 18, 2023
View reviewed changes
stores/json/store.go Outdated Show resolved Hide resolved
hiddeco
hiddeco reviewed Sep 18, 2023
View reviewed changes
stores/json/store.go Outdated Show resolved Hide resolved
hiddeco
hiddeco reviewed Sep 18, 2023
View reviewed changes
stores/json/store.go Outdated Show resolved Hide resolved
hiddeco
hiddeco approved these changes Sep 19, 2023
View reviewed changes
Copy link
Member

@hiddeco hiddeco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last commit requires as sign-off, and typically commit summaries do not end with a dot. Other than these nitpicks, it looks good to me 🙇

stores/json/store.go Outdated
// EmitPlainFile produces plaintext json file's bytes from its corresponding sops.TreeBranches object
func (store BinaryStore) EmitPlainFile(in sops.TreeBranches) ([]byte, error) {
if len(in) != 1 {
return nil, BinaryStoreEmitPlainError{message: "There must be exactly one tree branch"}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for future reference for other Go projects, and not applicable here:

  1. Normally in Go, error strings should not be capitalized (https://github.com/golang/go/wiki/CodeReviewComments#error-strings). However, that ship has sailed in this code base :-).
  2. The more idiomatic name would be something along the lines of ErrEmitPlain. However, I can see you probably based this on NotImplementedError, decryptGroupError, etc. already existing in the code base.

@hiddeco hiddeco added this to the v3.8.1 milestone Sep 19, 2023
holiman
holiman reviewed Sep 20, 2023
View reviewed changes
cmd/sops/decrypt.go Outdated Show resolved Hide resolved
@felixfontein felixfontein force-pushed the binary-data-error branch 2 times, most recently from 51956d9 to b7c2142 Compare September 20, 2023 19:35
hiddeco
hiddeco reviewed Sep 20, 2023
View reviewed changes
cmd/sops/decrypt.go Outdated Show resolved Hide resolved
hiddeco
hiddeco approved these changes Sep 21, 2023
View reviewed changes
Copy link
Member

@hiddeco hiddeco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you @felixfontein 🥇

@felixfontein
Copy link
Contributor Author

@hiddeco should I squash the commits?

@hiddeco
Copy link
Member

hiddeco commented Sep 21, 2023

Probably best, yes.

@felixfontein
Improve handling of errors when binary store handles bad data
f258e6c 
Makes sure that the 'data' key refers to strings. Also
improves error messages, and on CLI hints at the --output-type
option.

Signed-off-by: Felix Fontein <[email protected]>
@felixfontein felixfontein merged commit 80ec1d0 into getsops:main Sep 21, 2023
9 checks passed
@felixfontein felixfontein deleted the binary-data-error branch September 21, 2023 06:31
@felixfontein
Copy link
Contributor Author

@hiddeco @holiman thanks a lot for reviewing!

mergify bot referenced this pull request in dbsystel/cdk-sops-secrets Oct 11, 2023
@renovate
fix(deps): update module github.com/getsops/sops/v3 to v3.8.1 (#878)
218ddf3 
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/getsops/sops/v3](https://github.com/getsops/sops) | require | patch | `v3.8.0` -> `v3.8.1` |

---

### Release Notes

<details>
<summary>getsops/sops (github.com/getsops/sops/v3)</summary>

### [`v3.8.1`](https://github.com/getsops/sops/releases/tag/v3.8.1)

[Compare Source](https://github.com/getsops/sops/compare/v3.8.0...v3.8.1)

#### Note from the Maintainers

In this release of SOPS, we have focused on landing a variety of bug fixes to improve the overall user experience.

For a comprehensive list of changes, please refer to [`CHANGELOG.rst`](https://github.com/getsops/sops/blob/v3.8.1/CHANGELOG.rst).

##### Important Information for SOPS SDK Users

With the project transitioning from the Mozilla Foundation to the CNCF, the Go module path has been updated to reflect this change in ownership. If you use `go.mozilla.org/sops/v3` as a library, going forward, import the Go Module using `github.com/getsops/sops/v3`. Apart from this small adjustment, the SDK's API remains fully backward compatible.

For a one-liner to quickly implement this change throughout your codebase, please refer to: [https://github.com/getsops/sops/issues/1246#issuecomment-1625526429](https://github.com/getsops/sops/issues/1246#issuecomment-1625526429)

#### Installation

To install `sops`, download one of the pre-built binaries provided for your platform from the artifacts attached to this release.

For instance, if you are using Linux on an AMD64 architecture:

```shell

### Download the binary
curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.linux.amd64

### Move the binary in to your PATH
mv sops-v3.8.1.linux.amd64 /usr/local/bin/sops

### Make the binary executable
chmod +x /usr/local/bin/sops
```

##### Verify checksums file signature

The checksums file provided within the artifacts attached to this release is signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of this file, run the following commands:

```shell

### Download the checksums file, certificate and signature
curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.checksums.txt
curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.checksums.pem
curl -LO https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.checksums.sig

### Verify the checksums file
cosign verify-blob sops-v3.8.1.checksums.txt \
  --certificate sops-v3.8.1.checksums.pem \
  --signature sops-v3.8.1.checksums.sig \
  --certificate-identity-regexp=https://github.com/getsops \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com
```

##### Verify binary integrity

To verify the integrity of the downloaded binary, you can utilize the checksums file after having validated its signature:

```shell

### Verify the binary using the checksums file
sha256sum -c sops-v3.8.1.checksums.txt --ignore-missing
```

##### Verify artifact provenance

The [SLSA provenance](https://slsa.dev/provenance/v0.2) of the binaries, packages, and SBOMs can be found within the artifacts associated with this release. It is presented through an [in-toto](https://in-toto.io/) link metadata file named `sops-v3.8.1.intoto.jsonl`. To verify the provenance of an artifact, you can utilize the [`slsa-verifier`](https://github.com/slsa-framework/slsa-verifier#artifacts) tool:

```shell

### Download the metadata file
curl -LO  https://github.com/getsops/sops/releases/download/v3.8.1/sops-v3.8.1.intoto.jsonl

### Verify the provenance of the artifact
slsa-verifier verify-artifact <artifact> \
  --provenance-path sops-v3.8.1.intoto.jsonl \
  --source-uri github.com/getsops/sops \
  --source-tag v3.8.1
```

#### Container Images

The `sops` binaries are also available as container images, based on Debian (slim) and Alpine Linux. The Debian-based container images include any dependencies which may be required to make use of certain key services, such as GnuPG, AWS KMS, Azure Key Vault, and Google Cloud KMS. The Alpine-based container images are smaller in size, but do not include these dependencies.

These container images are available for the following architectures: `linux/amd64` and `linux/arm64`.

##### GitHub Container Registry

-   `ghcr.io/getsops/sops:v3.8.1`
-   `ghcr.io/getsops/sops:v3.8.1-alpine`

##### Quay.io

-   `quay.io/getsops/sops:v3.8.1`
-   `quay.io/getsops/sops:v3.8.1-alpine`

##### Verify container image signature

The container images are signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of an image, run the following command:

```shell
cosign verify ghcr.io/getsops/sops:v3.8.1 \
  --certificate-identity-regexp=https://github.com/getsops \
  --certificate-oidc-issuer=https://token.actions.githubusercontent.com \
  -o text
```

##### Verify container image provenance

The container images include [SLSA provenance](https://slsa.dev/provenance/v0.2) attestations. For more information around the verification of this, please refer to the [`slsa-verifier` documentation](https://github.com/slsa-framework/slsa-verifier#containers).

#### Software Bill of Materials

The Software Bill of Materials (SBOM) for each binary is accessible within the artifacts enclosed with this release. It is presented as an [SPDX](https://spdx.dev/) JSON file, formatted as `<binary>.spdx.sbom.json`.

#### What's Changed

-   build(deps): Bump the ci group with 3 updates by [@&#8203;dependabot](https://github.com/dependabot) in [https://github.com/getsops/sops/pull/1295](https://github.com/getsops/sops/pull/1295)
-   pgp: improve handling of GnuPG home dir by [@&#8203;hiddeco](https://github.com/hiddeco) in [https://github.com/getsops/sops/pull/1298](https://github.com/getsops/sops/pull/1298)
-   Delete sops encrypted file we don't have keys for by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1288](https://github.com/getsops/sops/pull/1288)
-   Improve handling of errors when binary store handles bad data by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1289](https://github.com/getsops/sops/pull/1289)
-   On macOS, prefer XDG_CONFIG_HOME over os.UserConfigDir() by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1291](https://github.com/getsops/sops/pull/1291)
-   Do not crash if an empty YAML file is encrypted by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1290](https://github.com/getsops/sops/pull/1290)
-   Fix descriptions of unencrypted-regex and encrypted-regex flags, and ensure unencrypted_regex is considered in config validation by [@&#8203;mitar](https://github.com/mitar) in [https://github.com/getsops/sops/pull/1300](https://github.com/getsops/sops/pull/1300)
-   build(deps): Bump the go group with 4 updates by [@&#8203;dependabot](https://github.com/dependabot) in [https://github.com/getsops/sops/pull/1306](https://github.com/getsops/sops/pull/1306)
-   build(deps): Bump the ci group with 1 update by [@&#8203;dependabot](https://github.com/dependabot) in [https://github.com/getsops/sops/pull/1301](https://github.com/getsops/sops/pull/1301)
-   Handle return values of dec.Token() to improve error messages by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1307](https://github.com/getsops/sops/pull/1307)
-   pgp: make error the last return value by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1310](https://github.com/getsops/sops/pull/1310)
-   pgp: do not require abs path for SopsGpgExecEnv  by [@&#8203;holiman](https://github.com/holiman) in [https://github.com/getsops/sops/pull/1309](https://github.com/getsops/sops/pull/1309)
-   decrypt: fix dropped error by [@&#8203;alrs](https://github.com/alrs) in [https://github.com/getsops/sops/pull/1304](https://github.com/getsops/sops/pull/1304)
-   Handle errors by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1311](https://github.com/getsops/sops/pull/1311)
-   Report key rotation errors by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1317](https://github.com/getsops/sops/pull/1317)
-   cmd/sops/main.go: make sure to wrap raw errors with toExitError() by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1318](https://github.com/getsops/sops/pull/1318)
-   build(deps): Bump the go group with 7 updates by [@&#8203;dependabot](https://github.com/dependabot) in [https://github.com/getsops/sops/pull/1319](https://github.com/getsops/sops/pull/1319)
-   Enrich AWS authentication documentation by [@&#8203;nsantiago2719](https://github.com/nsantiago2719) in [https://github.com/getsops/sops/pull/1272](https://github.com/getsops/sops/pull/1272)
-   Better error reporting for missing gpg binary by [@&#8203;makkes](https://github.com/makkes) in [https://github.com/getsops/sops/pull/1286](https://github.com/getsops/sops/pull/1286)
-   Improve RST and MD files by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1320](https://github.com/getsops/sops/pull/1320)
-   Add linting for RST and MD files by [@&#8203;felixfontein](https://github.com/felixfontein) in [https://github.com/getsops/sops/pull/1287](https://github.com/getsops/sops/pull/1287)
-   Update dependencies by [@&#8203;hiddeco](https://github.com/hiddeco) in [https://github.com/getsops/sops/pull/1325](https://github.com/getsops/sops/pull/1325)
-   Prepare v3.8.1 by [@&#8203;hiddeco](https://github.com/hiddeco) in [https://github.com/getsops/sops/pull/1324](https://github.com/getsops/sops/pull/1324)

#### New Contributors

-   [@&#8203;mitar](https://github.com/mitar) made their first contribution in [https://github.com/getsops/sops/pull/1300](https://github.com/getsops/sops/pull/1300)
-   [@&#8203;holiman](https://github.com/holiman) made their first contribution in [https://github.com/getsops/sops/pull/1309](https://github.com/getsops/sops/pull/1309)
-   [@&#8203;alrs](https://github.com/alrs) made their first contribution in [https://github.com/getsops/sops/pull/1304](https://github.com/getsops/sops/pull/1304)
-   [@&#8203;nsantiago2719](https://github.com/nsantiago2719) made their first contribution in [https://github.com/getsops/sops/pull/1272](https://github.com/getsops/sops/pull/1272)
-   [@&#8203;makkes](https://github.com/makkes) made their first contribution in [https://github.com/getsops/sops/pull/1286](https://github.com/getsops/sops/pull/1286)

**Full Changelog**: getsops/sops@v3.8.0...v3.8.1

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/markussiebert/cdk-sops-secrets).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@holiman holiman holiman left review comments

@hiddeco hiddeco hiddeco approved these changes

Assignees
No one assigned
Labels
area/CLI bug stores/json
Projects
None yet
Milestone
v3.8.1
Development

Successfully merging this pull request may close these issues.

SOPS panics when input type is not binary but output type is and input contains a 'data' mapping key.
3 participants
@felixfontein @hiddeco @holiman