Merge branch 'master' into vlt091-plugin-testing

* master: (31 commits) changelog++ changelog++ Ui/replication status discoverability (#8705) Update CHANGELOG.md Counter that increments on every secret engine lease creation. (#9244) Add password_policy field to Azure docs (#9249) Replaced ClusterMetricSink's cluster name with an atomic.Value. (#9252) Fix database creds rotation panic for nil resp (#9258) changelog++ changelog++ Move sdk/helper/random -> helper/random (#9226) UI: Disallow kv2 with too large 'max versions' value (#9242) Allow mTLS for mysql secrets engine (#9181) docs: add sample revocation for mongodb (#9245) Add new Telemetry config options (#9238) Add a simple sealed gauge, updated when seal status changes (#9177) Test Shamir-to-Transit and Transit-to-Shamir Seal Migration for post-1.4 Vault. (#9214) Configure metrics wrapper with the "global" object, not just the fanout. (#9099) changelog++ Add backend type to audit logs (#9167) ...
hashicorp · Jun 19, 2020 · c1dad08 · c1dad08
2 parents 9c2a34d + 2f1e5b6
commit c1dad08
Show file tree

Hide file tree

Showing 304 changed files with 8,939 additions and 4,666 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -14,6 +14,8 @@ CHANGES:
 
 IMPROVEMENTS:
 
+* audit: Replication status requests are no longer audited. [[GH-8877](https://github.com/hashicorp/vault/pull/8877)]
+* auth/aws: Add support for Web Identity credentials [[GH-7738](https://github.com/hashicorp/vault/pull/7738)]
 * core: Add the Go version used to build a Vault binary to the server message output. [[GH-9078](https://github.com/hashicorp/vault/pull/9078)]
 * core: Added Password Policies for user-configurable password generation [[GH-8637](https://github.com/hashicorp/vault/pull/8637)]
 * cli: Support reading TLS parameters from file for the `vault operator raft join` command. [[GH-9060](https://github.com/hashicorp/vault/pull/9060)]
@@ -32,14 +34,25 @@ IMPROVEMENTS:
 * ui: Update TTL picker styling on SSH secret engine [[GH-8891](https://github.com/hashicorp/vault/pull/8891)]
 * ui: Only render the JWT input field of the Vault login form on mounts configured for JWT auth [[GH-8952](https://github.com/hashicorp/vault/pull/8952)]
 * cli: Add a new subcommand, `vault monitor`, for tailing server logs in the console. [[GH-8477](https://github.com/hashicorp/vault/pull/8477)]
+* ui: Add replication dashboards.  Improve replication management workflows. [[GH-8705]](https://github.com/hashicorp/vault/pull/8705).
 
 BUG FIXES:
 
+* agent: Restart template server when it shuts down [[GH-9200](https://github.com/hashicorp/vault/pull/9200)]
+* auth/oci: Fix issue where users of the Oracle Cloud Infrastructure (OCI) auth method could not authenticate when the plugin backend was mounted at a non-default path. [[GH-7](https://github.com/hashicorp/vault-plugin-auth-oci/pull/7)]
+* core: Extend replicated cubbyhole fix in 1.4.0 to cover case where a performance primary is also a DR primary [[GH-9148](https://github.com/hashicorp/vault/pull/9148)]
+* secrets/aws: Fix issue where performance standbys weren't able to generate STS credentials after an IAM access key rotation in AWS and root IAM credential update in Vault [[GH-9186](https://github.com/hashicorp/vault/pull/9186)]
 * secrets/database: Fix issue where rotating root database credentials while Vault's storage backend is unavailable causes Vault to lose access to the database [[GH-8782](https://github.com/hashicorp/vault/pull/8782)]
+* secrets/database: Fix issue that prevents performance standbys from connecting to databases after a root credential rotation [[GH-9129](https://github.com/hashicorp/vault/pull/9129)]
+* secrets/gcp: Fix issue were updates were not being applied to the `token_scopes` of a roleset. [[GH-90](https://github.com/hashicorp/vault-plugin-secrets-gcp/pull/90)]
 * secrets/kv: Return the value of delete_version_after when reading kv/config, even if it is set to the default. [[GH-42](https://github.com/hashicorp/vault-plugin-secrets-kv/pull/42)]
 * ui: Add Toggle component into core addon so it is available in KMIP and other Ember Engines.[[GH-8913]](https://github.com/hashicorp/vault/pull/8913)
-* secrets/database: Fix issue that prevents performance standbys from connecting to databases after a root credential rotation [[GH-9129](https://github.com/hashicorp/vault/pull/9129)]
-* secrets/aws: Fix issue where performance standbys weren't able to generate STS credentials after an IAM access key rotation in AWS and root IAM credential update in Vault [[GH-9186](https://github.com/hashicorp/vault/pull/9186)]
+* ui: Disallow max versions value of large than 9999999999999999 on kv2 secrets engine. [[GH-9242](https://github.com/hashicorp/vault/pull/9242)]
+
+## 1.4.3 (TBD)
+
+IMPROVEMENTS:
+* auth/aws: Add support for Web Identity credentials [[GH-9251](https://github.com/hashicorp/vault/pull/9251)]
 
 ## 1.4.2 (May 21st, 2020)
 

diff --git a/api/go.sum b/api/go.sum
diff --git a/audit/format.go b/audit/format.go
@@ -114,6 +114,7 @@ func (f *AuditFormatter) FormatRequest(ctx context.Context, w io.Writer, config
 			ClientToken:         req.ClientToken,
 			ClientTokenAccessor: req.ClientTokenAccessor,
 			Operation:           req.Operation,
+			MountType:           req.MountType,
 			Namespace: &AuditNamespace{
 				ID:   ns.ID,
 				Path: ns.Path,
@@ -275,6 +276,7 @@ func (f *AuditFormatter) FormatResponse(ctx context.Context, w io.Writer, config
 			ClientToken:         req.ClientToken,
 			ClientTokenAccessor: req.ClientTokenAccessor,
 			Operation:           req.Operation,
+			MountType:           req.MountType,
 			Namespace: &AuditNamespace{
 				ID:   ns.ID,
 				Path: ns.Path,
@@ -289,13 +291,14 @@ func (f *AuditFormatter) FormatResponse(ctx context.Context, w io.Writer, config
 		},
 
 		Response: &AuditResponse{
-			Auth:     respAuth,
-			Secret:   respSecret,
-			Data:     resp.Data,
-			Warnings: resp.Warnings,
-			Redirect: resp.Redirect,
-			WrapInfo: respWrapInfo,
-			Headers:  resp.Headers,
+			MountType: req.MountType,
+			Auth:      respAuth,
+			Secret:    respSecret,
+			Data:      resp.Data,
+			Warnings:  resp.Warnings,
+			Redirect:  resp.Redirect,
+			WrapInfo:  respWrapInfo,
+			Headers:   resp.Headers,
 		},
 	}
 
@@ -336,6 +339,7 @@ type AuditRequest struct {
 	ID                            string                 `json:"id,omitempty"`
 	ReplicationCluster            string                 `json:"replication_cluster,omitempty"`
 	Operation                     logical.Operation      `json:"operation,omitempty"`
+	MountType                     string                 `json:"mount_type,omitempty"`
 	ClientToken                   string                 `json:"client_token,omitempty"`
 	ClientTokenAccessor           string                 `json:"client_token_accessor,omitempty"`
 	Namespace                     *AuditNamespace        `json:"namespace,omitempty"`
@@ -349,13 +353,14 @@ type AuditRequest struct {
 }
 
 type AuditResponse struct {
-	Auth     *AuditAuth             `json:"auth,omitempty"`
-	Secret   *AuditSecret           `json:"secret,omitempty"`
-	Data     map[string]interface{} `json:"data,omitempty"`
-	Warnings []string               `json:"warnings,omitempty"`
-	Redirect string                 `json:"redirect,omitempty"`
-	WrapInfo *AuditResponseWrapInfo `json:"wrap_info,omitempty"`
-	Headers  map[string][]string    `json:"headers,omitempty"`
+	Auth      *AuditAuth             `json:"auth,omitempty"`
+	MountType string                 `json:"mount_type,omitempty"`
+	Secret    *AuditSecret           `json:"secret,omitempty"`
+	Data      map[string]interface{} `json:"data,omitempty"`
+	Warnings  []string               `json:"warnings,omitempty"`
+	Redirect  string                 `json:"redirect,omitempty"`
+	WrapInfo  *AuditResponseWrapInfo `json:"wrap_info,omitempty"`
+	Headers   map[string][]string    `json:"headers,omitempty"`
 }
 
 type AuditAuth struct {

diff --git a/builtin/logical/database/path_rotate_credentials.go b/builtin/logical/database/path_rotate_credentials.go
@@ -187,7 +187,7 @@ func (b *databaseBackend) pathRotateRoleCredentialsUpdate() framework.OperationF
 			item.Priority = time.Now().Add(10 * time.Second).Unix()
 
 			// Preserve the WALID if it was returned
-			if resp.WALID != "" {
+			if resp != nil && resp.WALID != "" {
 				item.Value = resp.WALID
 			}
 		} else {

diff --git a/builtin/logical/rabbitmq/backend_test.go b/builtin/logical/rabbitmq/backend_test.go
@@ -9,10 +9,10 @@ import (
 	"testing"
 
 	"github.com/hashicorp/vault/helper/testhelpers/docker"
-	logicaltest "github.com/hashicorp/vault/sdk/testing/stepwise"
+	"github.com/hashicorp/vault/sdk/helper/base62"
 	"github.com/hashicorp/vault/sdk/helper/jsonutil"
-	"github.com/hashicorp/vault/sdk/helper/random"
 	"github.com/hashicorp/vault/sdk/logical"
+	logicaltest "github.com/hashicorp/vault/sdk/testing/stepwise"
 	rabbithole "github.com/michaelklishin/rabbit-hole"
 	"github.com/mitchellh/mapstructure"
 	"github.com/ory/dockertest"
@@ -163,7 +163,10 @@ func TestBackend_roleWithPasswordPolicy(t *testing.T) {
 	}
 
 	backendConfig := logical.TestBackendConfig()
-	backendConfig.System.(*logical.StaticSystemView).SetPasswordPolicy("testpolicy", random.DefaultStringGenerator)
+	passGen := func() (password string, err error) {
+		return base62.Random(30)
+	}
+	backendConfig.System.(*logical.StaticSystemView).SetPasswordPolicy("testpolicy", passGen)
 	b, _ := Factory(context.Background(), backendConfig)
 
 	cleanup, uri, _ := prepareRabbitMQTestContainer(t)

diff --git a/command/agent/template/template.go b/command/agent/template/template.go
@@ -153,7 +153,13 @@ func (ts *Server) Run(ctx context.Context, incoming chan string, templates []*ct
 			}
 		case err := <-ts.runner.ErrCh:
 			ts.logger.Error("template server error", "error", err.Error())
-			return
+			ts.runner.StopImmediately()
+			ts.runner, err = manager.NewRunner(runnerConfig, false)
+			if err != nil {
+				ts.logger.Error("template server failed to create", "error", err)
+				return
+			}
+			go ts.runner.Start()
 		case <-ts.runner.TemplateRenderedCh():
 			// A template has been rendered, figure out what to do
 			events := ts.runner.RenderEvents()

diff --git a/command/server/config_test_helpers.go b/command/server/config_test_helpers.go
@@ -66,6 +66,8 @@ func testLoadConfigFile_topLevel(t *testing.T, entropy *configutil.Entropy) {
 				DogStatsDAddr:           "127.0.0.1:7254",
 				DogStatsDTags:           []string{"tag_1:val_1", "tag_2:val_2"},
 				PrometheusRetentionTime: 30 * time.Second,
+				UsageGaugePeriod:        5 * time.Minute,
+				MaximumGaugeCardinality: 125,
 			},
 
 			DisableMlock: true,
@@ -170,6 +172,8 @@ func testLoadConfigFile_json2(t *testing.T, entropy *configutil.Entropy) {
 				StatsiteAddr:                       "foo",
 				StatsdAddr:                         "bar",
 				DisableHostname:                    true,
+				UsageGaugePeriod:                   5 * time.Minute,
+				MaximumGaugeCardinality:            125,
 				CirconusAPIToken:                   "0",
 				CirconusAPIApp:                     "vault",
 				CirconusAPIURL:                     "http://api.circonus.com/v2",
@@ -364,6 +368,8 @@ func testLoadConfigFile(t *testing.T) {
 				StatsdAddr:              "bar",
 				StatsiteAddr:            "foo",
 				DisableHostname:         false,
+				UsageGaugePeriod:        5 * time.Minute,
+				MaximumGaugeCardinality: 100,
 				DogStatsDAddr:           "127.0.0.1:7254",
 				DogStatsDTags:           []string{"tag_1:val_1", "tag_2:val_2"},
 				PrometheusRetentionTime: configutil.PrometheusDefaultRetentionTime,
@@ -446,6 +452,8 @@ func testLoadConfigFile_json(t *testing.T) {
 				StatsiteAddr:                       "baz",
 				StatsdAddr:                         "",
 				DisableHostname:                    false,
+				UsageGaugePeriod:                   5 * time.Minute,
+				MaximumGaugeCardinality:            100,
 				CirconusAPIToken:                   "",
 				CirconusAPIApp:                     "",
 				CirconusAPIURL:                     "",
@@ -523,6 +531,8 @@ func testLoadConfigDir(t *testing.T) {
 				StatsiteAddr:            "qux",
 				StatsdAddr:              "baz",
 				DisableHostname:         true,
+				UsageGaugePeriod:        5 * time.Minute,
+				MaximumGaugeCardinality: 100,
 				PrometheusRetentionTime: configutil.PrometheusDefaultRetentionTime,
 			},
 			ClusterName: "testcluster",
@@ -616,6 +626,8 @@ func testConfig_Sanitized(t *testing.T) {
 			"type": "consul",
 		},
 		"telemetry": map[string]interface{}{
+			"usage_gauge_period":                     5 * time.Minute,
+			"maximum_gauge_cardinality":              100,
 			"circonus_api_app":                       "",
 			"circonus_api_token":                     "",
 			"circonus_api_url":                       "",

diff --git a/command/server/test-fixtures/config-dir/baz.hcl b/command/server/test-fixtures/config-dir/baz.hcl
@@ -2,6 +2,8 @@ telemetry {
     statsd_address = "baz"
     statsite_address = "qux"
     disable_hostname = true
+    usage_gauge_period = "5m"
+    maximum_gauge_cardinality = 100
 }
 ui=true
 raw_storage_endpoint=true

diff --git a/command/server/test-fixtures/config.hcl b/command/server/test-fixtures/config.hcl
@@ -25,6 +25,9 @@ service_registration "consul" {
 
 telemetry {
     statsd_address = "bar"
+    usage_gauge_period = "5m"
+    maximum_gauge_cardinality = 100
+
     statsite_address = "foo"
     dogstatsd_addr = "127.0.0.1:7254"
     dogstatsd_tags = ["tag_1:val_1", "tag_2:val_2"]

diff --git a/command/server/test-fixtures/config.hcl.json b/command/server/test-fixtures/config.hcl.json
@@ -17,7 +17,9 @@
 		}
 	},
 	"telemetry": {
-		"statsite_address": "baz"
+		"statsite_address": "baz",
+		"usage_gauge_period": "5m",
+		"maximum_gauge_cardinality": 100
 	},
 	"max_lease_ttl": "10h",
 	"default_lease_ttl": "10h",

diff --git a/command/server/test-fixtures/config2.hcl b/command/server/test-fixtures/config2.hcl
@@ -27,6 +27,8 @@ service_registration "consul" {
 
 telemetry {
     statsd_address = "bar"
+    usage_gauge_period = "5m"
+    maximum_gauge_cardinality = 125
     statsite_address = "foo"
     dogstatsd_addr = "127.0.0.1:7254"
     dogstatsd_tags = ["tag_1:val_1", "tag_2:val_2"]

diff --git a/command/server/test-fixtures/config2.hcl.json b/command/server/test-fixtures/config2.hcl.json
@@ -35,6 +35,9 @@
     "statsd_address":"bar",
     "statsite_address":"foo",
     "disable_hostname":true,
+    "usage_gauge_period": "5m",
+    "maximum_gauge_cardinality": 125,
+
     "circonus_api_token": "0",
     "circonus_api_app": "vault",
     "circonus_api_url": "http://api.circonus.com/v2",

diff --git a/command/server/test-fixtures/config3.hcl b/command/server/test-fixtures/config3.hcl
@@ -30,6 +30,8 @@ telemetry {
   statsd_address = "bar"
   circonus_api_token = "baz"
   metrics_prefix = "pfx"
+  usage_gauge_period = "5m"
+  maximum_gauge_cardinality = 100
 }
 
 seal "awskms" {