Mock server to handle benchmarking requests. Added cucumber tests for

performance benchmarks

Refactor to make code more usable

Refactor to make code more usable

Refactor to not use singleton for mock server

Ensures methods from findAndGetMethod will return method who succeed the request

checking out what is in upstream/master

profiling memory tests

Logging

renaming struct

credentials: Expose provider names

credentials: Add tests covering provider names

Using whitelists instead of blacklists.

Added new method to return the signature while hoisting all headers

Added new method to return the signature while hoisting all headers

Update to use whitelists and blacklists depending on whether it is a presign function or not

updated names and using http.Header

Update unit test

more tests and verified that x-amz-meta-* are not good in query.

Changed filter to Validator. Allow for rules to be added to the validator

Fixed typo

Changing to be private

Changing buildQuery to use the rule. Fixing pattern for one of the validators

More tests and added more rules

service/waf: add support for http body inspection and size contraints

AWS WAF

You can now configure AWS WAF to block, allow, or monitor (count) requests
based on the content in HTTP request bodies. This is the part of a request
that contains any additional data that you want to send to your web server
as the HTTP request body, such as data from an HTML form.

You can also set size constraints on specified parts of the requests, which
let AWS WAF allow, block, or count web requests based on the lengths of
specified parts of the requests such as query strings, URIs, or request body.

service/ssm: Add support for EC2 instance id length increase

Amazon Simple Systems Management Service

Add paginators and remove of instance id length constraint

Tag release v1.1.0

References:
  aws#512

Usage of enviroment variables to change outputer

Performance tests and mock server

Moving things around. Added new reflection method

Response are stored. Fix saving to file bug

removing this blank file

Fixed marshaller

Fixing import

Gofmt fix for request pagination

credentials: Expose provider names

credentials: Add tests covering provider names

Update to use whitelists and blacklists depending on whether it is a presign function or not

updated names and using http.Header

Update unit test

more tests and verified that x-amz-meta-* are not good in query.

Changed filter to Validator. Allow for rules to be added to the validator

Fixed typo

Changing to be private

More tests and added more rules

service/waf: add support for http body inspection and size contraints

AWS WAF

You can now configure AWS WAF to block, allow, or monitor (count) requests
based on the content in HTTP request bodies. This is the part of a request
that contains any additional data that you want to send to your web server
as the HTTP request body, such as data from an HTML form.

You can also set size constraints on specified parts of the requests, which
let AWS WAF allow, block, or count web requests based on the lengths of
specified parts of the requests such as query strings, URIs, or request body.

service/ssm: Add support for EC2 instance id length increase

Amazon Simple Systems Management Service

Add paginators and remove of instance id length constraint

Tag release v1.1.0

References:
  aws#512

Add support for converting data of type []byte to DynamodbAttribute.

Document ConvertTo []byte support is only for map[string]interface[]

Don't unmarshal unmodeled API operation responses

Updates the SDK to not unmarshal the HTTP response body of opertions
which do not have modeled output shapes. This ensures the SDK does not
attempt to unmarshal invalid response from a service. If the response
has a body it will be discarded and closed.

This change also adds named request handlers for build and unmarshaler.
This allows you to remove protocol build and unmarshal handlers by name.
The name pattern is:

```
awssdk.<protocol>.Build|Unmarshal|UnmarshalMeta|UnmarshalError
```

Add Generated service clients for named request handlers

Fix up example code for unwrapping awserr.Error

Update go-jmespath to 0b12d6b52

Fix aws#457
Fix aws#535

Replace API gen shape tag strings with types to improve maintainability

Add idempotency token auto fill

Adds support for service models which specify a shape member is the
idempotency token to be auto filled with a random UUID if the member's
value is not already set.

This replaces the needs for customizations per service per request when
an idempotency token should always be provided, but optionally set by the
user.

Update protocol marshaler test definitions for idempotency token

Code generate protocol marshaler tests for idempotency token

Fix go tip vet failing

Mock server to handle benchmarking requests. Added cucumber tests for
performance benchmarks

Refactor to make code more usable

Refactor to make code more usable

Refactor to not use singleton for mock server

Ensures methods from findAndGetMethod will return method who succeed the request

Makefile

@@ -17,6 +17,7 @@ help:
 	@echo "  build                   to go build the SDK"
 	@echo "  unit                    to run unit tests"
 	@echo "  integration             to run integration tests"
+	@echo "  performance             to run performance tests"
 	@echo "  verify                  to verify tests"
 	@echo "  lint                    to lint the SDK"
 	@echo "  vet                     to vet the SDK"
@@ -54,7 +55,10 @@ unit-with-race-cover: get-deps-tests build verify
 
 integration: get-deps-tests
 	go test -tags=integration ./awstesting/integration/customizations/...
-	gucumber ./awstesting/integration/smoke
+	AWS_TESTING_OUTPUT=$(output) AWS_TESTING_REGION=$(region) gucumber ./awstesting/integration/smoke
+
+performance: get-deps-tests
+	AWS_TESTING_OUTPUT=$(output) AWS_TESTING_REGION=$(region) AWS_MOCK_DATA=$(mock) gucumber ./awstesting/performance
 
 verify: get-deps-verify lint vet
 

aws/awserr/error.go

@@ -14,13 +14,13 @@ package awserr
 //     if err != nil {
 //         if awsErr, ok := err.(awserr.Error); ok {
 //             // Get error details
-//             log.Println("Error:", err.Code(), err.Message())
+//             log.Println("Error:", awsErr.Code(), awsErr.Message())
 //
 //             // Prints out full error message, including original error if there was one.
-//             log.Println("Error:", err.Error())
+//             log.Println("Error:", awsErr.Error())
 //
 //             // Get original error
-//             if origErr := err.Err(); origErr != nil {
+//             if origErr := awsErr.OrigErr(); origErr != nil {
 //                 // operate on original error.
 //             }
 //         } else {

aws/corehandlers/handlers_test.go

@@ -50,7 +50,7 @@ type mockCredsProvider struct {
 
 func (m *mockCredsProvider) Retrieve() (credentials.Value, error) {
 	m.retrieveCalled = true
-	return credentials.Value{}, nil
+	return credentials.Value{ProviderName: "mockCredsProvider"}, nil
 }
 
 func (m *mockCredsProvider) IsExpired() bool {

aws/credentials/chain_provider_test.go

@@ -7,6 +7,54 @@ import (
 	"github.com/stretchr/testify/assert"
 )
 
+type secondStubProvider struct {
+	creds   Value
+	expired bool
+	err     error
+}
+
+func (s *secondStubProvider) Retrieve() (Value, error) {
+	s.expired = false
+	s.creds.ProviderName = "secondStubProvider"
+	return s.creds, s.err
+}
+func (s *secondStubProvider) IsExpired() bool {
+	return s.expired
+}
+
+func TestChainProviderWithNames(t *testing.T) {
+	p := &ChainProvider{
+		Providers: []Provider{
+			&stubProvider{err: awserr.New("FirstError", "first provider error", nil)},
+			&stubProvider{err: awserr.New("SecondError", "second provider error", nil)},
+			&secondStubProvider{
+				creds: Value{
+					AccessKeyID:     "AKIF",
+					SecretAccessKey: "NOSECRET",
+					SessionToken:    "",
+				},
+			},
+			&stubProvider{
+				creds: Value{
+					AccessKeyID:     "AKID",
+					SecretAccessKey: "SECRET",
+					SessionToken:    "",
+				},
+			},
+		},
+	}
+
+	creds, err := p.Retrieve()
+	assert.Nil(t, err, "Expect no error")
+	assert.Equal(t, "secondStubProvider", creds.ProviderName, "Expect provider name to match")
+
+	// Also check credentials
+	assert.Equal(t, "AKIF", creds.AccessKeyID, "Expect access key ID to match")
+	assert.Equal(t, "NOSECRET", creds.SecretAccessKey, "Expect secret access key to match")
+	assert.Empty(t, creds.SessionToken, "Expect session token to be empty")
+
+}
+
 func TestChainProviderGet(t *testing.T) {
 	p := &ChainProvider{
 		Providers: []Provider{

aws/credentials/credentials.go

@@ -76,6 +76,9 @@ type Value struct {
 
 	// AWS Session Token
 	SessionToken string
+
+	// Provider used to get credentials
+	ProviderName string
 }
 
 // A Provider is the interface for any component which will provide credentials

aws/credentials/credentials_test.go

@@ -15,6 +15,7 @@ type stubProvider struct {
 
 func (s *stubProvider) Retrieve() (Value, error) {
 	s.expired = false
+	s.creds.ProviderName = "stubProvider"
 	return s.creds, s.err
 }
 func (s *stubProvider) IsExpired() bool {
@@ -60,3 +61,13 @@ func TestCredentialsExpire(t *testing.T) {
 	stub.expired = true
 	assert.True(t, c.IsExpired(), "Expected to be expired")
 }
+
+func TestCredentialsGetWithProviderName(t *testing.T) {
+	stub := &stubProvider{}
+
+	c := NewCredentials(stub)
+
+	creds, err := c.Get()
+	assert.Nil(t, err, "Expected no error")
+	assert.Equal(t, creds.ProviderName, "stubProvider", "Expected provider name to match")
+}

aws/credentials/ec2rolecreds/ec2_role_provider.go

@@ -14,6 +14,9 @@ import (
 	"github.com/aws/aws-sdk-go/aws/ec2metadata"
 )
 
+// ProviderName provides a name of EC2Role provider
+const ProviderName = "EC2RoleProvider"
+
 // A EC2RoleProvider retrieves credentials from the EC2 service, and keeps track if
 // those credentials are expired.
 //
@@ -85,17 +88,17 @@ func NewCredentialsWithClient(client *ec2metadata.EC2Metadata, options ...func(*
 func (m *EC2RoleProvider) Retrieve() (credentials.Value, error) {
 	credsList, err := requestCredList(m.Client)
 	if err != nil {
-		return credentials.Value{}, err
+		return credentials.Value{ProviderName: ProviderName}, err
 	}
 
 	if len(credsList) == 0 {
-		return credentials.Value{}, awserr.New("EmptyEC2RoleList", "empty EC2 Role list", nil)
+		return credentials.Value{ProviderName: ProviderName}, awserr.New("EmptyEC2RoleList", "empty EC2 Role list", nil)
 	}
 	credsName := credsList[0]
 
 	roleCreds, err := requestCred(m.Client, credsName)
 	if err != nil {
-		return credentials.Value{}, err
+		return credentials.Value{ProviderName: ProviderName}, err
 	}
 
 	m.SetExpiration(roleCreds.Expiration, m.ExpiryWindow)
@@ -104,6 +107,7 @@ func (m *EC2RoleProvider) Retrieve() (credentials.Value, error) {
 		AccessKeyID:     roleCreds.AccessKeyID,
 		SecretAccessKey: roleCreds.SecretAccessKey,
 		SessionToken:    roleCreds.Token,
+		ProviderName:    ProviderName,
 	}, nil
 }
 

aws/credentials/env_provider.go

@@ -6,6 +6,9 @@ import (
 	"github.com/aws/aws-sdk-go/aws/awserr"
 )
 
+// EnvProviderName provides a name of Env provider
+const EnvProviderName = "EnvProvider"
+
 var (
 	// ErrAccessKeyIDNotFound is returned when the AWS Access Key ID can't be
 	// found in the process's environment.
@@ -52,18 +55,19 @@ func (e *EnvProvider) Retrieve() (Value, error) {
 	}
 
 	if id == "" {
-		return Value{}, ErrAccessKeyIDNotFound
+		return Value{ProviderName: EnvProviderName}, ErrAccessKeyIDNotFound
 	}
 
 	if secret == "" {
-		return Value{}, ErrSecretAccessKeyNotFound
+		return Value{ProviderName: EnvProviderName}, ErrSecretAccessKeyNotFound
 	}
 
 	e.retrieved = true
 	return Value{
 		AccessKeyID:     id,
 		SecretAccessKey: secret,
 		SessionToken:    os.Getenv("AWS_SESSION_TOKEN"),
+		ProviderName:    EnvProviderName,
 	}, nil
 }
 

aws/credentials/shared_credentials_provider.go

@@ -10,6 +10,9 @@ import (
 	"github.com/aws/aws-sdk-go/aws/awserr"
 )
 
+// SharedCredsProviderName provides a name of SharedCreds provider
+const SharedCredsProviderName = "SharedCredentialsProvider"
+
 var (
 	// ErrSharedCredentialsHomeNotFound is emitted when the user directory cannot be found.
 	//
@@ -55,12 +58,12 @@ func (p *SharedCredentialsProvider) Retrieve() (Value, error) {
 
 	filename, err := p.filename()
 	if err != nil {
-		return Value{}, err
+		return Value{ProviderName: SharedCredsProviderName}, err
 	}
 
 	creds, err := loadProfile(filename, p.profile())
 	if err != nil {
-		return Value{}, err
+		return Value{ProviderName: SharedCredsProviderName}, err
 	}
 
 	p.retrieved = true
@@ -78,23 +81,23 @@ func (p *SharedCredentialsProvider) IsExpired() bool {
 func loadProfile(filename, profile string) (Value, error) {
 	config, err := ini.Load(filename)
 	if err != nil {
-		return Value{}, awserr.New("SharedCredsLoad", "failed to load shared credentials file", err)
+		return Value{ProviderName: SharedCredsProviderName}, awserr.New("SharedCredsLoad", "failed to load shared credentials file", err)
 	}
 	iniProfile, err := config.GetSection(profile)
 	if err != nil {
-		return Value{}, awserr.New("SharedCredsLoad", "failed to get profile", err)
+		return Value{ProviderName: SharedCredsProviderName}, awserr.New("SharedCredsLoad", "failed to get profile", err)
 	}
 
 	id, err := iniProfile.GetKey("aws_access_key_id")
 	if err != nil {
-		return Value{}, awserr.New("SharedCredsAccessKey",
+		return Value{ProviderName: SharedCredsProviderName}, awserr.New("SharedCredsAccessKey",
 			fmt.Sprintf("shared credentials %s in %s did not contain aws_access_key_id", profile, filename),
 			err)
 	}
 
 	secret, err := iniProfile.GetKey("aws_secret_access_key")
 	if err != nil {
-		return Value{}, awserr.New("SharedCredsSecret",
+		return Value{ProviderName: SharedCredsProviderName}, awserr.New("SharedCredsSecret",
 			fmt.Sprintf("shared credentials %s in %s did not contain aws_secret_access_key", profile, filename),
 			nil)
 	}
@@ -106,6 +109,7 @@ func loadProfile(filename, profile string) (Value, error) {
 		AccessKeyID:     id.String(),
 		SecretAccessKey: secret.String(),
 		SessionToken:    token.String(),
+		ProviderName:    SharedCredsProviderName,
 	}, nil
 }
 

aws/credentials/static_provider.go

@@ -4,6 +4,9 @@ import (
 	"github.com/aws/aws-sdk-go/aws/awserr"
 )
 
+// StaticProviderName provides a name of Static provider
+const StaticProviderName = "StaticProvider"
+
 var (
 	// ErrStaticCredentialsEmpty is emitted when static credentials are empty.
 	//
@@ -30,9 +33,10 @@ func NewStaticCredentials(id, secret, token string) *Credentials {
 // Retrieve returns the credentials or error if the credentials are invalid.
 func (s *StaticProvider) Retrieve() (Value, error) {
 	if s.AccessKeyID == "" || s.SecretAccessKey == "" {
-		return Value{}, ErrStaticCredentialsEmpty
+		return Value{ProviderName: StaticProviderName}, ErrStaticCredentialsEmpty
 	}
 
+	s.Value.ProviderName = StaticProviderName
 	return s.Value, nil
 }
 

aws/credentials/stscreds/assume_role_provider.go

@@ -14,6 +14,9 @@ import (
 	"github.com/aws/aws-sdk-go/service/sts"
 )
 
+// ProviderName provides a name of AssumeRole provider
+const ProviderName = "AssumeRoleProvider"
+
 // AssumeRoler represents the minimal subset of the STS client API used by this provider.
 type AssumeRoler interface {
 	AssumeRole(input *sts.AssumeRoleInput) (*sts.AssumeRoleOutput, error)
@@ -116,7 +119,7 @@ func (p *AssumeRoleProvider) Retrieve() (credentials.Value, error) {
 	})
 
 	if err != nil {
-		return credentials.Value{}, err
+		return credentials.Value{ProviderName: ProviderName}, err
 	}
 
 	// We will proactively generate new credentials before they expire.
@@ -126,5 +129,6 @@ func (p *AssumeRoleProvider) Retrieve() (credentials.Value, error) {
 		AccessKeyID:     *roleOutput.Credentials.AccessKeyId,
 		SecretAccessKey: *roleOutput.Credentials.SecretAccessKey,
 		SessionToken:    *roleOutput.Credentials.SessionToken,
+		ProviderName:    ProviderName,
 	}, nil
 }

aws/request/request.go

@@ -22,20 +22,22 @@ type Request struct {
 	Handlers   Handlers
 
 	Retryer
-	Time         time.Time
-	ExpireTime   time.Duration
-	Operation    *Operation
-	HTTPRequest  *http.Request
-	HTTPResponse *http.Response
-	Body         io.ReadSeeker
-	BodyStart    int64 // offset from beginning of Body that the request body starts
-	Params       interface{}
-	Error        error
-	Data         interface{}
-	RequestID    string
-	RetryCount   int
-	Retryable    *bool
-	RetryDelay   time.Duration
+	Time             time.Time
+	ExpireTime       time.Duration
+	Operation        *Operation
+	HTTPRequest      *http.Request
+	HTTPResponse     *http.Response
+	Body             io.ReadSeeker
+	BodyStart        int64 // offset from beginning of Body that the request body starts
+	Params           interface{}
+	Error            error
+	Data             interface{}
+	RequestID        string
+	RetryCount       int
+	Retryable        *bool
+	RetryDelay       time.Duration
+	NotHoist         bool
+	SignedHeaderVals http.Header
 
 	built bool
 }
@@ -137,13 +139,26 @@ func (r *Request) SetReaderBody(reader io.ReadSeeker) {
 // if the signing fails.
 func (r *Request) Presign(expireTime time.Duration) (string, error) {
 	r.ExpireTime = expireTime
+	r.NotHoist = false
 	r.Sign()
 	if r.Error != nil {
 		return "", r.Error
 	}
 	return r.HTTPRequest.URL.String(), nil
 }
 
+// PresignRequest behaves just like presign, but hoists all headers and signs them.
+// Also returns the signed hash back to the user
+func (r *Request) PresignRequest(expireTime time.Duration) (string, http.Header, error) {
+	r.ExpireTime = expireTime
+	r.NotHoist = true
+	r.Sign()
+	if r.Error != nil {
+		return "", nil, r.Error
+	}
+	return r.HTTPRequest.URL.String(), r.SignedHeaderVals, nil
+}
+
 func debugLogReqError(r *Request, stage string, retrying bool, err error) {
 	if !r.Config.LogLevel.Matches(aws.LogDebugWithRequestErrors) {
 		return

aws/version.go

@@ -5,4 +5,4 @@ package aws
 const SDKName = "aws-sdk-go"
 
 // SDKVersion is the version of this SDK
-const SDKVersion = "1.0.11"
+const SDKVersion = "1.1.0"