Add TMDS initialization functionality to ecs-agent module

[amogh09]

Summary

Add TMDS initialization functionality to ecs-agent module. The server read/write timeout, request rate limits, listen address, and router need to be provided by the consumer. The server is configured with a request rate limiter and logging middleware just like the TMDS in agent module. agent module will be updated in a separate PR to consume this new functionality for initializing TMDS.

Also making integration test make targets run integration tests of ecs-agent module.

Implementation details

• Add a new tmds package to ecs-agent module and add NewServer() function to it. The function takes an instance of AuditLogger and an arbitrary number of configuration options and initializes a net/http server for TMDS. Configuration options must include a listen address and a router for the server.
• The NewServer() function creates request rate limiter and logger middlewares and adds them to the server it initializes and returns the server.
• Update make run-integ-tests target to run integration tests for ecs-agent module too.

Testing

Not a part of this PR but I replaced the current TMDS initialization in agent module with the new TMDS initialization from ecs-agent module and built Agent from source. I then sent some requests to TMDS which were successful. More comprehensive tests will be performed for the next CR that will include changes I used for the testing.

Added integration tests for server write timeout and rate limiter.

New tests cover the changes: yes

Description for the changelog

Add TMDS initialization functionality to ecs-agent module

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[amogh09]

Server setup here is taken from agent module

amazon-ecs-agent/agent/handlers/task_server_setup.go

Lines 81 to 100 in 0202ad2

	limiter := tollbooth.NewLimiter(float64(steadyStateRate), nil)
	limiter.SetOnLimitReached(handlersutils.LimitReachedHandler(auditLogger))
	limiter.SetBurst(burstRate)
	// Log all requests and then pass through to muxRouter.
	loggingMuxRouter := mux.NewRouter()
	// rootPath is a path for any traffic to this endpoint, "root" mux name will not be used.
	rootPath := "/" + handlersutils.ConstructMuxVar("root", handlersutils.AnythingRegEx)
	loggingMuxRouter.Handle(rootPath, tollbooth.LimitHandler(
	limiter, NewLoggingHandler(muxRouter)))
	loggingMuxRouter.SkipClean(false)
	server := http.Server{
	Addr: "127.0.0.1:" + strconv.Itoa(config.AgentCredentialsPort),
	Handler: loggingMuxRouter,
	ReadTimeout: readTimeout,
	WriteTimeout: writeTimeout,
	}

[amogh09]

This handler is taken from agent module

amazon-ecs-agent/agent/handlers/utils/helpers.go

Lines 128 to 135 in 0202ad2

	func LimitReachedHandler(auditLogger audit.AuditLogger) func(http.ResponseWriter, *http.Request) {
	return func(w http.ResponseWriter, r *http.Request) {
	logRequest := request.LogRequest{
	Request: r,
	}
	auditLogger.Log(logRequest, http.StatusTooManyRequests, "")
	}
	}

[amogh09]

This is taken from agent module.

amazon-ecs-agent/agent/handlers/logging_handler.go

Lines 22 to 34 in 0202ad2

	// LoggingHandler is used to log all requests for an endpoint.
	type LoggingHandler struct{ h http.Handler }
	// NewLoggingHandler creates a new LoggingHandler object.
	func NewLoggingHandler(handler http.Handler) LoggingHandler {
	return LoggingHandler{h: handler}
	}
	// ServeHTTP logs the method and remote address of the request.
	func (lh LoggingHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	seelog.Debug("Handling http request", "method", r.Method, "from", r.RemoteAddr)
	lh.h.ServeHTTP(w, r)
	}

[amogh09]

You can see how this function will be consumed by agent module in the draft PR below.

https://github.com/aws/amazon-ecs-agent/pull/3661/files#diff-c3f3c7cbd08c019cffa42ad0328b4dbf7b9e94567c8f27e4a5580c30ca7c97f9L79-L102

[prateekchaudhry]

nit/q: probably not very significant here.. should this sleep be before Get call? I think we may always want to sleep at least once, and the last sleep (i==9) is redundant if we put the sleep here.

[amogh09]

Good point. I will move the sleep to before the HTTP call!

[amogh09]

Updated in d7da34f