Code

[WIP] Container Metadata#913

Code

Closed

huerdong wants to merge 106 commits intoaws:devaws/amazon-ecs-agent:devfrom huerdong:mdservicehuerdong/amazon-ecs-agent:mdserviceCopy head branch name to clipboard

+2,596-98

Commits

Commits on Aug 9, 2017

• Added volume management support in engine and initial metadata service files

  

  huerdong

  committed
  9b1404c

  Copy full SHA for 9b1404c

• Fixed volume configuration to mount a host directory.

  

  huerdong

  committed
  1ab2131

  Copy full SHA for 1ab2131

• Fixed write metadata.json issues.

  

  huerdong

  committed
  4202cac

  Copy full SHA for 4202cac

• Refactored metadataservice and removed unnecessary functions. Added temporary fix to metadata file cleanup

  

  huerdong

  committed
  822fef7

  Copy full SHA for 822fef7

• Added metadata file cleanup at container stop

  

  huerdong

  committed
  33cdfb5

  Copy full SHA for 33cdfb5

• Implemented metadata file updating. Made additional refactoring to support these updates

  

  huerdong

  committed
  c0d38f8

  Copy full SHA for c0d38f8

• Made json format easier to read

  

  huerdong

  committed
  9965df4

  Copy full SHA for 9965df4

• Fixed issue of metadata cleanup not occurring when containers stop naturally

  

  huerdong

  committed
  448f24f

  Copy full SHA for 448f24f

• Fixed issue of metadata cleanup not occurring when containers stop naturally

  

  huerdong

  committed
  f48574b

  Copy full SHA for f48574b

• Added comments to metadata service and made additional refactoring

  

  huerdong

  committed
  0ca09f7

  Copy full SHA for 0ca09f7

• Removed unncessary elements in DockerContainerMetadata

  

  huerdong

  committed
  d3dcb00

  Copy full SHA for d3dcb00

• Refactored docker_task_engine.go and metadataservice/types.go

  

  huerdong

  committed
  a793b83

  Copy full SHA for a793b83

• Resolved merge conflicts

  

  huerdong

  committed
  b4da96f

  Copy full SHA for b4da96f

• Improved code style and refactored code to bring constants to configuration

  

  huerdong

  committed
  9694b09

  Copy full SHA for 9694b09

• Fixed engine tests and nil pointer edge cases in metadataservice/acquire_metadata.go

  

  huerdong

  committed
  7cd00d2

  Copy full SHA for 7cd00d2

• Added networking data to file

  

  huerdong

  committed
  ba29690

  Copy full SHA for ba29690

• Revert "Added networking data to file"

  Show description for 998676c
  

  huerdong

  committed
  998676c

  Copy full SHA for 998676c

• Fixed formatting issues and temporarily removed networking features in metadata service

  

  huerdong

  committed
  dcb02c7

  Copy full SHA for dcb02c7

• Fixed most style issues but there may be some remaining. Added preliminary network metadata

  

  huerdong

  committed
  cfb2286

  Copy full SHA for cfb2286

• Made one variable name more descriptive

  

  huerdong

  committed
  1dda955

  Copy full SHA for 1dda955

• Refactored containermetadta and docker_task_engine.go to make function calls more modular

  

  huerdong

  committed
  994de96

  Copy full SHA for 994de96

• Fixed code style errors from refactoring

  

  huerdong

  committed
  2650223

  Copy full SHA for 2650223

• Added preliminary support for version and instance arn.

  

  huerdong

  committed
  079e8b8

  Copy full SHA for 079e8b8

• Fixed naming conflicts, style problems, and extraneous code

  

  huerdong

  committed
  aaae37c

  Copy full SHA for aaae37c

• Eliminated redundant struct and fixed more style issues

  

  huerdong

  committed
  abc5823

  Copy full SHA for abc5823

• Removed unncessary functions and added additional comments to add clarity

  

  huerdong

  committed
  133b720

  Copy full SHA for 133b720

• Fixed broken engine tests

  

  huerdong

  committed
  65effe9

  Copy full SHA for 65effe9

• Fix for AppVeyor tests

  

  huerdong

  committed
  683f28e

  Copy full SHA for 683f28e

• Test if windows test TestBatchContainerHappyPath continues to break

  

  huerdong

  committed
  183e070

  Copy full SHA for 183e070

• Test if removing waiting for version call fixes anything

  

  huerdong

  committed
  2a7e89e

  Copy full SHA for 2a7e89e

• Revert "Test if removing waiting for version call fixes anything"

  Show description for c85b915
  

  huerdong

  committed
  c85b915

  Copy full SHA for c85b915

• Test what happens when busy wait for version is removed

  

  huerdong

  committed
  1c0c2a2

  Copy full SHA for 1c0c2a2

• Handled edge case in UpdateMetadata when metadata file does not exist

  

  huerdong

  committed
  6cfbe98

  Copy full SHA for 6cfbe98

• Removed MarshalJSON functions

  

  huerdong

  committed
  f5471d1

  Copy full SHA for f5471d1

• Investigation of whether UpdateTask is breaking tests

  

  huerdong

  committed
  b108bb3

  Copy full SHA for b108bb3

• Investigate if client.Version() is breaking tests

  

  huerdong

  committed
  95dcc4b

  Copy full SHA for 95dcc4b

• Fixed build issues of previous commit

  

  huerdong

  committed
  a465fac

  Copy full SHA for a465fac

• Fixed build issues

  

  huerdong

  committed
  2f97d97

  Copy full SHA for 2f97d97

• Added back docker version to check if that is breaking unit tests

  

  huerdong

  committed
  28acc3c

  Copy full SHA for 28acc3c

• Added back UpdateMetadata and removed docker version to see if that breaks

  

  huerdong

  committed
  8b1f083

  Copy full SHA for 8b1f083

• Added back version. Will investigate adding conditions in unit tests to verify that file has been created

  

  huerdong

  committed
  6d2a6d5

  Copy full SHA for 6d2a6d5

• Made fixes for most engine tests

  

  huerdong

  committed
  3622132

  Copy full SHA for 3622132

• Refactored containermetadata to allow gomocks, created gomocks for containermetadata, and fixed unit tests

  

  huerdong

  committed
  b11dcf7

  Copy full SHA for b11dcf7

• Fixed engine integration test

  

  huerdong

  committed
  7c4b698

  Copy full SHA for 7c4b698

• Added check to ensure update metadata does not occur if container is stopped and made updates serial with start container (rather than parallel)

  

  huerdong

  committed
  548046d

  Copy full SHA for 548046d

• Made UpdateMetadata call serial rather than parallel

  

  huerdong

  committed
  aa8b6cb

  Copy full SHA for aa8b6cb

• Test if removing waiting for version call fixes anything

  

  huerdong

  committed
  e6e4d07

  Copy full SHA for e6e4d07

• acquire_metadata.go

  

  huerdong

  committed
  e5a630f

  Copy full SHA for e5a630f

• Removed unused MarshalJSON function

  

  huerdong

  committed
  72247f8

  Copy full SHA for 72247f8

• Changed most of write_metadata.go to use filepath.Join to be more operating system agnostic

  

  huerdong

  committed
  d9379e0

  Copy full SHA for d9379e0

• Made preliminary changes to enable container metadata in windows agent

  

  huerdong

  committed
  d383c2e

  Copy full SHA for d383c2e

• Made changes based on code review

  

  huerdong

  committed
  88e93b0

  Copy full SHA for 88e93b0

• Removed unused constant

  

  huerdong

  committed
  436dea5

  Copy full SHA for 436dea5

• Fixed TestSteadyStatePoll missing CleanMetadata mock call

  

  huerdong

  committed
  182e2c0

  Copy full SHA for 182e2c0

• Undid windows changes

  

  huerdong

  committed
  f94bba5

  Copy full SHA for f94bba5

• Fixed go formatting

  

  huerdong

  committed
  b40e8c4

  Copy full SHA for b40e8c4

• Fixed flakiness in TestSteadyStatePoll

  

  huerdong

  committed
  cbe03e4

  Copy full SHA for cbe03e4

• Made changes based on code review. Added more logs and error handling

  

  huerdong

  committed
  4f69e1c

  Copy full SHA for 4f69e1c

• Made windows adjustments. May need to modify mount path or DataDirOnHost

  

  huerdong

  committed
  8625c1a

  Copy full SHA for 8625c1a

• Added operating system build tags for metadata_manager

  

  huerdong

  committed
  d49047c

  Copy full SHA for d49047c

• Check if non-formatted strings fixes the triple escpae issue

  

  huerdong

  committed
  83614fa

  Copy full SHA for 83614fa

• Check if redundant file path is issue

  

  huerdong

  committed
  b74c04a

  Copy full SHA for b74c04a

• Populated PortBindings with correct source. Fixed omitempty typos

  

  huerdong

  committed
  66a58a5

  Copy full SHA for 66a58a5

• Added new config variable to enable/disable container metadata

  

  huerdong

  committed
  06d1a90

  Copy full SHA for 06d1a90

• Added enable/disable for container metadata and fixed client version issue for container metadata

  

  huerdong

  committed
  3474d82

  Copy full SHA for 3474d82

• Changed version for container metadata to 1.21 rather than latest

  

  huerdong

  committed
  af909f4

  Copy full SHA for af909f4

• Made review changes. Removed unnecessary checks and streamlined logging and error checking

  

  huerdong

  committed
  e19b477

  Copy full SHA for e19b477

• Fixed engine tests

  

  huerdong

  committed
  38c782f

  Copy full SHA for 38c782f

• Added Docker client version and Container Instance Arn to set of metadata. Refactored containermetadata package

  

  huerdong

  committed
  02fa6b4

  Copy full SHA for 02fa6b4

• Made review changes. Revised set of metadata available and added time stamps to metadata file

  

  huerdong

  committed
  7c36f62

  Copy full SHA for 7c36f62

• Made massive set of review changes. Gave metadata manager its own client of appropriate version rather than recycling original client.

  

  huerdong

  committed
  90dfa3c

  Copy full SHA for 90dfa3c

• Moved containermetadata package outside engine package and made further review changes

  

  huerdong

  committed
  ee1c67f

  Copy full SHA for ee1c67f

• Fixed build problem because containermetadata package was moved

  

  huerdong

  committed
  00ab68f

  Copy full SHA for 00ab68f

• Fixed broken agent/app and agent/engine tests for Linux. Possibly still broken in Windows

  

  huerdong

  committed
  74e0e29

  Copy full SHA for 74e0e29

• Made additonal changes based on review comments. Removed unused logs and warnings. Clarified container name field. Added config field for DataDirOnHost but should not be used

  

  huerdong

  committed
  a38895c

  Copy full SHA for a38895c

• Most pull request review comments should be addressed. Need to change network metadata to account for multiple interfaces

  

  huerdong

  committed
  b671319

  Copy full SHA for b671319

• Fixed windows build issue

  

  huerdong

  committed
  682222f

  Copy full SHA for 682222f

• Made most review changes. Changed format of metadata parsing and made logging clearer. Added a unit test in engine package

  

  huerdong

  committed
  b7fbcdf

  Copy full SHA for b7fbcdf

• Added environment variable to access metadata file

  

  huerdong

  committed
  78d7847

  Copy full SHA for 78d7847

• Fixed mocks

  

  huerdong

  committed
  5909ee9

  Copy full SHA for 5909ee9

• Fixed windows failing to mount directory because chmod is unsupported. For Linux default md file permission is 0600

  

  huerdong

  committed
  c9ecb97

  Copy full SHA for c9ecb97

• Removed all references to read only permission constant

  

  huerdong

  committed
  3d37f32

  Copy full SHA for 3d37f32

• Fixed a comment case incorrectness

  

  huerdong

  committed
  35b058c

  Copy full SHA for 35b058c

• Made bind mount destination randomized to avoid user mount conflicts. Reverted unit testing for now

  

  huerdong

  committed
  d9f934f

  Copy full SHA for d9f934f

• Changed README to include the new configuration parameters and removed check for non-existent metadata file in update

  

  huerdong

  committed
  0b72345

  Copy full SHA for 0b72345

• Changed file permission to 0644 on Linux and used temporary file writing for windows as well

  

  huerdong

  committed
  19da3f5

  Copy full SHA for 19da3f5

• Fixed windows build error and removed redundant functions

  

  huerdong

  committed
  8dfcda2

  Copy full SHA for 8dfcda2

• Made review changes. Added/fixed metadata fields. Reformatted logging

  

  huerdong

  committed
  4c3c3f8

  Copy full SHA for 4c3c3f8

• Removed redundant error checks

  

  huerdong

  committed
  675d7ef

  Copy full SHA for 675d7ef

• Undid temp file change for windows

  

  huerdong

  committed
  1803430

  Copy full SHA for 1803430

• Fixed build issue with windows

  

  huerdong

  committed
  faa5833

  Copy full SHA for faa5833

• Fixed windows metadata file not exist issue

  

  huerdong

  committed
  7b37076

  Copy full SHA for 7b37076

• Another fix to windows

  

  huerdong

  committed
  e78ac9c

  Copy full SHA for e78ac9c

• Made Update Metadata parallel

  

  huerdong

  committed
  df3bf0f

  Copy full SHA for df3bf0f

• Made changes to remove unnecessary use of pointers

  

  huerdong

  committed
  8418abb

  Copy full SHA for 8418abb

• Made some review changes. Pushing only to perform validation on windows

  

  huerdong

  committed
  e2751fd

  Copy full SHA for e2751fd

• Fixed broken mocks

  

  huerdong

  committed
  81cb6e3

  Copy full SHA for 81cb6e3

• Added unit tests for containermetadata package

  

  huerdong

  committed
  ebc5e6c

  Copy full SHA for ebc5e6c

• Addressed review changes and reverted metadata service to use default version of API. Added tests in config package for new configurations

  

  huerdong

  committed
  72df463

  Copy full SHA for 72df463

• Fixed non-compiling integ test

  

  huerdong

  committed
  b3c0662

  Copy full SHA for b3c0662

Commits on Aug 10, 2017

• Added wrappers for os and ioutil packages for testing. Changed containermetadata unit tests to mock os and ioutil calls

  

  huerdong

  committed
  00663ee

  Copy full SHA for 00663ee

• Increase coverage of containermetadata unit tests to 96.2%

  

  huerdong

  committed
  3a024a4

  Copy full SHA for 3a024a4

• Fixed non-compiling windows tests

  

  huerdong

  committed
  9fc1c76

  Copy full SHA for 9fc1c76

• Fixed TestWriteFileWriteFail

  

  huerdong

  committed
  e8aa107

  Copy full SHA for e8aa107

• Added metadata service enabled unit tests to docker_task_engine_test.go

  

  huerdong

  committed
  5f4cb20

  Copy full SHA for 5f4cb20

• Made description of ENABLE_CONTAINER_METADATA and HOST_DATA_DIR more concise. Added TODO in docker_task_engine.go about metadataManager.Update

  

  huerdong

  committed
  16c9df0

  Copy full SHA for 16c9df0