Skip to content
This repository has been archived by the owner on Aug 23, 2021. It is now read-only.

Remove deepmerge dependency #56

Merged
merged 1 commit into from
Sep 24, 2020

Conversation

akutz
Copy link
Contributor

@akutz akutz commented Sep 22, 2020

Overview

This patch removes the deepmerge module as a required dependency of this cloud-init datasource. There is still an option to use the deepmerge module if the environment variable CLOUD_INIT_VMWARE_GUEST_INFO_MERGE_STRATEGY is set to deepmerge. Otherwise a deep merge is performed with a recursive strategy using Python built-ins.

This does not remove the dependency on the netifaces module.

Testing

The following test cases were executed to assert this works as designed:

Python Version Merge Strategy Command Results
2.7.17 deepmerge python2 DataSourceVMwareGuestInfo.py 1>p2-deepmerge.json (link)
stdlib python2 DataSourceVMwareGuestInfo.py 1>p2-stdlib.json (link)
3.7.3 deepmerge python3 DataSourceVMwareGuestInfo.py 1>p3-deepmerge.json (link)
stdlib python3 DataSourceVMwareGuestInfo.py 1>p3-stdlib.json (link)

Note: The above commands will redirect the merged JSON into the specified file. However, the logger will emit a message to stderr on the console to indicate the selected merge strategy. This log entry will not be part of the contents written to the file. For example:

# stdlib merge strategy
$ python3 DataSourceVMwareGuestInfo.py 1>p3-stdlib.json
2020-09-22 18:43:47,701 - DataSourceVMwareGuestInfo.py[INFO]: merging dictionaries with stdlib strategy

# deepmerge strategy
$ CLOUD_INIT_VMWARE_GUEST_INFO_MERGE_STRATEGY=deepmerge \
  python3 DataSourceVMwareGuestInfo.py 1>p3-deepmerge.json
2020-09-22 18:44:24,219 - DataSourceVMwareGuestInfo.py[INFO]: merging dictionaries with deepmerge strategy

The following command will diff all permutations (I'm not inclined to figure out how to eliminate the repetition), and the result is no differences between any of the above files:

BASE_URL=https://gist.githubusercontent.com/akutz/2bc95ef7887b653ddeed4c1bd5b2de14/raw/bac0403b9675dfa86679adaf240335fda63bbac2 && \
for AB in {2..3}{a..b}-{2..3}{a..b}; do \
  A="${AB%-*}"; B="${AB#*-}"; \
  [ "${A}" != "${B}" ] || continue; \
  [ "${A}" == "2a" ] && f1="p2-deepmerge.json"; \
  [ "${A}" == "2b" ] && f1="p2-stdlib.json"; \
  [ "${A}" == "3a" ] && f1="p3-deepmerge.json"; \
  [ "${A}" == "3b" ] && f1="p3-stdlib.json"; \
  [ "${B}" == "2a" ] && f2="p2-deepmerge.json"; \
  [ "${B}" == "2b" ] && f2="p2-stdlib.json"; \
  [ "${B}" == "3a" ] && f2="p3-deepmerge.json"; \
  [ "${B}" == "3b" ] && f2="p3-stdlib.json"; \
  echo "diff ${f1} ${f2}"; \
  diff <(curl -sSL "${BASE_URL}/${f1}") <(curl -sSL "${BASE_URL}/${f2}"); \
done

Does anyone care to do some additional testing on this? Thank you!

cc @yvespp @powersj @sshedi

This patch removes the deepmerge module as a required dependency
of this cloud-init datasource. There is still an option to use
the deepmerge module if the environment variable
CLOUD_INIT_VMWARE_GUEST_INFO_MERGE_STRATEGY is set to 'deepmerge'.
Otherwise a deep merge is performed with a recursive strategy using
Python built-ins.
@yvespp
Copy link
Contributor

yvespp commented Sep 24, 2020

@akutz it would be good to write a unit test that covers the cases here: https://github.com/vmware/cloud-init-vmware-guestinfo/pull/56/files#diff-887b9ab215181ea8c8772e608c0408fdL165
Or you could but your manual test into a unit test.

@yvespp
Copy link
Contributor

yvespp commented Sep 24, 2020

I tested it with Ubuntu 20.04 and everything seems to be working correctly.
I made sure deepmerge was not installed on the system.

return always_merger.merge(a, b)


def merge_dicts_with_stdlib(a, b):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use meaningful names instead of a, b. (applies throughout)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sshedi,

Sure, with a small function though, a and b are often used as the variables for two operands, such as src and dst.

return merge_dicts_with_deep_merge(a, b)
except Exception as err:
LOG.error("deep merge failed: %s" % err)
LOG.info('merging dictionaries with stdlib strategy')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this stdlib strategy? IMO it should be recursive strategy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it has no dependencies other than Python stdlib.

@sshedi
Copy link
Contributor

sshedi commented Sep 24, 2020

@akutz I tried this in my dev environment, LGTM.

@akutz akutz merged commit d4e9962 into vmware-archive:master Sep 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants