[App] Increased DeepDiff's verbose level to properly handle dict changes

[adam-lightning]

What does this PR do?

Problem

While building a streamlit UI we found that if we do the following:

state.my_dict["some_key"] = "some_value"

And later trigger the status update the App will fail with something of the following error:

  File "/opt/miniconda3/envs/lai-pip/lib/python3.8/site-packages/lightning/core/lightning_app.py", line 306, in maybe_apply_changes
    state += delta
  File "/opt/miniconda3/envs/lai-pip/lib/python3.8/site-packages/deepdiff/delta.py", line 131, in __add__
    self._do_dictionary_item_added()
  File "/opt/miniconda3/envs/lai-pip/lib/python3.8/site-packages/deepdiff/delta.py", line 276, in _do_dictionary_item_added
    self._do_item_added(dictionary_item_added, sort=False)
  File "/opt/miniconda3/envs/lai-pip/lib/python3.8/site-packages/deepdiff/delta.py", line 297, in _do_item_added
    items = items.items()
AttributeError: 'list' object has no attribute 'items'

Solution

It turns out that DeepDiff on current verbose_level = 1 (default) while creating diff only records what keys were added or changed, and not recording the new value. This diff is not applicable to the state with += operation (because it's missing a new value to set) so it fails with the error above. Increasing verbose_level=2 now includes the new item value -> solves the issue.

Disclaimer: We know that just setting state.my_dict["some_key"] = "some_value" does not trigger a state update by itself (the value will not be visible outside). We need other assignment operation to do that (any use of = on state variable would do that). This PR does not address this issue at all.

Does your PR introduce any breaking changes? If yes, please list them.

Nothing I know of. But would be great if someone chimes in and verifies that changing verbose_level in all places is actually safe ;)

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

cc @Borda

[lantiga]

Thank you @adam-lightning, this is indeed the intended use of verbose_level (https://zepworks.com/deepdiff/current/diff.html) so I think it's safe to do this.

Can you please open a separate issue related to the assignment?

[manskx]

Disclaimer: We know that just setting state.my_dict["some_key"] = "some_value" does not trigger a state update by itself (the value will not be visible outside). We need other assignment operation to do that (any use of = on state variable would do that). This PR does not address this issue at all.

This issue is been partially solved in https://github.com/Lightning-AI/lightning/blob/4c35867b618b4c36bfac5428756b95223f7f526a/src/lightning_app/utilities/proxies.py#L173

[manskx]

LGTM

[adam-lightning]

Disclaimer: We know that just setting state.my_dict["some_key"] = "some_value" does not trigger a state update by itself (the value will not be visible outside). We need other assignment operation to do that (any use of = on state variable would do that). This PR does not address this issue at all.

This issue is been partially solved in

https://github.com/Lightning-AI/lightning/blob/4c35867b618b4c36bfac5428756b95223f7f526a/src/lightning_app/utilities/proxies.py#L173

Do you know if anyone have tested that observer with streamlit ? Just curious.

[tchaton]

LGTM !