[Bug] Misleading overriding behavior for "empty" (null) keys

[YodaEmbedding]

🐛 Bug

Description

Input:

# dataset/imagenet.yaml

type: "ImageNetDataset"
path: "/datasets/imagenet"

# example.yaml

defaults:
  - dataset/imagenet
  - _self_

dataset:
  type: "Overriden"

Output (as expected):

dataset:
  type: 'Overriden'
  path: '/datasets/imagenet'

---

However, if we change example.yaml to:

# example.yaml

defaults:
  - dataset/imagenet
  - _self_

dataset:

Output (unexpected, but correct):

dataset: null

The intuitive expected output would have been:

dataset:
  type: 'ImageNetDataset'
  path: '/datasets/imagenet'

Checklist

• I checked on the latest version of Hydra
• I created a minimal repro (See this for tips).

To reproduce

See above.

Expected Behavior

When overriding existing dicts with a single null value, hydra could perhaps produce a warning.

System information

• Hydra Version : 1.3.2
• Python version : 3.11.9
• Virtual environment type and version : user
• Operating system : MacOS

Additional context