Externalized Configuration ignores null values in SPRING_APPLICATION_JSON

[rgordeev]

E.g. there is a configuration application.yaml

application:
  a: some string
  b:
    - 1
    - 2

I try to override these properties with

SPRING_APPLICATION_JSON={"application": {"a": null, "b": null}}

And it doesn't override values from yaml file, but when values in json are not null everything works fine.

[philwebb]

Here's a small test that reproduces the issue:

@Test
void nulls() {
	TestPropertySourceUtils.addInlinedPropertiesToEnvironment(this.environment, "foo=bar");
	assertThat(this.environment.resolvePlaceholders("${foo}")).isEqualTo("bar");
	TestPropertySourceUtils.addInlinedPropertiesToEnvironment(this.environment,
			"SPRING_APPLICATION_JSON={\"foo\":null}");
	this.processor.postProcessEnvironment(this.environment, null);
	assertThat(this.environment.resolvePlaceholders("${foo}")).isNull();
}

Debugging things it appears that we SpringApplicationJsonEnvironmentPostProcessor does actually insert a null entry in the appropriate place.

The problem is that PropertySourcesPropertyResolver does not use containsValue and always skips over nulls. I'll open an issue with the Spring Framework team to discuss options.

[philwebb]

I've raised spring-projects/spring-framework#25142. Please track that issue to see what the framework team have to say. For now, I suggest using an empty string instead of null.

[rgordeev]

@philwebb

For now, I suggest using an empty string instead of null.

But the problem concerns not only null values for strings, but empty values for lists or maps - it's impossible to override values from application.yaml with SPRING_APPLICATION_JSON, that contains empty values or nulls.
E.g.

application:
  a: some string
  b:
    - 1
    - 2

If I try to override b property with

SPRING_APPLICATION_JSON={"application": {"b": null}}

or

SPRING_APPLICATION_JSON={"application": {"b": {}}}

or

SPRING_APPLICATION_JSON={"application": {"b": []}}

nothing happens, application will get

application:
  a: some string
  b:
    - 1
    - 2

in every mentioned case.

[philwebb]

Reopening to consider spring-projects/spring-framework#25142 (comment)

[philwebb]

@rgordeev Does SPRING_APPLICATION_JSON={"application": {"b": ""}} work?

[rgordeev]

@philwebb Yes, that works for all types of properties, but it not obvious at all. You may see test cases
tests

Test case Defaults

application-test.yaml

application:
  a: "some string"
  b:
    - 1
    - 2
    - 3
  c:
    - "one"
    - "two"
    - "three"
  m:
    - one: 1
    - two: 2

Expecting                    Actual
 a == ""                     a == ""
 b == [1,2,3]                b == [1,2,3]
 c == ["one","two","three"]  c == ["one","two","three"]
 m == {"one":1, "two":2}     m == {"one":1, "two":2}

Test case With Empty Lists

application-test.yaml

application:
  a: "some string"
  b:
    - 1
    - 2
    - 3
  c:
    - "one"
    - "two"
    - "three"
  m:
    - one: 1
    - two: 2

We are trying to override

• string property with null,
• list properties with empty json array [],
• map with empty json object {}

SPRING_APPLICATION_JSON = {
    "application": {
        "a": null,
        "b": [],
        "c": [],
        "m": {}
    }
}

Expecting                            Actual
 a == null                           a == "some string"
 b == null | empty array list        b == [1,2,3]
 c == null | empty array list        c == ["one","two","three"]
 m == null | empty map               m == {"one":1, "two":2}

Test case With Empty Objects

application-test.yaml

application:
  a: "some string"
  b:
    - 1
    - 2
    - 3
  c:
    - "one"
    - "two"
    - "three"
  m:
    - one: 1
    - two: 2

We are trying to override

• string property with null,
• list properties with empty object {},
• map with empty json object {}

SPRING_APPLICATION_JSON = {
    "application": {
        "a": null,
        "b": {},
        "c": {},
        "m": {}
    }
}

Expecting                            Actual
 a == null                           a == "some string"
 b == null | empty array list        b == [1,2,3]
 c == null | empty array list        c == ["one","two","three"]
 m == null | empty map               m == {"one":1, "two":2}

Test case With Empty Strings

application-test.yaml

application:
  a: "some string"
  b:
    - 1
    - 2
    - 3
  c:
    - "one"
    - "two"
    - "three"
  m:
    - one: 1
    - two: 2

We are trying to override

• string property with empty string "",
• list properties with empty string "",
• map properties with empty string ""

SPRING_APPLICATION_JSON = {
    "application": {
        "a": "",
        "b": "",
        "c": "",
        "m": ""
    }
}

Expecting                            Actual
 a == ""                             a == ""
 b == null | empty array list        b == []
 c == null | empty array list        c == []
 m == null | empty map               m == {}

Test case With Non null Values

application-test.yaml

application:
  a: "some string"
  b:
    - 1
    - 2
    - 3
  c:
    - "one"
    - "two"
    - "three"
  m:
    - one: 1
    - two: 2

We are trying to override

• string property with empty string "",
• list properties with non empty json arrays [9, 10] and ["ten", "nine"],
• map properties with non empty json object {"nine": 9}

SPRING_APPLICATION_JSON = {
    "application": {
        "a": "",
        "b": [9,10],
        "c": ["ten", "nine"],
        "m": {"nine": 9}
    }
}

Expecting                    Actual
 a == ""                     a == ""
 b == [9,10]                 b == [9,10]
 c == ["ten","nine"]         c == ["ten","nine"]
 m == {"nine": 9}            m == {"one":1, "two":2, "nine":9}

Test case With Null values

application-test.yaml

application:
  a: "some string"
  b:
    - 1
    - 2
    - 3
  c:
    - "one"
    - "two"
    - "three"
  m:
    - one: 1
    - two: 2

We are trying to override

• string property with null,
• list properties with null,
• map properties with null

SPRING_APPLICATION_JSON = {
    "application": {
        "a": null,
        "b": null,
        "c": null,
        "m": null
    }
}

Expecting                    Actual
 a == ""                     a == "some string"
 b == null | empty           b == [1,2,3]
 c == null | empty           c == ["one","two","three"]
 m == null | empty           m == {"one":1, "two":2}

[mbhave]

I think this is because of the way flatten works. If the collection is empty, it skips adding the value. This looks like a bug to me. Let's see what the rest of the team thinks.