Include all options in enum error messages

[binste]

Fixes what @mattijn found in #2949 (comment). So far, for the error messages only the errors at the same leaf level of the "error tree" where considered. For errors which show that a value is not part of an enum, it seems to make sense to find and list all enums in leaves.

[mattijn]

Thanks @binste! Now the following returns the right error suggestions:

import altair as alt
x_tu = alt.X().timeUnit('yearmonthdatehour')
alt.Chart().encode(x_tu)

SchemaValidationError: 'yearmonthdatehour' is an invalid value for `timeUnit`:

'yearmonthdatehour' is not one of ['year', 'quarter', 'month', 'week', 'day', 'dayofyear', 'date', 'hours', 'minutes', 'seconds', 'milliseconds']
'yearmonthdatehour' is not one of ['utcyear', 'utcquarter', 'utcmonth', 'utcweek', 'utcday', 'utcdayofyear', 'utcdate', 'utchours', 'utcminutes', 'utcseconds', 'utcmilliseconds']
'yearmonthdatehour' is not one of ['yearquarter', 'yearquartermonth', 'yearmonth', 'yearmonthdate', 'yearmonthdatehours', 'yearmonthdatehoursminutes', 'yearmonthdatehoursminutesseconds', 'yearweek', 'yearweekday', 'yearweekdayhours', 'yearweekdayhoursminutes', 'yearweekdayhoursminutesseconds', 'yeardayofyear', 'quartermonth', 'monthdate', 'monthdatehours', 'monthdatehoursminutes', 'monthdatehoursminutesseconds', 'weekday', 'weeksdayhours', 'weekdayhoursminutes', 'weekdayhoursminutesseconds', 'dayhours', 'dayhoursminutes', 'dayhoursminutesseconds', 'hoursminutes', 'hoursminutesseconds', 'minutesseconds', 'secondsmilliseconds']
'yearmonthdatehour' is not one of ['utcyearquarter', 'utcyearquartermonth', 'utcyearmonth', 'utcyearmonthdate', 'utcyearmonthdatehours', 'utcyearmonthdatehoursminutes', 'utcyearmonthdatehoursminutesseconds', 'utcyearweek', 'utcyearweekday', 'utcyearweekdayhours', 'utcyearweekdayhoursminutes', 'utcyearweekdayhoursminutesseconds', 'utcyeardayofyear', 'utcquartermonth', 'utcmonthdate', 'utcmonthdatehours', 'utcmonthdatehoursminutes', 'utcmonthdatehoursminutesseconds', 'utcweekday', 'utcweeksdayhours', 'utcweekdayhoursminutes', 'utcweekdayhoursminutesseconds', 'utcdayhours', 'utcdayhoursminutes', 'utcdayhoursminutesseconds', 'utchoursminutes', 'utchoursminutesseconds', 'utcminutesseconds', 'utcsecondsmilliseconds']

One question, maybe you now this directly, why isn't the following directly returning an error:

x_tu

X({
  timeUnit: 'yearmonthdatehour'
})

Is it because the encoding channel properties are only inferred based on the encoding channel in which it is defined?
So an encoding channel property can not be tested against the jsonschema without a corresponding encoding channel?

[binste]

The main part where the schema is validated is when .to_dict is called, see https://github.com/altair-viz/altair/blob/master/altair/utils/schemapi.py#L551. When you simply print x_tu which still represents the X encoding channel, not the timeUnit property, this does not happen but it happens when you display a chart as the dictionary is used in the renderer.

Some objects are already being validated when there instantiated by calling to_dict in the __init__ method, see https://github.com/altair-viz/altair/blob/master/altair/utils/schemapi.py#L367. For encoding channels this is deactivated as it requires the context of the chart to infer some attributes. I think expanding shorthands is such a case.

alt.TimeUnit does not depend on its context and is therefore valid at instantiation and gets validated:

alt.TimeUnit is never actually constructed when calling X.timeUnit and it therefore does not raise an error. I think that's fine as it would also not be constructed when doing alt.X(timeUnit="...").

[mattijn]

Thanks for digging deep! So we are seeing the behavior introduced by #546. More clear now.

In line with your comment here, #2850 (comment), maybe better to alias DEBUG_MODE with VALIDATE_ON_INIT and deprecate the former. Will open another issue for this.