Bring e2e in *for real* left-overs

[evgeniiaraz]

What's left to do? Reading in e2e data

• [x] Bring back YAML reading for end-to-end data

Research

• [x] Store in trackers whether intent / text were used to predict next action
• [x] In prediction, get the tracker history up to the current state depending on what action was predicted from (get the intent/text in corresponding steps)
• [x] figure out the condition when TED should use text or intent
• [x] TEDPolicy vs RulesPolicy: figure out what to do when TED uses text input, but rules predict an action based on intent
• [x] TEDPolicy vs RulesPolicy: what to do with nlu_fallback
• [x] create and test on edge cases
• [x] Make some mixed test stories to check it works (they can be Sara based)
• [x] 4D Tensors
• [x] Add utterance transformer to TED;
• [x] Ability to exclude the output of an NLU featurizer from being used for training and inference
• [x] increase test coverage of state and tracker featurizers
• [x] bring entities into TED https://github.com/RasaHQ/research/issues/121
• [x] remove fake features, and use 4d-3d converter instead
• [x] check if training entities in DIET is still working check if training entities in DIET is still working on e2e #7289
• [x] nuke from orbit entity labels from text as user input
• [x] pass entities extracted by ted to tracker Add entities to UserUttered event if they are predicted via a policy #7443
• [x] create e2ebot example that makes sense
• [x] docs!!!!

Engineering

• [x] Bot utterances E2E actions should result in BotUtterances #6410
• [x] policies return events Next steps End-to-end: Implementation Proposal + Implementation #7039
• [x] Check what model/components needs to be trained when rasa train is executed. Make sure to cover all scenarios: no NLU data, but NLU pipeline configured; no data at all; NLU data, but no NLU pipeline; etc. add warning/error if rasa train core is called on stories with e2e data make sure that rasa train commands run gracefully with e2e #7129
• [x] Make sure the stories are printed correctly Make sure the e2e stories are printed correctly #7130

Post release:

• [x] fix Story structure validation https://github.com/RasaHQ/research/issues/114
• Test the system
• [x] factor out combining sparse/dense and sequence/sentence features into re-usable layer components Turn feature combining in TED/DIET into re-usable layers #7187
• refactor tf helper methods of ted and diet
• implement test entities extracted by TED, e.g. via rasa test. Enable rasa test usage for e2e #7578
• [x] "auto" fill slots by entities extracted by TED
• [x] use bilou tags in TED implement BILOU tags in core featurizers #7579
• predict entity roles/group in TED predict entity roles/groups in TED  #7580
• create "smart" retraining heuristics change the logic of retraining models #7581
• fix rasa test rasa test skips stories that rasa test core does not skip #7677
• [x] Add split_entities_config to TEDPolicy Add split_entities_config to TEDPolicy for e2e #7707
• Warn users if their tests contain bot utterances not found in training data Warn users if their tests contain bot utterances not found in training data #8074

[wochinge]

Exalate commented:

wochinge commented:

@Ghostvv I guess bot utterances have the highest priority as a next step?

[Ghostvv]

Exalate commented:

Ghostvv commented:

yes, then, we can actually run e2e bots

[joejuzl]

Exalate commented:

joejuzl commented:

Added this to 2.2 milestone, I think this issue provides enough of an overview that we don't need the e2e milestone.

[m-vdb]

Closing as won't do for now