Don't report score and duration on interacted

[garemoko]

This content type reports score and duration (and completion with my changes) for both 'answered' and 'interacted'. This means duplicated data (which can confusing reporting), and also it's not really appropriate to report score when the user did not submit the question yet.

I propose not sending the result object with the interacted statement.

[falcon-git]

I don't think it breaks any rules in the spec? We actually need this in our integration with the Drupal Quiz module, so hopefully this is something that can be fixed by LRSs or in the LRS plugin? If the results from interacted events aren't interesting they probably should be ignored? And also they probably should have results.completed = false?

Sending statements to an LRS is just 50 % of the reason we have implemented xAPI support. We use xAPI just as much for communication with other systems and even internally in H5P.

[garemoko]

Certainly this isn't against anything in the spec. The spec doesn't cover content at all. This is something that would make the data less useful and reports inaccurate in Watershed and other tools handling the data in a similar way to Watershed. The issue isn't that they are appearing in the interacted statement, but that they appear multiple times, making it look like this is multiple attempts at the same question.

Now, perhaps we do want to say that every time the learner moves a draggable, that's a new attempt at the question, in which case this is fine. But I don't think that is what most instructional designers would want, especially where the learner has to drag multiple items, it seems harsh to score them at 0 on their first drag.

I don't think it's reasonable to expect the LRS to filter these out. The LRS shouldn't have to have different approaches for different activity providers.

Removing the result in the LRS plugin could possibly work, if 'interacted' is used consistently throughout the H5P ecosystem.

results.completed = false is another interesting option. I'm not aware of any activity providers using this at the moment, but it would imply that further results might be expected.

Can you give some more details on how/why/what is needed in the Drupal quiz module?

[falcon-git]

The drupal quiz module has it's own "native" question types, but also support H5P "question types". The integration listens for xAPI events from H5P and stores the score found in the statement. So an end user might take a native multichoice as question 1, then an H5P Drag and Drop as question 2, then a native true/false as question 3. The user typically won't press H5P's check button on question 2, he will press quiz's next button and the "answered" event will never happen. Since the Drag and Drop runs in an iframe it is complicated for quiz to talk to the drag and drop instance, but it receives it's events.

Generally I think if xAPI practices are optimized for LRS and reporting purposes we will loose a lot. It is better to have as much data as possible in the events, and let the LRSs do the filtering. The sources of the data should only worry about providing as much as possible.

xAPI is still young so if it is moving in a direction where the sole purpose of it is to facilitate reporting and make LRS implementations as simple as possible it is not too late to turn.

[falcon-git]

To be more concrete I can't see why the LRS shouldn't handle statements with different verbs differently. H5P is saying that User interacted with this object, and the score is now x of y. H5P will later say that User answered this object, and the score is now x of y", and if you're creating a report of attempts you probably shouldn't include interacted. But I can see that result.completed would be very useful here instead of using verbs since there might be custom verbs triggered both for complete and incomplete

[garemoko]

I agree with what you're saying in principle and I've raised a Jira ticket for the Watershed developers to see what we can do to identify multiple result objects as not representing multiple attempts in a stream of statements.

It sounds like in this case though that the result data really relates to the learner answering the question, it's just that there's challenges in capturing that they answered the question. They didn't really receive a score for the interaction. I can see why it really is a challenge to track 'answered' on clicking next though.

Hmm...

Completion = false is a bad solution in this particular case too because you'd never have completion = true.

Stripping the result in the LRS plugin is also bad because you'd never get the result if they don't click check.

I'll have to think some more on this.

[falcon-git]

Yeah, I think in this case if this were to be reported to the LRS the quiz module probably should have done it. It should have triggered 'answered' when next was pushed, and 'completed' when the quiz was finished, but quiz isn't xAPI enabled, so the statements here should perhaps not be sent to an LRS at all until the Drupal quiz module gets xAPI support

[garemoko]

Having slept on this, I can see that there is value to reporting the score (and, in fact, response) of the learner as they complete the interaction. I think there's three things we can do to help the LRS figure out delimitations between attempts:

1. Send a statement with the verb "attempted" to mark the start of each attempt.
2. Use completion=false until the end of the attempt
3. Use context.registration (this would be sent by the LRS plugin)

Unfortunately none of these are universally helpful for the LRS because:

1. Some activity providers send "attempted" at the beginning and end of an attempt.
2. H5P will never be sending completion=true so this is not very useful as an attempt delimiter.
3. The concept of registration is not equal to the concept of attempt. Some activity providers will use the same registration to cover multiple attempts. In fact, even in H5P's case we'd probably need to tie registration to viewing a particular CMS page, which might incorporate re-tries at the question.

So, it's not a complete solution yet.

[falcon-git]

Good, in terms of "Unfortunately 2" H5P could send completion = true for answered and completed statements.

[garemoko]

Right. I was thinking that the answered statement wouldn't always be sent, but actually in those cases the registration would change between attempts. 

Sent from Samsung Mobile

[falcon-git]

This will remain as it is, but we'll start reporting attempted, which belongs in a different repository.