You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi Luca, I am actually working on Knowledge Tracing for my thesis. I looked at your code to see if I could find a DKT model as baseline.
While trying it I found 2 errors:
distraction error in metrics.py in TruePositives function, which extends tensorflow TrueNegatives (instead of truePositives) resulting in wrong ACC and AUC.
using "get_target" function before computing loss function and metrics does not support the use of automatic masking.
I know you changed all -1 s to zeros, making it work, but this way the padded values will be considered in your metrics and loss as correct anwered values, modifying real Accuracy and AUC for example. I suggest you to try incorporating "get_target" function in a final custom layer of the model instead, giving as input the target skill to the model too. This way you can use directly a custom layer (remember to define compute_mask function too) to do the reduce_sum over the skill_true array. I have done it and values of AUC, ACC and loss change a lot in the case you have very different length of the sequences as input (due to large padding).
Despite this errors thanks for the code available, I used to understand better how to implement my own DKT as baseline
Bye ;)
The text was updated successfully, but these errors were encountered:
SimoneSartoni
changed the title
2 errors found: TrueNegatives typos, "get_target" function does not support
2 errors found: TrueNegatives typos, "get_target" function does not support masking
Dec 13, 2021
Hi Luca, I am actually working on Knowledge Tracing for my thesis. I looked at your code to see if I could find a DKT model as baseline.
While trying it I found 2 errors:
I know you changed all -1 s to zeros, making it work, but this way the padded values will be considered in your metrics and loss as correct anwered values, modifying real Accuracy and AUC for example. I suggest you to try incorporating "get_target" function in a final custom layer of the model instead, giving as input the target skill to the model too. This way you can use directly a custom layer (remember to define compute_mask function too) to do the reduce_sum over the skill_true array. I have done it and values of AUC, ACC and loss change a lot in the case you have very different length of the sequences as input (due to large padding).
Despite this errors thanks for the code available, I used to understand better how to implement my own DKT as baseline
Bye ;)
The text was updated successfully, but these errors were encountered: