Consistent loss and acc difference between shadow models and target model

[henrikfo]

Issue

Problem Description

Consistent loss and acc difference between shadow models and target model during training.

Expected Behavior

For many runs of leakpro, training taget and shadow model from scratch, they should have the same average of loss and acc.
Right now the target model is on average 2-3 % points better than the average of shadow models trained.

What Needs to be Done

Figure out the difference within the training loop, amount of data, what data the target model is trained on etc.

How Can It Be Tested or Reproduced

Run leakpro with retrain=True and train and some shadow models for 6 or more epochs a few times

[henrikfo]

target model is not using the same indices for multiple runs, opimizer configuration is set correctly. One hint might be that the target model takes <50% of the time for training one epoch