Commits on Mar 14, 2024

1. Avoid keeping references to full trajectories …

   Previously we stored a view into the trajectory in the preference
   comparison dataset. This view is a reference to the original trajectory,
   and therefore keeps it from getting garbage collected for as long as the
   view exists (i.e., however long the comparison is stored in the
   dataset).
   
   This is problematic when trajectories are large and long, e.g., in the
   case of atari (images) with SEALS (long episodes). It can cause the RAM
   to fill up quite quickly in that setting.
   
   We can fix it by copying the fragments we want to store. That avoids
   keeping a reference to the original trajectory alive, we only need to
   store the fragment. The tradeoff is that copying adds some overhead and
   overlapping fragments are no longer deduplicated.

   

   timokau committed Mar 14, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 71ac4f3
   • Browse repository at this point

   Copy the full SHA

   71ac4f3 View commit details

   Browse the repository at this point in the history