Enable Datastore to watch multiple directories

[fm3]

Steps to test:

• create second dataset directory next to binaryData, insert it in application.conf braingames.binary.additionalFolders
• datasets there should be listed, data loading should work
• for the original directory also.

@normanrz do you think we should do some caching for the lookup determining which directory exists? I’m not sure how cache invalidation would work for this. But right now it feels like we’re introducing many additional disk accesses.

Issues:

• contributes to Give WK direct access to /tmpscratch #3310

---

• Updated changelog
• [ ] Updated migration guide if applicable
• [ ] Updated documentation if applicable
• Needs datastore update after deployment
• Ready for review

[normanrz]

Maybe we could implement it like this: We keep one main folder, and have additional folders that are watched. When new folders in the additional folder are detected, a symlink is created to the main folder? That would also prevent duplicate dataset names.

[fm3]

Good idea, I changed it according to your suggestion. Do you think we should also scan for and delete dangling symlinks?

[normanrz]

Yes, I think we should also remove dangling links.

[youri-k]

LGTM