Packaging modern language environments

[nrybowski]

New languages such as Rust or Go pull and build the libraries required by the projects on the fly.
This would imply a strong load in term of network and cpu usage for INGInious.

We should consider a viable solution to package such environments, e.g. specialized containers embedding directly the required libraries (heavy and task specific) or somehow implement a caching system of pre-compiled libraries.

[javawolfpack]

Wouldn't that be effectively covered in the documentation:

https://docs.inginious.org/en/v0.8.2/admin_doc/install_doc/create_container.html

I assumed we could build our custom container with the specialized prerequisites already created. Rust could likely have the Dockerfile built with a skeleton file that a student's submission would replace to at least build all the dependencies in the build path where the submission would load.

Mind you, I also recently discovered file submission (and no idea if multi-file submission exists) is not in best practices, so already reconsidering if this is the best platform for us... or if I'd have to invest significant time to build a plugin to do what I need functionality wise. But I have plans to adopt Rust in a course in the future, so I had expected I could build a custom container since that was in the documents but if I'm wrong would be good to know.

[anthonygego]

Those best practices were written almost 10 years ago and are probably not relevant anymore. Besides, as you know, software usage often deviates from what the developers thought initially. These recommandations were written at a time where storage was not as easily available and certainly not as fast as today, and were specifically written towards users that bypassed the lack of a Scala environment by telling the students to include the Scala runtime with their project files, filling the small disks of our local instance at a rapid pace. Therefore, the main idea remains: avoid designing tasks where the students have to submit something that can probably be cached somewhere. But certainly asking students for files is not a bad practice at all.

The issue @nrybowski address here is quite different as the issue is no more submission storage but in the runtime itself. And yes, building your own container with cached data is something good. However, only the platform administrator is able to add new environments and caching specific project files is something of the course administrator responsibility. Two solutions are currently discussed:

• As @nrybowski suggests, to find something more generic that would not require course administrator action.
• As you would go, designing course-specific environments with the required files in there. We started building a solution in that way, called "judge", in which the course administrator would be able to specify their own environments and also reduce each task definition. Currently, the first steps are implemented and you can build a container with all your grading files located in /course/common. The container /course/common/run path is a valid entrypoint and will run with low priority (if there is no run file in the task).

[nrybowski]

As a first step, Rust packages used in exercises could be statically compiled and embedded in the /course/common directory. Student code should then be linked on such static libraries.

That being said, a better solution will be the usage of judges directly embedding the crates installed with cargo.