[frontend] Changing accessibility and registration data structure

[AlexandreDoneux]

This pull request has as goal the change of accessibility and registration data structure in in the code as well as for the storage in DB and yaml files. The objective is also to store the dates as ISO dates in the database and yaml files. This concerns course accessibility, course registration period and task accessibility.

These accessibilities where previously a single value (accessible/registration for courses and accessibility for tasks) and were structured as follows :

• a true when always accessible
• a false when never accessible
• a concatenated string containing all the dates separated by a "/"
  (ex: "2023-12-06 09:00:00/2023-12-13 09:00:00/2023-12-13 09:00:00")

We will now only use the dates and use minimal and maximal dates to define the cases where the the courses and tasks are always or never accessible. Those minimal and maximal dates are also used in a custom accessibility when certain dates are not defined. You can for example only define a start date in the form and INGInious will then assume there is no end date to the task and it will always be accessible after the given start date.

TO DO :

• Change datastructure in AccessibleTime class and it's usage
  • adapt AccessibleTime
  • adapt usage basic INGInious app
  • adapt usage in contest plugin
  • Change AccessibleTime default parameters and it's use
• Changing init structure
  • courses
  • tasks
• Adapting forms to new structure
  • Transforming datetime objects into strings
  • course settings
  • task accessibility edit
  • task accessibility grouped edit
• Dates as ISODate() objects in MongoDB
  • Storing (transform in datetime objects before storing in DB)
  • Retrieving
  • Correctly handling min and max dates
• Storing dates as timestamps objects in yaml files
  • Storing
  • Retrieving
  • Correctly handling min and max dates
• Automatically change structure in DB and yaml files when switching version
  • DB
    • [ ] In already existing collections => not needed
    • During "courses" collection init in legacy courses import
    • During legacy task import in task list edit
  • YAML files
    • During legacy task import in course template

[AlexandreDoneux]

Rebase on cf8141c.

[AlexandreDoneux]

Rebase on 1095603.

[AlexandreDoneux]

Rebase on 4de3a17.

[anthonygego]

I think at this stage that you can extract and movethe i18n changes to another PR so that this one gets clearer.

It seems there is no complete consistency for years "0001-" and "1-".

I would recommend, for the max date, to only work with a "one second" accuracy to avoid troubles between the code, the OS-specific accuracy and pymongo.

I eventually woner if keeping support for the older string format in AccessibleTime constructor could not make the code (and migration one) more cleaner.

[AlexandreDoneux]

Some code might need to be rewritten in settings.py :

INGInious/inginious/frontend/pages/course_admin/settings.py

Lines 24 to 51 in 2306071

	def POST_AUTH(self, courseid): # pylint: disable=arguments-differ
	""" POST request """
	course, __ = self.get_course_and_check_rights(courseid)
	errors = []
	course_content = {}
	data = flask.request.form
	course_content = self.course_factory.get_course_descriptor_content(courseid)
	course_content['name'] = data['name']
	if course_content['name'] == "":
	errors.append(_('Invalid name'))
	course_content['description'] = data['description']
	course_content['admins'] = list(map(str.strip, data['admins'].split(','))) if data['admins'].strip() else []
	if not self.user_manager.user_is_superadmin() and self.user_manager.session_username() not in course_content['admins']:
	errors.append(_('You cannot remove yourself from the administrators of this course'))
	course_content['groups_student_choice'] = True if data["groups_student_choice"] == "true" else False
	if data["accessible"] == "custom":
	course_content['accessible']["start"] = datetime.strptime(data["accessible_start"], '%Y-%m-%d %H:%M:%S') if data["accessible_start"] != "" else course._accessible.min
	course_content['accessible']["end"] = datetime.strptime(data["accessible_end"], '%Y-%m-%d %H:%M:%S') if data["accessible_end"] != "" else course._accessible.max
	elif data["accessible"] == "true":
	course_content['accessible']["start"] = course._accessible.min
	course_content['accessible']["end"] = course._accessible.max
	else:
	course_content['accessible']["start"] = course._accessible.max
	course_content['accessible']["end"] = course._accessible.max

The data of the course is accessed through the course_factory :

INGInious/inginious/frontend/pages/course_admin/settings.py

Line 32 in 2306071

course_content = self.course_factory.get_course_descriptor_content(courseid)

However, the course is retrieved at the start of the method. We could rewrite a bit to use the course instead of self.course_factory.get_course_descriptor_content(courseid).

There is also the use of a protected member to use the min and max dates instead of datetime.min and datetime.max.replace(microsecond=0). A method on course retrieving the content would allow us acces to those values.

[AlexandreDoneux]

Keeping support for the legacy access format and changing the format on saving the settings of a course or task is not as easy as we thought for the moment.

As you suggested we can add support in the AccessibleTime init. For a course accessibility and registration period it only needs some tweaks. However task accessibility is more difficult as the accessibility of a task for the edit pages (taskset template and course tasks) is not passed through AccessibleTime.

For example in toc.py, task_config is passed to the jinja template and contains the data of the different tasks.

    def render_edit(self, template_helper, element, task_data, task_errors):
        """ Returns the formatted task list edition form """
        config_fields = {
            "closed": SectionConfigItem(_("Closed by default"), "checkbox", False),
            "hidden_if_empty": SectionConfigItem(_("Hidden if empty"),"checkbox",False)
        }

        taskset = element if isinstance(element, inginious.frontend.tasksets.Taskset) else None
        course = element if isinstance(element, inginious.frontend.courses.Course) else None
        task_config = dict_data_datetimes_to_str(self._task_config)

        return template_helper.render("task_dispensers_admin/toc.html", element=element, course=course, taskset=taskset,
                                      dispenser_structure=self._toc, dispenser_config=task_config, tasks=task_data,
                                      task_errors=task_errors, config_fields=config_fields)

However, when tracing back from where the data comes from we can see it is retrieved from the database here, in course_factory.py, and is never processed by AccessibleTime :

    def get_course_descriptor_content(self, courseid):
        return self._database.courses.find_one({"_id": courseid})

Edit:
I changed the access structure change to be applied when importing legacy tasks. The has_legacy_tasks() method has been modified to detect if tasks have the old access structure. The legacy tasks import button will be displayed if it's the case (in task_list.html and template.html).

[AlexandreDoneux]

Rebase on b3a4095.