Proposed AQUA integration

[johnml1135]

Here is a potential way to integrate AQUA:

• AQUA assessments done in k8s in a separate docker container
• Pre and post processing done using Serval repo, main AQUA algorithm in separate repo, as well as deployment scripts
• Needed alignments done in machine

A bit more in depth:

• Machine changes:

  • Merge together machine-engine and machine-job, now that the SMT jobs are done through ClearML
  • Expose alignments in machine, serving the same way as pretranslations

• Serval changes:

  • Allow engines to have "parent" and "child" links (or similar), which connect to different engines.
  • There is a new state when building an engine called "check preconditions"
  • When a child engine finishes building, it checks any parents to see if all preconditions have been met because the child finished.

• Assessment

  • Specifies multiple alignment engines that need to be completed at a specific revision as "preconditions" before the build starts.
  • The flow would be: Setup all alignment engines and the assessment engine. When starting the build, specify the minimum revision of the build of each engine to be checked in preconditions
  • When the assessment starts, it already has everything to complete the assessment without interacting with machine again
  • When an updated assessment is requested, Serval asks the child engine for the alignment and then the assessment engine for the updated assessment.

@ddaspit, what do you think?

[johnml1135]

Planning updates:

• AQUA done with internal instance, not IDX's instance
• Main AQUA algorithm running in ClearML in new container
• Utilize existing Dotnet infrastructure for data marshalling and job handling
• For the job, Serval python code calls AQUA python code which calls machine.py
  • Can we publish a docker image to a private repo: https://stackoverflow.com/questions/28349392/how-to-push-a-docker-image-to-a-private-repository

Updates to code:

• Machine.py job code moved to Serval
• Code added to Serval to call AQUA
  • Serval has extra poetry AQUA dependency referencing git tag: { git = "AQUA.git", tag = "v1.2.3" }
• AQUA updated to be callable with these functions:
  • aqua = AQUA(config)
  • aqua.run(parallelCorpus, listOfReferenceAssessments)
  • assessments = aqua.get_assessment()
  • path = aqua.save_model()

Phase 2: - live inference

• The saved model will be on the S3 bucket
• Using python.net, the Serval/AQUA dotnet code will call AQUA code to run the incremental algorithm, utilizing the saved models and any other needed assessments.