Forcast target

[dulte]

Change Summary

Adds new fairmode engine as well as persistent model capability for cams2_83

Checklist

• Start with a draft-PR
• The PR title is a good summary of the changes
• PR is set to AeroTools and a tentative milestone
• Documentation reflects the changes where applicable
• Tests for the changes exist where applicable
• Tests pass locally
• Tests pass on CI
• At least 1 reviewer is selected
• Make PR ready to review

[charlienegri]

2 issues to be tackled are mos experiments (how to read in the extra needed day since it's an only_json case) and missing conco3mda8, a couple of notes:

• mda8 is in the colocated data when a standard eval is run, written also on disk, so I think the cams283 classes just need to know about it
• the mos experiments do not have it because colocated data is pre-prepared, but as a consequence it is already the case now that the current target plots for the mos experiments do not have o3mda8 and are empty for o3 because we do not calculate fairmode stats for it, so we can argue that if it will be a requirement to have conco3mda8 for mos experiments it has to be done in the library that writes the data, mos_reader (which will require me to implement pyaerocom's mda8_colocated_data there basically, but at least not your problem)
• in the same spirit we could perhaps make it so that the library that writes the colocated data for mos reads in an extra day and writes it keeping the files named as they are.. but I am not sure how this would affect the normal part of the evaluation then...I have to check, if the data is sliced by period as I remember it should be, then the extra day will be cut off as we need it there, but will be on disk for when your code comes in

[charlienegri]

2 issues to be tackled are mos experiments (how to read in the extra needed day since it's an only_json case) and missing conco3mda8, a couple of notes:

* mda8 is in the colocated data when a standard eval is run, written also on disk, so I think the cams283 classes just need to know about it

* the mos experiments do not have it because colocated data is pre-prepared, but as a consequence it is already the case now that the current target plots for the mos experiments do not have o3mda8 and are empty for o3 because we do not calculate fairmode stats for it, so we can argue that if it will be a requirement to have conco3mda8 for mos experiments it has to be done in the library that writes the data, mos_reader (which will require me to implement pyaerocom's mda8_colocated_data there basically, but at least not your problem)

* in the same spirit we could perhaps make it so that the library that writes the colocated data for mos reads in an extra day and writes it keeping the files named as they are.. but I am not sure how this would affect the normal part of the evaluation then...I have to check, if the data is sliced by period as I remember it should be, then the extra day will be cut off as we need it there, but will be on disk for when your code comes in

my commit should address point 1
about the 3rd point: it seems like the CAMS2_83_Processer is coded such that in the only_json case the CAMS2-83-persistent*/ folders are expected to be in place, so this is the extra content that the mos_reader will need to create, on the top of calculating conco3mda

[codecov]

Codecov Report

Attention: Patch coverage is 63.30532% with 131 lines in your changes missing coverage. Please review.

Project coverage is 76.96%. Comparing base (5261319) to head (bd56247).
Report is 1 commits behind head on main-dev.

Files with missing lines	Patch %	Lines
pyaerocom/aeroval/fairmode_engine.py	61.94%	51 Missing ⚠️
pyaerocom/scripts/cams2_83/engine.py	77.30%	32 Missing ⚠️
pyaerocom/aeroval/coldatatojson_helpers.py	4.54%	21 Missing ⚠️
pyaerocom/aeroval/coldatatojson_engine.py	27.77%	13 Missing ⚠️
pyaerocom/scripts/cams2_83/processer.py	35.00%	13 Missing ⚠️
pyaerocom/scripts/cams2_83/cli.py	83.33%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           main-dev    #1551      +/-   ##
============================================
- Coverage     77.19%   76.96%   -0.24%     
============================================
  Files           147      148       +1     
  Lines         21344    21655     +311     
============================================
+ Hits          16476    16666     +190     
- Misses         4868     4989     +121     

Flag	Coverage Δ
unittests	76.96% <63.30%> (-0.24%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[charlienegri]

the colocated data written with the code in this PR is coherent with this PR in the mos_reader library https://gitlab.met.no/alvarov/mos_reader/-/merge_requests/6

[charlienegri]

how does this skipping work?

[charlienegri]

still wondering about this case