Contributing: Adding Data Sources

This page explains how to new data sources for BUVIC.

Editing the settings

As a first step, we edit the settings so that the the user can select the data source he wants.

Two data sources are currently defined in the DataSource enum in the file settings.py:

1. Files: data is parsed from files
2. EUBREWNET: data is retrieved via calls to the eubrewnet API

These data source can be chosen for UV data, Ozone and UVR data. For the rest of the data (parameters, arf,...), only the files are available.

If you just want to add a new data source to the existing choice, you can add it to the DataSource enum and it will be available in the settings.

If you want to add the choice to a new type of data, you will need to add a new field in the Settings object and a new widget to the GUI.

Adding a new field to the settings

Let's say we want to add source choice for the arf. The first step is to add a field in the Settings class in the file settings.py. We add the new field with the following line:

arf_data_source: DataSource = DataSource.FILES

Adding a new widget to the settings' GUI

At the end of the __init__ method of the SettingsWidget class in file widgets.py, add the following lines (before the line self.append(self._source_container)) to add a dropdown:

# Initialize the dropdown
self._arf_source_selection = gui.DropDown()

# Add the data source as options
for source in DataSource:
    self._arf_source_selection.append(gui.DropDownItem(source))

# Set the selected value of the dropdown to the one from the settings
self._arf_source_selection.set_value(settings.arf_data_source)

# Create a widget with the description and the dropdown
uvr_source_input = Input("ARF data source", self._arf_source_selection, style="margin-bottom: 10px")

# Add the widget to the container
self._source_container.append(uvr_source_input)

The widget is created in the GUI but its value is not saved yet. In the save method, add the following line to retrieve the selected value when saving:

arf_data_source = self._arf_source_selection.get_value()

and add it to the Settings constructor just below with the line:

DataSource(arf_data_source),

Your data can now be correctly edited, saved and loaded in the settings.

Creating the provider

Most of the data types have some kind of "data provider" class. In this example we will look at the ARFProvider class defined in file arf_file.py.

To create a eubrewnet provider for arf, we create a subclass of ARFProvider:

class EubrewnetARFProvider(ARFProvider):
    _url_string: str

    def __init__(self, brewer_id: str):
        self._url_string = f"http://rbcce.aemet.es/eubrewnet/data/get/TODO/PATH/TO/API/POINT?brewerid={brewer_id}"

    def get_arf(self) -> ARF:
        LOG.info("Retrieving arf data from %s", self._url_string)
        try:
            response = requests.get(self._url_string, auth=requests.auth.HTTPBasicAuth("user_name", "password"))
            data = json.loads(response.text)
            szas = []
            values = []

            # TODO: read data values into szas and values lists

            return ARF(szas, values)
        except Exception as e:
            raise Exception(f"Error while trying to access eubrewnet ({self._url_string}). {e}") from e

Finally, in the arf method of class CalculationInput, make a check for the settings' value and use the corresponding provider:

if self.settings.arf_data_source == DataSource.FILES:
    if self.arf_file_name is None:
        LOG.warning("No arf file specified. Cos correction will not be applied")
        warn(f"ARF file was not found. Cos correction has not been applied")
        return None
    return FileARFProvider(self.arf_file_name.full_path, self.settings.arf_column).get_arf()
else:
    return EubrewnetARFProvider(self.brewer_id).get_arf()