Add country field to Address. Deprecate countryName.

[duncandewhurst]

Closes #524

Once merged:

• Add the instructions for downloading ISO3166 in machine-readable form to https://ocds-standard-development-handbook.readthedocs.io/en/latest/standard/technical/deployment.html#perform-periodic-updates

[duncandewhurst]

@jpmckinney @yolile a couple of questions to resolve before this PR is ready for review:

1. Currently, this PR deprecates Address.countryName based on the justification in Add Address.country field (with ISO3166-1 alpha-2 codelist) #524 (comment). On reflection, I have two concerns about this:
   1. I anticipate some publishers in lower-tech contexts will collect country as a free-text field, so we will have helpdesk requests about where to put this data in OCDS.
   2. ISO3166-1 is not freely/readily available, so we may be creating a barrier to data use by removing the human-readable country name from the standard.
2. I've integrated the script from https://github.com/open-contracting-extensions/ocds_countryCode_extension/tree/master/script into manage.py. Is https://ocds-standard-development-handbook.readthedocs.io/en/latest/standard/technical/deployment.html#perform-periodic-updates the right place to put the instructions for downloading ISO3166 in machine-readable form?

[yolile]

I anticipate some publishers in lower-tech contexts will collect country as a free-text field, so we will have helpdesk requests about where to put this data in OCDS.

Hmm, true, but, maybe, before publishing OCDS data (eg as part of the OCDS transformation process), they can map the names into their ISO codes?

ISO3166-1 is not freely/readily available, so we may be creating a barrier to data use by removing the human-readable country name from the standard.

Also true, but as I understand, we will publish the CSV with the codes and names, right? And I guess it also depends on the use case. For example, if the use case is to show the data in a website/mobile app etc, the developers can do the map and show the human-readable in the website using our CSV as the base, and as mentioned in this #524 (comment), having the codes is better for that case. Then, another use case, let say that you want to analyze the suppliers by country or similar, it is easier to do that if the names/codes are standardized already and I think it doesn't matter too much if the name is a human-readable name or a code (same if you are planning to use the data for an AI project, etc)

I've integrated the script from https://github.com/open-contracting-extensions/ocds_countryCode_extension/tree/master/script into manage.py. Is https://ocds-standard-development-handbook.readthedocs.io/en/latest/standard/technical/deployment.html#perform-periodic-updates the right place to put the instructions for downloading ISO3166 in machine-readable form?

Yes, I think there is a good place to put it

[duncandewhurst]

I anticipate some publishers in lower-tech contexts will collect country as a free-text field, so we will have helpdesk requests about where to put this data in OCDS.

Hmm, true, but, maybe, before publishing OCDS data (eg as part of the OCDS transformation process), they can map the names into their ISO codes?

Ideally, yes. But, if the values are free-text (i.e. users type the country names rather than choosing them from a list) the mapping would only cover historic data - users might enter new acronyms, abbreviations or misspellings after OCDS is implemented.

ISO3166-1 is not freely/readily available, so we may be creating a barrier to data use by removing the human-readable country name from the standard.

Also true, but as I understand, we will publish the CSV with the codes and names, right? And I guess it also depends on the use case. For example, if the use case is to show the data in a website/mobile app etc, the developers can do the map and show the human-readable in the website using our CSV as the base, and as mentioned in this #524 (comment), having the codes is better for that case. Then, another use case, let say that you want to analyze the suppliers by country or similar, it is easier to do that if the names/codes are standardized already and I think it doesn't matter too much if the name is a human-readable name or a code (same if you are planning to use the data for an AI project, etc)

Good points, I forgot we were publishing the names ourselves. I agree that, depending on the application, codes might be better - I wasn't suggested to not add the code field. I also agree that it's not much of a barrier for more capable users to download the CSV and look up the human-readable names. However, from a spreadsheet user / low capacity perspective, users are likely to want the names in the results of their analysis and importing and joining the CSV might be a barrier.

I've integrated the script from https://github.com/open-contracting-extensions/ocds_countryCode_extension/tree/master/script into manage.py. Is https://ocds-standard-development-handbook.readthedocs.io/en/latest/standard/technical/deployment.html#perform-periodic-updates the right place to put the instructions for downloading ISO3166 in machine-readable form?

Yes, I think there is a good place to put it

Great. I'll do that once this PR is merged.

[jpmckinney]

Regarding whether to have two fields or one: This is similar to the discussion in #1295 about valueEntered. When data is known to have quality issues, it sometimes makes sense to segment the "good" data from the "possibly bad" data.

For fields like numerical values, this is important, because major use cases become more difficult if the quality is mixed.

For address data, if the quality is bad, you just don't know the address and can't geolocate it. So, no use cases are really harmed by putting non-ISO codes (like full country names) in the country field. Whether the bad data is segmented or not, your use case is frustrated.

That said, if a system has only non-standardized country names, then putting their bad data in the only available field will cause lots of structural errors. (But still, the deprecated field can instead be used, which yields a deprecation warning.)

The choice really depends on the prevalence of systems that have free-text data entry for country names. We can perhaps deprecate it, and consult on whether to deprecate it or not. I added it to my note here: https://crm.open-contracting.org/issues/6532#note-14

[jpmckinney]

I added a couple commits.

The instructions for update-country are now part of the --help message, so the handbook can just remind the reader to follow the instructions in ./manage.py update-country --help

[duncandewhurst]

PR for handbook update: open-contracting/standard-development-handbook#247