Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
2409152
Update next platform staging image tag to stage-ac3afa2b-1781013692
github-actions[bot] Jun 9, 2026
97fcf0d
Update analytics production image tag
github-actions[bot] Jun 9, 2026
44f1d01
Add api-routing helpers and migrate API URLs
Codebmk Jun 9, 2026
9a87f55
Use server API URL and fix v2 duplication
Codebmk Jun 9, 2026
b86a8ba
Fix OAuth profile, redirect, and session sync
Codebmk Jun 9, 2026
19cd20d
Update changelog.md
Codebmk Jun 9, 2026
a1e35f6
Hide social auth if API URL missing
Codebmk Jun 9, 2026
3d1564b
Fix server-side URL construction in submitNetworkRequest
Codebmk Jun 9, 2026
e5f3992
Use base origin for redirectAfter URL
Codebmk Jun 9, 2026
fa51217
Encode the value before building the URL
Codebmk Jun 9, 2026
25bcecb
Remove the unused import
Codebmk Jun 9, 2026
fd3f475
Use browser API URL for OAuth and show social auth
Codebmk Jun 9, 2026
829d0c3
Fix historical data overviews to direct users to Metadata API for ID …
Baalmart Jun 9, 2026
8b5701d
Protect social auth and use server API URL
Codebmk Jun 9, 2026
677f11b
fix(docs): clarify device_name vs device_id in partners historical-da…
Baalmart Jun 9, 2026
82c0268
Delegate OAuth route protection to client
Codebmk Jun 9, 2026
02c3739
Encode dynamic path segments in the status-update URL.
Codebmk Jun 9, 2026
d3b0533
Merge pull request #3606 from airqo-platform/docs-id-lookup
Baalmart Jun 9, 2026
9fae1da
Update docs staging image tag to stage-d3b0533e-1781033096
github-actions[bot] Jun 9, 2026
d142ed2
Merge pull request #3604 from airqo-platform/fix-login-vertex
Baalmart Jun 9, 2026
bdaab28
Add researcher guide and fair usage policy as new data-access docs se…
Baalmart Jun 9, 2026
47aab8f
fix(docs): standardise contact email to support@airqo.net across data…
Baalmart Jun 9, 2026
fe3aab4
Merge pull request #3609 from airqo-platform/docs-data-access
Baalmart Jun 9, 2026
c62e6cc
Update docs staging image tag to stage-fe3aab4d-1781035269
github-actions[bot] Jun 9, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
replicaCount: 2
image:
repository: airqoacr.azurecr.io/airqo-stage-docs
tag: stage-cd087882-1780953563
tag: stage-fe3aab4d-1781035269
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ''
Expand Down
2 changes: 1 addition & 1 deletion k8s/platform/k8s-aks/airqo-platform/values-prod.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
replicaCount: 1
image:
repository: airqoacr.azurecr.io/airqo-next-platform
tag: prod-8e2b7cdb-1780947081
tag: prod-ac63eb8a-1781013838
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ''
Expand Down
2 changes: 1 addition & 1 deletion k8s/platform/k8s-aks/airqo-platform/values-stage.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
replicaCount: 1
image:
repository: airqoacr.azurecr.io/airqo-stage-next-platform
tag: stage-67057238-1780943479
tag: stage-ac3afa2b-1781013692
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ''
Expand Down
2 changes: 1 addition & 1 deletion src/docs-website/docs/api/for-cities/historical-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Direct Grid ID filtering will be added to the Analytics API in a future release.

## Overview

To query historical data for your Grid, use the site names or device names from your Grid in the Analytics API request. You can discover the sites in your Grid by calling the [recent measurements endpoint](./recent-measurements.md) first and collecting the `site_id` and `device` values.
To query historical data for your Grid, supply the Site IDs or device names for that Grid in the Analytics API request. Use the [Metadata API](../reference/metadata.md#get-all-site-and-device-ids-for-a-grid) (`GET /api/v2/devices/grids/{GRID_ID}/generate`) to retrieve all site and device identifiers for your Grid.

---

Expand Down
4 changes: 2 additions & 2 deletions src/docs-website/docs/api/for-partners/historical-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Historical data access requires a **Standard Tier** subscription or above.
:::caution Cohort ID direct filtering — coming soon
The Analytics API does not yet accept `cohort_id` as a request parameter. You must supply individual **device names** (`device_names`) in the request body instead.

**Workaround:** Call the [Metadata API](../reference/metadata.md#get-all-site-and-device-ids-for-a-cohort) (`GET /api/v2/devices/cohorts/{COHORT_ID}/generate`) to retrieve all device identifiers for your Cohort, then include those in your Analytics API request.
**Workaround:** Call the [Metadata API](../reference/metadata.md#get-all-site-and-device-ids-for-a-cohort) (`GET /api/v2/devices/cohorts/{COHORT_ID}/generate`) to retrieve the `device_name` values for your Cohort, then pass them as the `device_names` parameter in your Analytics API request.

Direct Cohort ID filtering will be added to the Analytics API in a future release.
:::
Expand All @@ -23,7 +23,7 @@ Direct Cohort ID filtering will be added to the Analytics API in a future releas

## Overview

Historical data for your Cohort is fetched via the Analytics API by specifying the device names that belong to your Cohort. If you do not know the device names, use the recent measurements endpoint first — the `device` field in each measurement is the device name.
Historical data for your Cohort is fetched via the Analytics API by specifying the device names that belong to your Cohort. Use the [Metadata API](../reference/metadata.md#get-all-site-and-device-ids-for-a-cohort) (`GET /api/v2/devices/cohorts/{COHORT_ID}/generate`) to retrieve all device names for your Cohort.

---

Expand Down
200 changes: 200 additions & 0 deletions src/docs-website/docs/data-access/fair-usage-policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
---
sidebar_position: 2
sidebar_label: Fair Usage Policy
---

# Air Quality Data Fair Usage Policy

Guidelines for Responsible and Equitable Access to Air Quality Data

:::info Document version
Version 1.0 — January 2026
:::

---

## 1. Why This Policy Exists

AirQo is a non-commercial air quality monitoring initiative operated by Makerere University, funded by research grants and development partners. All data is provided free of charge in line with our commitment to open access and advancing air quality science across Africa.

Our data infrastructure (servers, databases, compute nodes, and network bandwidth) is shared across all users simultaneously: researchers, students, government agencies, NGOs, partner organisations, and the general public. These resources are finite and not infinitely scalable.

When any single user or process requests very large volumes of data at once, it creates a disproportionate computational burden that can:

- Slow down or interrupt access for all other users on the platform
- Cause system instability or outright service downtime
- Degrade the performance of real-time data pipelines that inform public health decisions
- Exhaust database query capacity, causing timeouts for everyone

:::caution
This is not a technical limitation we intend to remove. Download batching is an intentional design decision to ensure fair, stable, and equitable access for all users of a shared, free resource.
:::

---

## 2. How the Platform Works

### 2.1 Data Volume and Sampling Frequency

To appreciate why large downloads are resource-intensive, consider the raw data volumes involved:

| Period | Readings Per Monitor | Readings (50 Monitors) |
|--------|---------------------|------------------------|
| 1 Hour | 6 | 300 |
| 1 Day | 144 | 7,200 |
| 1 Month | ~4,320 | ~216,000 |
| 1 Year | ~52,560 | ~2,628,000 |
| 6 Years (2019–2024) | ~315,360 | ~15,768,000 |

Note: Kampala alone has over 50 active monitors. A single query for six years of hourly data across all Kampala monitors would request tens of millions of database rows, requiring significant compute time, memory, and bandwidth.

### 2.2 The Batch Download Limit

To protect system stability and ensure equitable access for all users, the AirQo Analytics Platform intentionally limits single downloads to approximately **three months of data per query**.

:::warning This limit is intentional and permanent
This limit is displayed clearly via tooltips in the platform interface. It applies universally to all users regardless of institutional affiliation, seniority, or research scope.
:::

---

## 3. How to Download Large Historical Datasets

Obtaining multi-year datasets is entirely possible. It simply requires breaking up your requests into manageable batches, which is standard practice in data-intensive research and reflects good data management hygiene.

### 3.1 Recommended Approach: Batch Downloads via the Analytics Platform

For most academic and public health researchers, the Analytics Platform is the simplest and most accessible option. Follow this workflow:

1. Navigate to the AirQo Analytics Platform: https://airqo.africa/products/analytics
2. Select your study area (e.g., Kampala) and the monitors of interest.
3. Choose a data frequency (hourly is recommended for seasonal analysis).
4. Set your date range to a three-month window (e.g., January – March 2019).
5. Export the data as a CSV file and save it with a clearly named file (e.g., `Kampala_hourly_2019_Q1.csv`).
6. Repeat for each subsequent quarter until you have covered your full study period.
7. Combine your quarterly files in your analysis software (Excel, R, Python, Stata, etc.).

:::tip
For a six-year study period (2019–2024), this means approximately 24 separate downloads — one per quarter per year. Each download takes only a few minutes to complete.
:::

### 3.2 Suggested Quarterly Download Schedule

| Quarter | Date Range | Suggested Filename |
|---------|------------|--------------------|
| Q1 | Jan 1 – Mar 31 | `Kampala_hourly_YYYY_Q1.csv` |
| Q2 | Apr 1 – Jun 30 | `Kampala_hourly_YYYY_Q2.csv` |
| Q3 | Jul 1 – Sep 30 | `Kampala_hourly_YYYY_Q3.csv` |
| Q4 | Oct 1 – Dec 31 | `Kampala_hourly_YYYY_Q4.csv` |

Repeat the above for each year in your study period (2019, 2020, 2021, 2022, 2023, 2024).

### 3.3 Advanced Option: Programmatic Access via the AirQo API

For users comfortable with programming (Python, R, or similar), the [AirQo API](../api/intro.md) provides a more efficient and automated pathway for downloading large datasets.

| | |
|---|---|
| **API Documentation** | [AirQo API →](../api/intro.md) |
| **Free Tier** | Suitable for standard academic research — $0/month |
| **Registration Required** | Yes — to obtain an API key for authentication |
| **Rate Limits** | Applied per tier to ensure equitable access across all API users |
| **Support** | [support@airqo.net](mailto:support@airqo.net) for guidance on large-scale API use |

Using the API, researchers can write scripts that automatically loop through date ranges, paginate through results, and save data incrementally. This is the recommended approach for very large datasets or when integrating AirQo data into automated analysis pipelines.

:::info Note on API rate limits
Even the API is subject to rate limiting. This is standard practice across all major data APIs (Google, OpenStreetMap, NOAA, EPA, etc.) and is not unique to AirQo. Rate limits protect infrastructure integrity and ensure no single user monopolises shared compute resources.
:::

---

## 4. Fair Usage Principles

All users — regardless of seniority, institutional affiliation, or urgency of need — are expected to adhere to the following principles when accessing AirQo data:

| Principle | What It Means in Practice |
|-----------|--------------------------|
| **Batch Your Downloads** | Break multi-year data requests into quarterly or monthly chunks. Do not attempt to circumvent batch limits through workarounds. |
| **Use the Right Tool** | Use the Analytics Platform for exploratory and moderate-sized downloads. Use the API for automated, large-scale, or programmatic access. |
| **Use Calibrated Data** | Always download calibrated (not raw) data for research. This reduces the need for repeated re-downloads to apply corrections. |
| **Download Once, Store Locally** | Save your downloaded files. Do not re-download the same data multiple times. Maintain a local archive for your study period. |
| **Apply Quality Control Locally** | AirQo recommends a ≥50% data completeness threshold. Apply QC filters after downloading, not by requesting new downloads. |
| **Plan Ahead** | Account for download time in your research timeline. If your deadline is imminent, you should have planned data access earlier. |
| **Respect Rate Limits** | Do not write automated scripts that hammer the API without appropriate delays (back-off intervals) between requests. |

---

## 5. What Not To Do

:::danger
- Do not request bulk exports of more than three months in a single query on the Analytics Platform.
- Do not send repeated automated API requests without rate-limiting your own scripts.
- Do not request that AirQo staff manually extract and send multi-year bulk datasets on your behalf as a workaround to platform limits.
- Do not share API credentials or export scripts with unlimited loop behaviour in public code repositories.
- Do not assume that because data is free, the infrastructure costs of delivering it are also zero.
:::

---

## 6. Data Quality and Completeness

When assembling multi-year datasets from batch downloads, researchers should apply the following quality control practices recommended by AirQo:

### 6.1 Data Completeness Threshold

AirQo recommends using only data from monitors that provide **≥50% of expected measurements** for any given analysis period.

- **Hourly analysis** — A monitor providing fewer than 12 readings in a 24-hour period should be flagged for that day.
- **Daily averages** — Fewer than 72 measurements out of 144 expected means the daily average may not be representative.

### 6.2 Missing Data

- AirQo does not interpolate or backfill missing values. Missing readings remain null/empty.
- This is intentional — air quality can change rapidly, making interpolated values unreliable for exposure assessment.
- When combining quarterly files, verify that your analysis software handles null values correctly and does not silently treat them as zeroes.

### 6.3 Coordinate Approximation

- Monitor coordinates are offset by approximately 0.5 km from actual physical locations for privacy reasons.
- This applies across all data access methods.
- For research requiring exact coordinates, contact [support@airqo.net](mailto:support@airqo.net).

---

## 7. Quick Reference Summary

| Topic | Summary |
|-------|---------|
| **Download Limit** | ~3 months per single query on Analytics Platform |
| **Reason** | Shared infrastructure; intentional and permanent design decision |
| **6-Year Dataset** | ~24 quarterly downloads via Analytics Platform, or scripted API access |
| **API Documentation** | [AirQo API →](../api/intro.md) |
| **Analytics Platform** | https://airqo.africa/products/analytics |
| **Recommended Data Type** | Calibrated data (not raw) |
| **Recommended Frequency** | Hourly data for seasonal/temporal analysis |
| **Completeness Threshold** | ≥50% of expected readings per monitor per period |
| **Support & Inquiries** | [support@airqo.net](mailto:support@airqo.net) |

---

## 8. Contact and Support

AirQo is committed to supporting researchers and will gladly provide guidance on data access, methodology, and collaboration opportunities.

| | |
|---|---|
| **General Enquiries & Support** | [support@airqo.net](mailto:support@airqo.net) |
| **Website** | https://airqo.africa |
| **API Documentation** | [AirQo API →](../api/intro.md) |
| **Analytics Platform** | https://airqo.africa/products/analytics |
| **Network Coverage** | https://airqo.net/solutions/network-coverage |
| **Response Time** | 3–5 business days (mark urgent requests in the subject line) |

:::tip Research collaboration
AirQo welcomes research collaboration. If your project is large-scale, cross-institutional, or policy-relevant, please reach out to discuss partnership arrangements, including data access support tailored to your research needs.
:::

---

*This document is prepared by AirQo, Makerere University, Kampala, Uganda. For the latest version, contact [support@airqo.net](mailto:support@airqo.net)*
Loading
Loading