| title | layout |
|---|---|
Weekly reports |
docs |
Note: As of Wednesday, May 1, 2024, the US COVID-19 Forecast Hub is no longer accepting submissions. Information provided on forecast submissions are kept for historical record.
We followed a weekly update cycle at the COVID-19 Forecast Hub. Every Tuesday morning, our team assembled the most recent forecasts from all teams that had submitted in the last week. We used these data to generate an ensemble forecast that synthesized predictions of COVID-19 cases, deaths, and hospitalizations from all eligible models. These forecasts were then reviewed, analyzed, and verified by our collaborators at the US CDC. On Thursday mornings, the CDC updated their COVID-19 Forecasting webpage with the latest data from the COVID-19 Forecast Hub.
Update: As of February 20, 2023, we are no longer generating ensemble case forecasts, as of March 6, 2023, we are no longer generating ensemble death forecasts, and as of May 4, 2024, we are no longer generating ensemble hospitalization forecasts. Reports up to March 13, 2023 include case and death forecast evaluations, and after that date they only included hospitalization forecast evaluations. The last forecast summary reports were generated on October 3, 2023.
Periodically, we evaluated the accuracy and precision of the ensemble forecast and component models over recent and historical forecasting periods. Models forecasting incident hospitalizations at a national and state level were evaluated using (adjusted relative weighted interval scores (WIS, a measure of distributional accuracy), and adjusted relative mean absolute error (MAE), and calibration scores. Scores were evaluated across weeks, locations, and targets. You can read a paper explaining these procedures in more detail, and look at the most recent monthly evaluation reports. The final report that includes case and death forecast evaluations is 2023-03-13, and the last hospitalization report is 2023-06-23.
In collaboration with the COVID-19 Forecast Hub, the Delphi Group at Carnegie Mellon University created a Forecast Evaluation Dashboard that compared the performance of models over time, using different outcome variables (incident cases, deaths, and hospitalizations), metrics (Weighted Interval Score, Absolute Error, and Coverage), horizons (1 through 4 week forecasts), and locations (US national and state levels). As of February 20, 2023, case forecasts are no longer being updated, as of March 6, 2023 death forecasts are no longer being updated, and as of March 13, 2024 hospitalization forecasts are no longer being updated. Learn more about the dashboard and download the code on GitHub.
Note: As of October 3, 2023 we are no longer generating weekly forecast summary reports. Prior to this, each week, we generated a weekly report that provided top-level summary numbers from the ensemble forecast. Browse past forecast summaries.