Skip to content

Latest commit

 

History

History
27 lines (14 loc) · 1.16 KB

README.md

File metadata and controls

27 lines (14 loc) · 1.16 KB

IATI Data Dump

A daily snapshot of all IATI data (and metadata) on the IATI registry.

Download all IATI data

Rationale

Lots of IATI products do a daily pull of all IATI XML data. That involves downloading gigabytes of data, via thousands of HTTP requests. So it’s quite slow.

Downloading a single archive file is significantly faster. That’s available here!

How does this work?

We perform a daily fetch of all IATI data, compress it, and upload the archive. There’s a metadata.json file inside the zip, that lets you know how fresh the data is.

This github gist is updated with the list of erroring datasets.

Acknowledgements

The original code relied heavily on IATI Registry Refresher, made by @caprenter and @Bjwebb.

The new, much faster code was contributed by @notshi and @xriss.

Contributing

Feel free to raise an issue or send a pull request.