Not too long ago, Turquoit#3987 from the Stable Diffusion Discord server shared a text file containing links to every image generated by the bot from August 6th (its inception) to approximately August 15, 2022. This repository contains some code to convert that file to a SQLite database for easy querying. A prebuilt ImageLinks.db file is provided here (3.2gb)
Structure of the database is two tables:
prompts, with a primary keyidand a stringpromptdatawhich has apromptIdand aurl, among extra metadata columns
The converter is written using Node.js and better-sqlite3, and took 162 seconds to convert on my desktop.
There is also an implementation with Bun, but it doesn't work for me due to a bug. Were it to work, it would run over 3x faster than the node implementation.
To run, you need an extracted copy of ImageLinks.zip (2.5gb). Then you can run bun convert.ts or node convert.mjs.
Maybe in the future, a web app / public web api will be made to query the database without storing a local copy. I could do this but I wouldn't be able to fund a server to hold the database (ideally use postgres instead of sqlite).
- Add indexes to the SQLite database, decreasing query times of "all images for a prompt id" from 8000ms to 3ms on my machine
- Initial Publication