Metric recording built documentation size

[Nemo157]

While discussing rust-lang/rust#115718 I had a thought that being able to see historical "typical" documentation sizes could be useful, and docs.rs might be a good place to gather that.

It'll likely be a pretty varying metric depending on what crates are being built each day, maybe too variable to be useful, but I think it should be pretty easy to add and then we could see whether it seems to give useful data.

[camelid]

It'd be important to record the median in particular since it's less susceptible to outliers. Percentiles could be useful too.

[Nemo157]

Yeah, I was thinking recording as a histogram, we've been recording build durations that way for a while now which can give some idea of the sort of volatility from the different sorts of crates that get published each day (25th, 50th, 75th, 90th, 95th, 99th percentile lines):

[syphar]

What's the use-case for this? It sounds useful.

Would "documentation size" rather be the amount of files? Megabytes? all targets?

Depending on how exact / how often / how far back to the history we need the numbers, we could also store it in the database.

[camelid]

Rustdoc recently had a regression (rust-lang/rust#115718) where the docs storage size of certain crates ballooned (from 27 MB to around 500 MB). So for the documentation size, perhaps it'd be the documentation size across all targets, divided by the number of targets for that crate? Storing the data in the database would probably be useful.

[syphar]

So, some thoughts here (aka, partial mentoring instructions).

• when running the build, we have all the documentation for all targets in one folder. It might be possible to figure out target-specific doc sizes from that, though it's tricky for the default target which doesn't have a subfolder.
• we have crates with a fairly big documentation (from what I remember stm32ral sums up to around 4 GiB compressed, with several million files, so we should perhaps at least benchmark the added build time to calculate sizes for the whole crate, or per target.
• depending on the use-case I could imagine that just tracking size for all targets together is good enough. But also, when we start tracking some data it might also be useful to directly start per-target. Or the average per target?

generally we could store this in the database per release, and/or report it to prometheus to be able to add it to our dashboards.