Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Begin to phase out use of event_json, rejections and state_events #9413

Closed
wants to merge 5 commits into from

Conversation

richvdh
Copy link
Member

@richvdh richvdh commented Feb 15, 2021

The idea here is to collect all the stuff that is commonly selected together into one table, rather than having it scattered around the filesystem.

I benchmarked the new _fetch_event_rows query based on 100 queries I pulled out of my slow query log. On my I/O constrained server, it reduced the average time taken to fetch a batch of events from 9.7 seconds to 5.4 seconds.

Unfortunately I don't have good metrics for how much storage increased by, but I don't think it was that bad (like 2%).

Obviously, we will need a background update to populate the new columns. I'm mostly floating this to gather comments at this stage.

Based on #9411.

@richvdh richvdh force-pushed the rav/kill_event_json branch 2 times, most recently from 3db500e to 955945e Compare February 16, 2021 10:22
@ShadowJonathan
Copy link
Contributor

I think this is a good idea, when i was poking at the database to find out where all events are stored, i was a little confused why there was event_json and events.

+1 to the idea of "background updates", though maybe communicate that better in changelogs, keep it around for a while, and then move the logic to a database "delta" script (so for larger version jumps, it happens on startup). Probably not the best experience to those servers who havent migrated from a version that did that background migration, but i think then it'll be pretty smooth.

@richvdh
Copy link
Member Author

richvdh commented Oct 19, 2021

I think I'm probably not going to take this forward in its current form.

@richvdh richvdh closed this Oct 19, 2021
@richvdh richvdh deleted the rav/kill_event_json branch April 6, 2022 12:52
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants