Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make log_time column implicit and potentially introduce ingest_time #1891

Open
Tracked by #1619
, #1898
...
teh-cmc opened this issue Apr 18, 2023 · 1 comment
Open
Tracked by #1619
, #1898
...

Make log_time column implicit and potentially introduce ingest_time #1891

teh-cmc opened this issue Apr 18, 2023 · 1 comment
Labels
📉 performance Optimization, memory use, etc ⛃ re_datastore affects the datastore itself

Comments

@teh-cmc
Copy link
Member

teh-cmc commented Apr 18, 2023 •
edited
Loading

We currently store the logging time twice: once in the MsgId (soon EventId) and once injected by the SDK (and they don't even match!).
We could just not inject the log_time, and instead derive a log_time column on the server using the timestamp in the EventId; especially since we probably want an auto-derived ingestion_time anyway.
The timestamp in EventId is not going away: it is what defines our global ordering!
@teh-cmc teh-cmc mentioned this issue Apr 18, 2023
Closed
18 tasks
@teh-cmc teh-cmc changed the title Drop log_time Make log_time column implicit and potentially introduce ingest_time Apr 18, 2023
@teh-cmc teh-cmc added ⛃ re_datastore affects the datastore itself 📉 performance Optimization, memory use, etc labels Apr 18, 2023
@teh-cmc teh-cmc mentioned this issue Apr 18, 2023
Closed
23 tasks
@emilk
Copy link
Member

emilk commented May 2, 2023

This sounds like two separate issues to me.

Getting rid of log_time is a space-optimization, but adds complexity.

Adding ingest_time is a new feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
📉 performance Optimization, memory use, etc ⛃ re_datastore affects the datastore itself
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@emilk @teh-cmc