Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Real time modeling with new materialization options #41

Open
leonard-henriquez opened this issue Aug 26, 2023 · 3 comments
Open

Real time modeling with new materialization options #41

leonard-henriquez opened this issue Aug 26, 2023 · 3 comments
Labels
category:models Related to the models in the package. priority:low Not on the roadmap. status:needs_triage Needs maintainer triage. type:enhancement New features or improvements to existing features.

Comments

@leonard-henriquez
Copy link

Is your feature request related to a problem? Please describe.

It's hard to have always update to date data with low latency (<1 minute) with this package.
This package doesn't provide a solution or documentation how to achieve this.

Describe the solution you'd like

Would it be possible to add support for other materialization option in the snowplow-web dbt package to achieve real-time transformation ?
In Snowflake Dynamic tables could provide a more efficient way to maintain near-real-time data, avoiding the need for frequently running the models.

Additional context

Data Warehouse: Snowflake (but similar concepts exist in other data warehouse)

Are you interested in contributing towards this feature?

While I'm relatively new to dbt, I'd be open to contributing towards this feature with some guidance to get started.

@leonard-henriquez leonard-henriquez added the type:enhancement New features or improvements to existing features. label Aug 26, 2023
@github-actions github-actions bot added the status:needs_triage Needs maintainer triage. label Aug 26, 2023
@rlh1994
Copy link
Contributor

rlh1994 commented Aug 29, 2023

For anyone landing here - here is a thread on the issue: https://discourse.snowplow.io/t/data-modeling-in-real-time/8978

This is something we will likely look into but it's not currently on the immediate road map.

@rlh1994 rlh1994 added priority:low Not on the roadmap. category:models Related to the models in the package. and removed status:needs_triage Needs maintainer triage. labels Aug 29, 2023
@rlh1994
Copy link
Contributor

rlh1994 commented Aug 29, 2023

Also linked to this issue snowplow/dbt-snowplow-utils#136

@miike miike transferred this issue from snowplow/dbt-snowplow-web Apr 15, 2024
@github-actions github-actions bot added the status:needs_triage Needs maintainer triage. label Apr 15, 2024
@miike
Copy link

miike commented Apr 15, 2024

Moving this issue to the unified repo as the web model is in maintenance mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:models Related to the models in the package. priority:low Not on the roadmap. status:needs_triage Needs maintainer triage. type:enhancement New features or improvements to existing features.
Projects
None yet
Development

No branches or pull requests

3 participants