Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SNOW-33] New ingest to projectsettingsnapshots table #62

Merged
merged 3 commits into from
Jun 19, 2024
Merged

[SNOW-33] New ingest to projectsettingsnapshots table #62

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9375413
New ingest to projectsettingsnapshots table
jaymedina Jun 18, 2024
23fe1de
Fix typo. Newline.
jaymedina Jun 19, 2024
37d455b
Add a comma
jaymedina Jun 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions synapse_data_warehouse/synapse_raw/tables/V2.20.0__ingest_into_projectsettingsnapshots.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
USE SCHEMA {{database_name}}.synapse_raw; --noqa: JJ01,PRS,TMP
USE WAREHOUSE COMPUTE_MEDIUM;

COPY INTO
PROJECTSETTINGSNAPSHOTS
FROM (
SELECT
$1:change_timestamp as change_timestamp,
$1:change_type as change_type,
$1:change_user_id as change_user_id,
$1:snapshot_timestamp as snapshot_timestamp,
$1:id as id,
$1:concrete_type as concrete_type,
$1:project_id as project_id,
$1:settings_type as settings_type,
$1:etag as etag,
$1:locations as locations,
NULLIF(
regexp_replace(
METADATA$FILENAME,
'.*projectsettingsnapshots\/snapshot_date\=(.*)\/.*',
'\\1'),
'__HIVE_DEFAULT_PARTITION__'
) as snapshot_date
from
@{{stage_storage_integration}}_stage/projectsettingsnapshots --noqa: TMP
)
pattern='.*projectsettingsnapshots/snapshot_date=.*/.*'
FORCE=TRUE
;