Skip to content

Commit 46be7e0

Browse files
Jesus89Jesus89
authored
release: 2023-01-30 (#383)
1 parent 773b3d2 commit 46be7e0

File tree

412 files changed

+1637
-1549
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

412 files changed

+1637
-1549
lines changed

‎.github/workflows/bigquery.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ jobs:
9191
- name: Run deploy
9292
run: |
9393
cd clouds/bigquery
94-
make deploy diff="$GIT_DIFF"
94+
make deploy diff="$GIT_DIFF" production=1
9595
9696
deploy:
9797
if: github.ref_name == 'stable'

‎.github/workflows/databricks.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ jobs:
107107
DB_TOKEN: ${{ steps.secrets.outputs.databricks-ci-token }}
108108
run: |
109109
cd clouds/databricks
110-
make deploy
110+
make deploy production=1
111111
112112
publish:
113113
needs: test

‎.github/workflows/postgres.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ jobs:
8181
- name: Run deploy
8282
run: |
8383
cd clouds/postgres
84-
make deploy diff="$GIT_DIFF"
84+
make deploy diff="$GIT_DIFF" production=1
8585
8686
publish:
8787
if: github.ref_name == 'stable'

‎.github/workflows/snowflake.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -77,10 +77,10 @@ jobs:
7777
- name: Run deploy
7878
run: |
7979
cd clouds/snowflake
80-
make deploy diff="$GIT_DIFF"
80+
make deploy diff="$GIT_DIFF" production=1
8181
8282
deploy-share:
83-
if: github.ref_name == 'master'
83+
if: github.ref_name == 'stable'
8484
needs: test
8585
runs-on: ubuntu-20.04
8686
timeout-minutes: 20

‎CHANGELOG.md

+6
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ CARTO Analytics Toolbox Core.
44

55
All notable commits to this project will be documented in this file.
66

7+
## 2023-01-30
8+
9+
- fix(bq,sf): use extended toBeCloseTo in tests (#381)
10+
- docs: adapted docs to gitbook (#380)
11+
- docs: remove additional examples from the reference (#382)
12+
713
## 2022-12-27
814

915
- docs(bq,sf|h3): add H3 INT/STRING functions reference (#369)

‎CONTRIBUTING.md

+138
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# How to contribute?
2+
3+
## Pull requests
4+
5+
Every change in the Analytics Toolbox must be included in a pull request. It will be merged by squashing the commits, resulting in one commit against `main` that uses the pull request name and number in the commit description.
6+
7+
The pull request must have a good name and description. We recommend following [Conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) for the naming, which helps have a more structured and descriptive git tree.
8+
9+
These are the main types to be used: `feat`, `fix`, `doc`, `test`, `chore`, and `refactor`, but others can also be used if needed.
10+
11+
The scope is included to refer to the cloud or clouds (`bq`, `sf`, `rs`, `pg`, `db`), and the module or modules affected (`h3`, `lds`, etc.). The format of the scope will be as follows:
12+
13+
```
14+
(<cloud(s)>|<module(s)>)
15+
```
16+
17+
Here there are some examples:
18+
19+
```
20+
feat(bq,sf,rs,pg|quadbin): add quadbin/quadkey conversion functions
21+
```
22+
23+
```
24+
fix(sf|lds): decrease isolines batch size
25+
```
26+
27+
## Documentation
28+
29+
The Analytics Toolbox repositories include all the SQL references for each function/procedure in all the modules. This is the source data for the public SQL reference in the documentation: https://docs.carto.com/data-and-analysis/analytics-toolbox-overview.
30+
31+
### Structure
32+
33+
This is the structure of a generic doc folder:
34+
35+
```
36+
clouds/<cloud>/modules/doc/<module>/
37+
- _INTRO.md
38+
- FUNCTION_A.md
39+
- PROCEDURE_B.md
40+
```
41+
42+
The language for documentation is Markdown extended with new metadata and special markers. Here is a guide to contributing to the documentation:
43+
44+
**Introduction**
45+
46+
The file *_INTRO.md* contains the introduction of the module's documentation. The template could vary for each cloud provider.
47+
48+
It allows passing a yml-like metadata header with the following information:
49+
- badges: core, advanced, beta, etc.
50+
- order (optional): list of the functions/procedures. By default, it sorts by alphabetical order.
51+
52+
```md
53+
---
54+
badges:
55+
- advanced
56+
- beta
57+
order:
58+
- PROCEDURE_B
59+
- FUNCTION_A
60+
---
61+
62+
# module
63+
64+
Description of the module.
65+
```
66+
67+
**Function**
68+
69+
````md
70+
## FUNCTION_A
71+
72+
```sql:signature
73+
carto.FUNCTION_A(param_a, param_b)
74+
```
75+
76+
**Description**
77+
78+
Description of the function.
79+
80+
* `param_a`: `STRING` description of param a.
81+
* `param_b` (optional): `INT64` description of param b.
82+
83+
**Return type**
84+
85+
`FLOAT64`
86+
87+
**Example**
88+
89+
```sql
90+
SELECT carto.FUNCTION_A('a', 123);
91+
-- 1234.0
92+
```
93+
````
94+
95+
**Procedure**
96+
97+
````md
98+
## PROCEDURE_B
99+
100+
```sql:signature
101+
carto.PROCEDURE_B(param_a, param_b)
102+
```
103+
104+
**Description**
105+
106+
Description of the procedure.
107+
108+
* `param_a`: `STRING` description of param a.
109+
* `param_b` (optional): `FLOAT64` description of param b.
110+
111+
**Output**
112+
113+
The result is a table with ...
114+
115+
**Example**
116+
117+
```sql
118+
CALL carto.PROCEDURE_B(
119+
'''
120+
SELECT * ...
121+
''',
122+
123
123+
);
124+
```
125+
````
126+
127+
### Extras
128+
129+
Additionally, the documentation allows hint blocks (info/warning):
130+
131+
**Hints**
132+
133+
`````
134+
````hint:info
135+
**Title**
136+
...
137+
````
138+
`````

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ The *CARTO Analytics Toolbox* is a set of UDFs and Stored Procedures to unlock S
44

55
| BigQuery | Snowflake | Redshift | Postgres | Databricks |
66
|:--------:|:---------:|:--------:|:--------:|:----------:|
7-
|<img src="https://docs.carto.com/img/icons/bigquery-analytics-toolbox.png" width=80 height=80>|<img src="https://docs.carto.com/img/icons/snowflake-analytics-toolbox.png" width=80 height=80>|<img src="https://docs.carto.com/img/icons/redshift-analytics-toolbox.png" width=80 height=80>|<img src="https://docs.carto.com/img/icons/postgres-analytics-toolbox.png" width=80 height=80>|<img src="https://docs.carto.com/img/icons/databricks-analytics-toolbox.png" width=80 height=80>|
7+
|<img src="./clouds/bigquery/common/analytics-toolbox-bigquery.png" width=80 height=80>|<img src="./clouds/snowflake/common/analytics-toolbox-snowflake.png" width=80 height=80>|<img src="./clouds/redshift/common/analytics-toolbox-redshift.png" width=80 height=80>|<img src="./clouds/postgres/common/analytics-toolbox-postgres.png" width=80 height=80>|<img src="./clouds/databricks/common/analytics-toolbox-databricks.png" width=80 height=80>|
88

99
This repo contains the core open-source modules of the toolbox. CARTO offers a set of premium modules that are available for CARTO users.
1010

‎clouds/bigquery/CHANGELOG.md

+6
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ CARTO Analytics Toolbox Core for BigQuery.
44

55
All notable commits to this project will be documented in this file.
66

7+
## [1.0.0] - 2023-01-30
8+
9+
- fix: use extended toBeCloseTo in tests (#381)
10+
- docs: adapted docs to gitbook (#380)
11+
- docs: remove additional examples from the reference (#382)
12+
713
## [0.2.0] - 2022-12-27
814

915
- docs(h3): add H3 INT/STRING functions reference (#369)

‎clouds/bigquery/README.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -81,4 +81,6 @@ make test-libraries modules=constructors test=constructors.test.js
8181
make test-libraries functions=ST_BUFFER test="transformations_buffer.test.js -t buffer"
8282
```
8383

84-
Additionally, `nodeps=1` filter can be passed to skip building and including the dependencies. It can be used to speed up re-deployments during development.
84+
The parameter `nodeps=1` filter can be passed to skip building and including the dependencies. It can be used to speed up re-deployments during development.
85+
86+
The parameter `testall=1` allows running all the tests even with fails. By default, it exits the test suite immediately upon one failing test suite.

‎clouds/bigquery/common/analytics-toolbox-bigquery.png

6.27 KB
Loading

‎clouds/bigquery/common/test-extend.js

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
const { expect } = require('@jest/globals');
2+
3+
expect.extend({
4+
toBeCloseTo (received, expected, precision = 10) {
5+
6+
function round (obj) {
7+
if (!obj) {
8+
return obj;
9+
}
10+
switch (typeof obj) {
11+
case 'array':
12+
return obj.map(round);
13+
case 'object':
14+
return Object.keys(obj).reduce((acc, key) => {
15+
acc[key] = round(obj[key]);
16+
return acc;
17+
}, {});
18+
case 'number':
19+
return +obj.toFixed(precision);
20+
default:
21+
return obj;
22+
}
23+
}
24+
25+
expect(round(received)).toEqual(round(expected));
26+
27+
return { pass: true };
28+
}
29+
});

‎clouds/bigquery/libraries/javascript/Makefile

+7-1
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,12 @@ MODULES_DIRS ?= $(ROOT_DIR)/../../modules
1313
ESLINTRC_DIR ?= $(ROOT_DIR)/../../../..
1414
COMMON_DIR = $(ROOT_DIR)/../../common
1515

16+
ifeq ($(testall),1)
17+
BAIL=
18+
else
19+
BAIL=--bail
20+
endif
21+
1622
include $(COMMON_DIR)/Makefile
1723

1824
.SILENT:
@@ -64,7 +70,7 @@ test:
6470
echo "Testing libraries..."
6571
UNIT_TEST=1 $(MAKE) build
6672
PATH="$(NODE_MODULES_DEV)/.bin/:$(PATH)" \
67-
jest --testTimeout=30000 --bail --verbose $(TEST_DIR)/$(test)
73+
jest --testTimeout=30000 $(BAIL) --verbose $(TEST_DIR)/$(test)
6874
ifdef MAKE_LIB
6975
echo "Testing extra libraries $(TEST_DIR)/$(MAKE_LIB)..."
7076
$(MAKE) -C $(TEST_DIR)/$(MAKE_LIB) test

‎clouds/bigquery/modules/Makefile

+8-1
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,12 @@ else
1919
export BQ_PACKAGE_VERSION ?= $(shell cat $(ROOT_DIR)/../version)-dev
2020
endif
2121

22+
ifeq ($(testall),1)
23+
BAIL=
24+
else
25+
BAIL=--bail
26+
endif
27+
2228
BQ_MODULE_LABEL ?= spatial_extension_module:core
2329
MODULE_PERMISSIONS_BASH ?= set_module_permissions.sh
2430

@@ -76,7 +82,8 @@ test: check $(NODE_MODULES_DEV)
7682
if [ ! -z "$$TESTS" ]; then \
7783
GOOGLE_APPLICATION_CREDENTIALS=$(GOOGLE_APPLICATION_CREDENTIALS) \
7884
PATH="$(NODE_MODULES_DEV)/.bin/:$(PATH)" \
79-
jest --testTimeout=120000 --bail --verbose --slowTestThreshold=20 --maxConcurrency=10 $$TESTS || exit 1; \
85+
jest --testTimeout=150000 $(BAIL) --verbose --slowTestThreshold=20 --maxConcurrency=10 $$TESTS \
86+
--setupFilesAfterEnv "$(COMMON_DIR)/test-extend.js" || exit 1; \
8087
OLD_TEST=$(TEST_DIR)/$$m/old-test; \
8188
if [ -d $$OLD_TEST ]; then \
8289
echo "Old tests with custon Makefile"; \

‎clouds/bigquery/modules/doc/accessors/ST_ENVELOPE.md

+4-6
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
### ST_ENVELOPE
1+
## ST_ENVELOPE
22

3-
{{% bannerNote type="code" %}}
3+
```sql:signature
44
carto.ST_ENVELOPE(geog)
5-
{{%/ bannerNote %}}
5+
```
66

77
**Description**
88

@@ -14,11 +14,9 @@ Takes any number of features and returns a rectangular Polygon that encompasses
1414

1515
`GEOGRAPHY`
1616

17-
{{% customSelector %}}
1817
**Example**
19-
{{%/ customSelector %}}
2018

21-
``` sql
19+
```sql
2220
SELECT `carto-os`.carto.ST_ENVELOPE([ST_GEOGPOINT(-75.833, 39.284), ST_GEOGPOINT(-75.6, 39.984), ST_GEOGPOINT(-75.221, 39.125)]);
2321
-- POLYGON((-75.833 39.125, -75.68 39.125 ...
2422
```

‎clouds/bigquery/modules/doc/accessors/_INTRO.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1-
## accessors
2-
3-
<div class="badges"><div class="core"></div></div>
1+
---
2+
badges:
3+
- core
4+
---
5+
# accessors
46

57
This module contains functions that provide information about internal geometries.

‎clouds/bigquery/modules/doc/clustering/ST_CLUSTERKMEANS.md

+3-10
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
### ST_CLUSTERKMEANS
1+
## ST_CLUSTERKMEANS
22

3-
{{% bannerNote type="code" %}}
3+
```sql:signature
44
carto.ST_CLUSTERKMEANS(geog, numberOfClusters)
5-
{{%/ bannerNote %}}
5+
```
66

77
**Description**
88

@@ -15,9 +15,7 @@ Takes a set of points as input and partitions them into clusters using the k-mea
1515

1616
`ARRAY<STRUCT<cluster INT64, geom GEOGRAPHY>>`
1717

18-
{{% customSelector %}}
1918
**Example**
20-
{{%/ customSelector %}}
2119

2220
```sql
2321
SELECT `carto-os`.carto.ST_CLUSTERKMEANS([ST_GEOGPOINT(0, 0), ST_GEOGPOINT(0, 1), ST_GEOGPOINT(5, 0), ST_GEOGPOINT(1, 0)], 2);
@@ -26,8 +24,3 @@ SELECT `carto-os`.carto.ST_CLUSTERKMEANS([ST_GEOGPOINT(0, 0), ST_GEOGPOINT(0, 1)
2624
-- {cluster: 0, geom: POINT(5 0)}
2725
-- {cluster: 1, geom: POINT(1 0)}
2826
```
29-
30-
{{% bannerNote type="note" title="ADDITIONAL EXAMPLES"%}}
31-
32-
* [New police stations based on Chicago crime location clusters](/analytics-toolbox-bigquery/examples/new-police-stations-based-on-chicago-crime-location-clusters/)
33-
{{%/ bannerNote %}}

‎clouds/bigquery/modules/doc/clustering/_INTRO.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
1-
## clustering
2-
3-
<div class="badges"><div class="core"></div></div>
1+
---
2+
badges:
3+
- core
4+
---
5+
# clustering
46

57
This module contains functions that perform clustering on geographies.

0 commit comments

Comments
 (0)