-
Notifications
You must be signed in to change notification settings - Fork 54
/
CHANGELOG
268 lines (178 loc) · 9.77 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
CHANGELOG
=========
See: https://github.com/ckan/ckanext-xloader/releases if this file has drifted.
2.0.0 2024-12-10
================
## Major
Dropped CKAN 2.9.x and Python2.
## Feat:
* Adds Strip White Space fields to the Data Dictionary (defualts to `True` for each field).
This will strip surrounding white space from data values prior to inserting them into the database.
* Adds support for ckanext-validation. Config `ckanext.xloader.validation.requires_successful_report`
controls whether a resource requires a successful validation report to be XLoadered.
By default, a resource would also require a Validation Schema, which can be turned off with
`ckanext.xloader.validation.enforce_schema`.
* Frontend Status Badges by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/224
## Fix:
* Properly handle REDIS queue timeouts to close/delete any temporary files.
* Fix automated PyPI publishing by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/231
## What's Changed
* Update README, migrate it to Markdown by @amercader in https://github.com/ckan/ckanext-xloader/pull/235
* chore: switch to pyproject.toml by @duttonw in https://github.com/ckan/ckanext-xloader/pull/236
* Validation Extension Support (Squashed) by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/237
* Strip White Space from Cell Values (Squashed) by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/238
* RQ Job Timeout Handling (Squashed) by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/239
* SQLAlchemy v2 support by @smotornyuk in https://github.com/ckan/ckanext-xloader/pull/225
**Full Changelog**: https://github.com/ckan/ckanext-xloader/compare/1.2.0...2.0.0
1.2.0 2024-11-21
================
## What's Changed
* Fix PyPI publishing by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/233
* Enhancement/Bugfix: Downstream qld-gov-au fix's by @duttonw in https://github.com/ckan/ckanext-xloader/pull/232
* feat: @JVickery-TBS work on validation integration (await successful validation prior to doing datastore work via 'IPipeValidation'
* fix: handle gracefully if tabulator load fails by trying 'direct load'
* fix: Excel blank header row bug
* fix: Datastore truncate, restart identity so numbering restarts from 0 again (when imported data has same columns and types
* fix: parital fix on DB deadlock by adding timeouts on DDL events
* test: test_simple_large_file, test_with_blanks, test_with_empty_lines, test_with_extra_blank_cells
* test: test_require_validation, test_enforce_validation_schema
* chore: min version requirements for cve's,
* requests>=2.32.0
* urllib3>=2.2.2
* zipp>=3.19.1
**Full Changelog**: https://github.com/ckan/ckanext-xloader/compare/1.1.2...1.2.0
1.1.1 2024-10-16
================
* feat: Add pypi cicd publish via github action via environment controls by @duttonw in https://github.com/ckan/ckanext-xloader/pull/228
**Full Changelog**: https://github.com/ckan/ckanext-xloader/compare/1.1.0...1.1.1
1.1.0 2024-10-16
================
Fixes:
* feat: Add pypi cicd publish via github action via environment controls by @duttonw in https://github.com/ckan/ckanext-xloader/pull/228
**Full Changelog**: https://github.com/ckan/ckanext-xloader/compare/1.1.0...1.1.1
1.1.0 2024-10-15
================
Fixes:
* add README note about running on separate server, #191 by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/192
* Use IDomainObjectModification Implementation by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/198
* Hide excessive numbers of resource_data log entries, #187 by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/188
* #182 Type guessing fixes by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/186
* Document the ckan.download_proxy setting, #176 by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/204
* Conditional DataStore Tab in Resource Edit by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/190
* Make locking behaviour more robust by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/205
* Delete Datastore Table Button by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/197
* Quality of life improvements by @duttonw in https://github.com/ckan/ckanext-xloader/pull/195
* Clean Datastore Tables Job by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/196
* strip extra space for column name by @mutantsan in https://github.com/ckan/ckanext-xloader/pull/210
* Skip empty lines instead of erroring by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/208
* add more options for maintainers to expedite XLoader runs, GitHub #202 by @ThrawnCA in https://github.com/ckan/ckanext-xloader/pull/212
* Add Mixed Integers Type Guessing to NUMERIC Tests by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/217
* PY2 & PY3 String/Binary Fixes by @JVickery-TBS in https://github.com/ckan/ckanext-xloader/pull/203
* In plugin.py, there is an fix of resource format key error by @Nisha1293 in https://github.com/ckan/ckanext-xloader/pull/209
* CKAN 2.11 support by @amercader in https://github.com/ckan/ckanext-xloader/pull/220
New Contributors:
* @JVickery-TBS made their first contribution in https://github.com/ckan/ckanext-xloader/pull/198
* @duttonw made their first contribution in https://github.com/ckan/ckanext-xloader/pull/195
* @mutantsan made their first contribution in https://github.com/ckan/ckanext-xloader/pull/210
* @Nisha1293 made their first contribution in https://github.com/ckan/ckanext-xloader/pull/209
**Full Changelog**: https://github.com/ckan/ckanext-xloader/compare/1.0.1...1.1.0
1.0.1 2024-04-04
================
Fixes:
* Include config_declaration.yaml into MANIFEST by @pdelboca in https://github.com/ckan/ckanext-xloader/pull/183
**Full Changelog**: https://github.com/ckan/ckanext-xloader/compare/1.0.0...1.0.1
1.0.1 2024-04-04
================
Fixes:
* Fixed date parsing while fetching entries for task_status by @muhammed-ajmal in https://github.com/ckan/ckanext-xloader/pull/179
* Drop support for old CKAN versions and add CSRF support by @pdelboca in https://github.com/ckan/ckanext-xloader/pull/180
* Refactor test_jobs.py by @pdelboca in https://github.com/ckan/ckanext-xloader/pull/181
New Contributors:
* @muhammed-ajmal made their first contribution in https://github.com/ckan/ckanext-xloader/pull/179
**Full Changelog**: https://github.com/ckan/ckanext-xloader/compare/0.12.2...1.0.0
0.9.0 2021-10-01
================
Fixes:
* Fix SQLAlchemy session exception on CKAN 2.9 #140
* Fix xloader status timestamps #141
* Fix to correctly report exceptions in stdout #141
0.8.1 2021-08-30
================
Features:
* Add ssl_verify option to callback_xloader_hook #136
Fixes:
* Fix bytes / str concat #138
* Stream request needs to be explicited closed #139
0.8.0 2021-06-11
================
Features:
* Click CLI for CKAN >= 2.9 #128
Fixes:
* Submit private datasets when using the `submit all` command #121
* Send user context to the resource patch function #134
* Add documentation for ssl_verify #135
0.7.0 2020-11-23
================
Features:
* Python 3 support #113
* CKAN 2.9 support #113
Fixes:
* Update resource hash after load to datastore #116
0.6.1 2020-05-03
================
Features:
* Add 'just_load_with_messytables' option #96
Fixes:
* When getting the resource from CKAN, it now copes with the edge case that CKAN hasn't quite added the resource yet - now it successfully retries #94
0.6.0 2020-04-27
================
Release withdrawn
0.5.0 2019-12-04
================
Features:
* migrate_types CLI command added for freezing/migrating data dictionaries created with datapusher #85
Fixes:
* DataStore tab missing from resource manage page, due to templates missing from PyPI package #74
0.4.1 2019-11-13
================
Fixes:
* populate_full_text_trigger error when doing 'datastore set-permissions' #72
* '%' in column name causes TypeError("'dict' object does not support indexing") #65
* numpy >= 1.16 causes 'RuntimeError: implement_array_function' on CKAN startup #79
* CKAN 2.9 compatibility - fix error about 'resource_revision_table' #81
v0.4.0 2019-06-21
=================
Features:
* 'ignore_hash' config option added to bypass the hash check which can skip loads #34
* File size is logged #35
* Retries are no done when downloading CSV #39
* xloader_status action now available to GET (not just POST) #46
* ANALYZE run after load, needed for CKAN 2.9 resource preview speed-up #47
* CLI load of multiple datasets now resumes following an error with a resource #57
* Added 'max_excerpt_lines' config to load an excerpt of files larger than max_content_length #63
Fixes:
* Unicode characters in CSV headers caused error #28
* Column headers longer than 63 characters caused error #14
* Floats in CSV headers caused error #49
* Integers in xlsx header caused error #53
* Extraneous commas in combination with non-ascii chars caused error #51
* CSV with no rows caused error #38
* Template compatibility with CKAN 2.9 #40
* CLI submitted resources with non-xloader formats #43
* ckanext.xloader.max_content_length wasn't working #62
* KeyError: 'resource_id' when xloading a resource #68
v0.3.1 2018-01-22
=================
* Fix for exception about 'ckan.datapusher.url' not being set when you check xload status #26
v0.3.0 2017-11-17
=================
* Trigger any resource_views to be created straight after load, rather than wait for the index.
v0.2.0 2017-11-10
=================
* Renamed ckanext-xloader
* Added to PyPI
* The user is given access to the data earlier in the job - the column indexing now occurs afterwards, since this is only an optimization for queries and takes much longer than the load itself
* Fixed exception during error-handling for files too long and of non-accepted schemes
v0.1 2017-11-03
===============
* Initial code, named ckanext-shift.