Releases: RunLLM/aqueduct
Releases · RunLLM/aqueduct
v0.0.7
Released on 8/1/2022
Enhancements
- Upgrades to go-chi v5.
- Removes need to provide API key and server address when running client and server on same machine.
- Adds support for operators with no input data.
Bugfixes
- Fixes bug where imported functions were not executed correctly.
- Improves CSV upload UI to make data preview accurate and more legible.
- Fixes bug where requirements.txt was not consistently used.
- Fixes bug where bottom sidesheet and DAG viewer were misaligned and improperly sized.
All Changes
All commits
- Allows user to specify S3 credentials for Airflow integration by @saurav-c in #250
- Adds workflow_dag engine_config column by @saurav-c in #209
- Update SDK's S3 integration api documentation by @cw75 in #252
- Fix a bug where we couldn't execute imported functions by @likawind in #259
- Add docs and Slack links to top of README by @vsreekanti in #261
- [backend] Upgrade go chi to v5 by @likawind in #260
- [UI] Show workflows associated with an integration in integration details page by @likawind in #254
- Add exec state to get_artifact_result by @kenxu95 in #253
- Add warning color to DAG in UI by @kenxu95 in #245
- [ENG-1412, 1413, 1414] Fixes CSV upload modal bugs by @vsreekanti in #262
- ENG- 1418 export spiraldemocard and sqlservercard by @agiron123 in #256
- [Bugfix] Fix bug where requirements.txt was not installing by @kenxu95 in #264
- [ENG-1354, 1415] Fixes bottom sidesheet and ReactFlow alignment issues by @vsreekanti in #267
- Update Integration Test Timeout by @agiron123 in #269
- Returns ObjDoesNotExist error for S3 storage implementation by @saurav-c in #271
- [ENG-1459] Removes need to provide API key, server address when running locally by @vsreekanti in #272
- Support operator with no input by @cw75 in #265
- Rename GetWorkflowTables -> GetWorkflowObjects by @eunice-chan in #239
- Bump version number and changelog for release 0.0.7 by @cw75 in #279
v0.0.6
Released on 7/25/2022
Enhancements
- Prints error message as part of preview execution stack trace, not above it.
Bugfixes
- Fixes bug where parameters argument to
head
function was unused. - Fixes bug where menu sidebar didn't link to home page.
- Fixes bug where operator zipfiles weren't cleaned up after workflow creation.
- Fixes bug where S3 connection listed all objects in bucket, causing connection to be extremely slow.
- Fixes bug where error and warning checks aren't properly distinguished.
All Changes
All commits
- Performs Code Structure/Naming changes for Orchestration Refactor by @kenxu95 in #207
- Adds UI for connecting Airflow integration by @saurav-c in #140
- Adds backend support for connecting Airflow integration by @saurav-c in #141
- Fixing embarrassing bug by @jegonzal in #220
- Rmeove unnecessary files from parcel bundle by @Boyuan-Deng in #219
- Refactors the orchestration engine to use new dag, operator, artifact, and orchestration interfaces. by @kenxu95 in #213
- Reenable gci in golangci linter by @kenxu95 in #224
- Eng 1345 create api endpoint to get tables associated for workflow_id by @eunice-chan in #216
- Fix bug where parameters argument to head() was unused. by @kenxu95 in #225
- ENG-1401: Fixes menu sidebar link to homepage by @vsreekanti in #230
- Updates example notebooks based on user feedback by @vsreekanti in #229
- [Server] Create server API endpoint to fetch operators associated with integration by @likawind in #208
- [Quality of life] Print the error context in the preview execution exception by @kenxu95 in #232
- Refactors python operator entrypoints by @saurav-c in #226
- Cleanup unzipped function directories properly on OSX by @kenxu95 in #233
- Fix logging for workflow name / type in executor code. by @likawind in #234
- Add ability to distinguish between warnings and error level checks on backend by @kenxu95 in #227
- Eng 1427 fix database integration tests by @saurav-c in #235
- Switch to a low-overhead way of authenticating S3 integration by @cw75 in #236
- Revert "Rmeove unnecessary files from parcel bundle" by @Boyuan-Deng in #243
- Disables connecting to Airflow from UI by @saurav-c in #244
- ENG-1430 Fix UI Build in install_local.py by @agiron123 in #237
- ENG-1407 Allow UI page sidebars to be passed in as prop. by @agiron123 in #242
- Fix error message for error severity check not passing by @kenxu95 in #246
- modify install_local scripts to only select necessary UI files by @Boyuan-Deng in #247
- Bump version number and update changelog for release v0.0.6 by @cw75 in #249
v0.0.5
Released on 7/14/2022
Enhancements
- Makes password optional when creating a Postgres connection.
- Adds
describe
method to every relational integration. - Improves log capture when executing user functions.
- Enables configuration of S3 storage backend for version snapshots and operator metadata.
- Displays workflow ID on workflow settings modal.
- Adds ability to fetch an individual artifact from a workflow run using the SDK.
- Supports reading multiple S3 files into a single Pandas DataFrame.
- Deprecates showing
pyplot
image in notebook on workflow creation; instead, provides link to UI.
Bugfixes
None! 🎉
All Changes
All commits
- Updates churn example to use correct port for UI by @hsubbaraj-spiral in #200
- Make password optional for the Postgres integration by @eunice-chan in #202
- ENG-868-Add
describe()
method to each of the relational integrations. by @Fanjia-Yan in #201 - Improve the linter script to auth-sort Python imports by @cw75 in #205
- Fixes run linter scripts when running commands by @likawind in #206
- [Python executor] Improve logging for user function execution by @likawind in #144
- Adds aqueduct storage command to CLI by @saurav-c in #193
- add workflow id to workflow setting by @Boyuan-Deng in #188
- Direct user UI instead of pyplot dag image by @Boyuan-Deng in #187
- Eng 1241 Add ability to fetch an artifact from an existing flow version by @Fanjia-Yan in #157
- Adds default values for Python connectors for Postgres.port and Extract.query_is_usable by @saurav-c in #210
- Support multi file inputs from S3 integration by @cw75 in #211
- Update README with latest tagline by @vsreekanti in #217
- Bump release version to 0.0.5 by @cw75 in #215
- Fix release 0.0.5 changelog typo by @cw75 in #218
v0.0.4
Released on 7/7/2022
Key Features
- Workflows can now have custom parameters! A workflow can have any numbers of parameters which can be used in Python operators or SQL queries. See here for more details.
Enhancements
- Add SDK support for fetching and pretty-printing workflow and workflow run metadata.
- Hide success notifications by default to avoid repetitive notifications.
- Allow for custom port selection for the Postgres integration.
- Allow requirements.txt to be set on an operator-by-operator basis.
- Add ability to copy SDK initialization snippet from account page.
- Allows metrics to be integers in addition to floats.
- Adds syntax candy for
head
onTableArtifact
s.
Bugfixes
- Fix bug that showed undefined in search bar when data search returned empty results.
- Fix bug where integration passwords were shown in plaintext on request headers.
- Fix bug where schema metadata was improperly persisted.
- Fix bug that disallowed non-CSV file uploads.
- Fix bug that caused unnecessary repetitive calls to the DAG render API.
- Fix a number of minor UI bugs -- margins, button placement, etc.
- Deprecates use of ipynbname in the SDK, which prevented the SDK from running in some notebook environments.
All Changes
All commits
- Fixes error handling in get_apikey by @vsreekanti in #130
- [SDK] Builds ability to fetch and inspect an existing flow in the system by @kenxu95 in #115
- Fix the automated notebook tests by @kenxu95 in #131
- Add list method for sys metrics by @HarisChoudhary in #137
- Add example for preset/system metrics in notebook by @HarisChoudhary in #134
- UI-finished implementation to hide success notification by @Boyuan-Deng in #94
- Eng 1256 refresh table list in integration by @eunice-chan in #133
- [UI] Fix data page search bar bug showing 'undefined' and not cancel properly by @likawind in #139
- Eng-1267-append-dag-result-id-to-workflow-id-page0-url by @Boyuan-Deng in #135
- Eng 1239 create pull request template by @cw75 in #138
- Move pull request template file location by @cw75 in #142
- Remove the unneeded header section in the PR template by @cw75 in #143
- Eng 1185 UI build by @agiron123 in #132
- Show parameters correctly on the UI by @kenxu95 in #125
- Update README for ui development. by @agiron123 in #146
- Obfuscate password in header for integration config by @HarisChoudhary in #118
- Eng 943 update getartifactversionsroute to include checks by @eunice-chan in #95
- ENG-1271 Show user a message saying that we do not support previewing base data for an S3 integration by @eunice-chan in #151
- Add executor files to integration tests server logs by @HarisChoudhary in #150
- [Backend] Adds Parameterization to SQL Queries by @kenxu95 in #152
- Implement a script to run all linters by @cw75 in #149
- Adds UI input to capture custom port for Postgres integration by @eunice-chan in #156
- Update pull_request_template.md by @eunice-chan in #163
- Creates pre-merge integration test check by @hsubbaraj-spiral in #155
- Fix bug where the schema metadata was written improperly by @kenxu95 in #166
- Eng 1104 Allow users to specify function-specific requirements by @Fanjia-Yan in #128
- Changes branch of reused workflow to main by @hsubbaraj-spiral in #164
- [SDK] Adds consistent parameter validation by @kenxu95 in #158
- Use absolute file path to serve UI files by @cw75 in #165
- Adds integration test step to dev workflow in contribution guide by @hsubbaraj-spiral in #170
- ENG-1281 Fix IntegrationFileUploadField to be usable for csv & other filetypes by @eunice-chan in #145
- Remove unnecessary triggers and reporting from integration tests by @kenxu95 in #169
- Changes the concurrency of integration tests to n=1 by @hsubbaraj-spiral in #173
- Follow up to Eng 1305 by @hsubbaraj-spiral in #174
- cleans logic to avoid unnecessary api call to dagPosition route by @Boyuan-Deng in #92
- Simplifies README and moves example to top level by @vsreekanti in #172
- Eng 1181, Eng 1281, Eng 1209 move download of binaries to setup.py by @eunice-chan in #167
- adds copy button to sdk initialization … by @Boyuan-Deng in #136
- fixed "report issue" button weird margin by @Boyuan-Deng in #179
- Eng 1181 move download of binaries to pip install (followup) by @eunice-chan in #180
- make-archive-all-button-visible-at-all-time by @Boyuan-Deng in #178
- Skips ci on non-code changes by @hsubbaraj-spiral in #175
- Follows up 1306 to change workflow branch to main by @hsubbaraj-spiral in #183
- Fixes bug in the pre-merge-ci config by @hsubbaraj-spiral in #185
- Allow the update script to update components separately by @cw75 in #171
- [UI] Add ability to trigger a flow with custom parameters by @kenxu95 in #181
- Adds periodic integration tests by @hsubbaraj-spiral in #182
- Write demo notebook for workflow parameterization by @kenxu95 in #186
- Updates periodic integration test by @hsubbaraj-spiral in #189
- Improves usage of ci tests by not relying on github paths by @hsubbaraj-spiral in #190
- Fix restrictive metric typing by @eunice-chan in #192
- adding live demo example by @jegonzal in #162
- ENG 1297: Set up build to package UI for sagemaker by @vsreekanti in #176
- Deprecate the use of ipynbname package in our SDK by @cw75 in #184
- Add a
head
function to Table artifacts by @Fanjia-Yan in #191 - Fix link in parameterization demo notebook by @kenxu95 in #196
- Move setup back to aqueduct start by @eunice-chan in #197
- Fix bugs in README script from set up call by @vsreekanti in #199
- Update install script to properly install for SageMaker instances and bump release numbers by @cw75 in #203
- Updates changelog for v0.0.4 release by @vsreekanti in #204
v0.0.3
Released on 6/21/2022
Key Features
- View what tables are present in an integration by clicking on the integration in the UI.
- View all data artifacts created by Aqueduct on the
/data
page on the UI. - Add support for pre-defined metrics and checks, including lower & upper bounds and equality checks.
- Implement support for capturing low-level metrics, such as compute time, CPU usage, and memory usage, on a per-operator basis.
Enhancements
- API keys can now be retrieved from the SDK if running on the same machine as the Aqueduct server with
aqueduct.get_apikey()
. - Add feature to automatically search for next available port when port 8080 is occupied.
- Users can upload custom data to the Aqueduct demo DB — navigate to the integrations page, click on the Aqueduct Demo database, and hit “Add CSV”.
- Allow users to optionally specify HTTP/S prefix when creating Aqueduct API client.
- Implements support for creating checks via Great Expectations.
- Simplifies notifications interface by reducing redundant text.
Bugfixes
- Fix bug where logs directory didn’t exist on upgraded installations.
- Fix bug where account page wasn’t previously being displayed.
All Changes
All commits
- Adds Changelog and updates for v0.0.2 release by @vsreekanti in #76
- Fix pwd in order to successfully read README by @cw75 in #75
- Adds contributors to Changelog by @vsreekanti in #77
- Replace mentions of
sp_client
withclient
by @kenxu95 in #66 - Fix bug where the logs directory does not exist so executor always fails by @kenxu95 in #81
- Eng-1218 Updated job.launch to add workflow retention job by @hsubbaraj-spiral in #70
- Wipe binaries in
bin
before copying over newer versions to avoid mismatching Mac signature by @cw75 in #84 - Add issue templates by @vsreekanti in #86
- Add CODE_OF_CONDUCT.md by @vsreekanti in #85
- Adds logo with colored background to README by @vsreekanti in #79
- Marks deprecated commands as deprecated in argparse help by @vsreekanti in #78
- Port over Python linter Github Action by @kenxu95 in #63
- Fix inflexible sleep in favor of a poll for Github Action integration tests by @kenxu95 in #73
- Fix bug in setup.py by @kenxu95 in #87
- Draft OSS contribution guide by @cw75 in #67
- Eng 1204 add sdk command to get local host apikey by @Fanjia-Yan in #72
- Eng 1180 automatically find the next available port for the server by @cw75 in #88
- Fix server port parsing bug in CLI by @cw75 in #89
- Port Unit Test Github Actions over by @kenxu95 in #65
- Eng 1237 update install script to update UI by @cw75 in #90
- Fix integration test bug where the aqueduct server doesn't start by @kenxu95 in #91
- add requirement delete unit test by @Fanjia-Yan in #93
- Eng 657 frontend allow users to view what tables by @eunice-chan in #55
- Allow users to realize artifacts with custom parameters by @kenxu95 in #96
- Improves README code example based on feedback by @vsreekanti in #99
- Add ability to trigger existing flows with different parameters by @kenxu95 in #98
- Remove parameter operator from being displayed in _show_dag() by @kenxu95 in #101
- Eng 1111 fix bug when using oss sdk in python by @eunice-chan in #105
- Fix broken integration tests by @kenxu95 in #106
- Fixes schema migrator create cmd by @saurav-c in #108
- eng-1247 account page by @eunice-chan in #110
- Eng 1086 create syntax sugar for sql queries by @Fanjia-Yan in #103
- Fix bug with incorrect spec type by @hsubbaraj-spiral in #113
- Add list methods for preset check/metrics by @HarisChoudhary in #112
- Implement Backend for System Metrics by @HarisChoudhary in #100
- Implement system metrics for SDK by @HarisChoudhary in #102
- Add migrator for system metrics schema change by @HarisChoudhary in #107
- Fix build error for parcel/css on m1 mac. by @agiron123 in #109
- Handle cases where the user supplies an http(s) prefix when initializing the client by @kenxu95 in #117
- UI-1035-reduce text in notification by @Boyuan-Deng in #30
- Report to our slack channel when tests or notebooks fail by @kenxu95 in #116
- Fixing metric in example code by @jegonzal in #114
- Add ability to create checks on table artifacts from Expectations by @HarisChoudhary in #104
- Eng 708 investigate how to list tables for by @eunice-chan in #111
- Make migrator when installing locally by @HarisChoudhary in #119
- @aqueducthq/common: export types for use in app and enterprise edition. by @agiron123 in #120
- Fix url typo by @eunice-chan in #121
- Remove checkbox selection by @eunice-chan in #122
- Parcel Build dist prefixing. by @agiron123 in #124
- Bump version number to 0.0.3 throughout our codebase by @cw75 in #126
- Revert the hack where we rely on sed by @cw75 in #127
- Adds v0.0.3 changelog by @vsreekanti in #129
v0.0.2
Released on 6/9/2022
Enhancements
- Allows users to start both the backend server and UI with
aqueduct start
- Removes NextJS from UI project, reverting to a vanilla React app packaged with Parcel
- Removes need for users to have
npm
installed by serving the UI from the same server as the backend - Automatically ships common library as a transpiled module, removing need for explicit transpilation
- Allows users to retrieve package version by running
aqueduct version
- Only binds server to
localhost
by default, removing requirement for firewall permissions - Improves the thread safety of the job manager
- Allow users to execute annotated functions by calling
fn.local(args)
Bugfixes
- Fixes incorrect use of
typing
library for Python3.7 - Fixes inconsistency in DAG rendering which would previously cause page load jitter
- Fixes bug where bounds on metrics were mislabeled
Contributors
v0.0.1
Initial release of the Aqueduct Prediction Infrastructure.