Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENG-1358 Set up backend integration tests & include in Github Actions #214

Conversation

eunice-chan
Copy link
Contributor

@eunice-chan eunice-chan commented Jul 14, 2022

Describe your changes and why you are making these changes

Suite of tests for pure endpoint testing. Checking reads in a stricter fashion.

Testing

Successfully ran Integration Tests Github Actions workflow with these changes: https://github.com/aqueducthq/aqueduct/actions/runs/2667299271

Related issue number (if any)

Eng 1358
Eng 1345 (#216): The changes being tested

Checklist before requesting a review

  • I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
  • I have performed a self-review of my code.
  • I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
  • [N/A] If this is a new feature, I have added unit tests and integration tests.
  • I have run the integration tests locally and they are passing.
  • I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
  • [N/A] All features on the UI continue to work correctly.
  • Added one of the following CI labels:
    • run_integration_test: Runs integration tests
    • skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)

Sorry, something went wrong.

@eunice-chan eunice-chan requested a review from kenxu95 July 14, 2022 00:03
@eunice-chan
Copy link
Contributor Author

Hi @kenxu95 made an initial draft of the backend integration test suite. Had some questions & would appreciate your feedback:

  1. How to handle if workflows already exist? e.g. try to create workflow_1 but already exists and overwrites existing. Workflow_dag check won't work because will have extra history from the existing workflow_1. I'm thinking we can have an account specific for testing. Not sure if we support multiple users in the OSS version.
  2. Could I grab the address and apikey from something? E.g. the .aqueduct config.yml file for apikey. Should I?
  3. Not really sure how to add to Github Actions.

@kenxu95
Copy link
Contributor

kenxu95 commented Jul 14, 2022

  1. So you should be able to assume that you start with a blank slate. So as long as the workflows you're creating in this suite don't collide with each other, you should be good. The SDK tests use uuid.New() to generate workflow names and avoid collisions, but for this suite we might be able to get away with better, human-readable names since all the tests will share the same workflows.
  2. You can do something similar to what the SDK integration tests do, which is grab the apikey from an env variable. See sdk/conftest.py for the details of how those integration tests were set up.
  3. You should be able to run the full suite with a pytest command right? We can just add that command to .github/workflows/integration-tests.yml

working-directory: integration_tests/backend
env:
SERVER_ADDRESS: localhost:8080
ADAPTER: http://
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the requests.get url

…ub.com:aqueducthq/aqueduct into eng-1358-set-up-backend-integration-tests-include
@eunice-chan eunice-chan added the skip_integration_test Skips required integration test (only documentation/UI changes) label Jul 14, 2022
@eunice-chan eunice-chan requested a review from likawind July 14, 2022 16:25
@eunice-chan eunice-chan marked this pull request as ready for review July 14, 2022 16:26
Copy link
Contributor

@kenxu95 kenxu95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test is really hard for me to read and understand, can we chat in person? Maybe we can pair a little :)

@eunice-chan eunice-chan requested a review from kenxu95 July 14, 2022 22:43
…ub.com:aqueducthq/aqueduct into eng-1358-set-up-backend-integration-tests-include
@eunice-chan
Copy link
Contributor Author

eunice-chan commented Jul 18, 2022

Action items:

Copy link
Contributor

@kenxu95 kenxu95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Love the documentation. A few more suggestions - just ping me when they're ready :)

try:
r = requests.get("http://"+url, headers=headers)
except:
raise Exception(f"Cannot connect to {url}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to use the same logical structure that we use in the api_client.py on the SDK? That way its 1) more consistent across our tests and 2) avoiding having nested try-except clauses which is a bit confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, should I use the use_http variable directly?
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm that might be better - how about we add a method on api_client called ui_and_server_address(), which will return the {prefix}{server_address} as a string that you can join with the routes? This'll be a useful internal method anyways - see generate_ui_url() in the SDK's utils.py

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like your suggestion of keeping the logic in one place, this just wraps it a little more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • using existing api_client._construct_full_url in tests
  • created api_client._construct_base_url
  • api_client._construct_full_url uses api_client._construct_base_url
  • made the arg use_http optional forapi_client._construct_full_url and api_client._construct_base_url (if use_http=None, use self.use_http)
  • refactor generate_ui_url to use api_client._construct_full_url
  • clean url in api_client so SERVER_ADDRESS=localhost:8080/ works (would error out before)

@eunice-chan eunice-chan requested a review from kenxu95 July 19, 2022 22:24
Copy link
Contributor

@kenxu95 kenxu95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Erroneous package-lock.json changes? I can stamp after those are fixed. Also dropped a few simplifying suggestions.

Copy link
Contributor

@kenxu95 kenxu95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for powering through this!

@eunice-chan eunice-chan merged commit 7cf2d1c into eng-1345-api-endpoint-to-get-tables-associated Jul 19, 2022
eunice-chan added a commit that referenced this pull request Jul 19, 2022
…#216)

* Add route url

* first draft

* Draft 2 -- custom sql

* SQL return all operators

* distinct load list

* lint

* Custom response struct

* package-lock update

* lint

* table_artifact merge conflict

* lint

* black

* Right version of black

* ENG-1358 Set up backend integration tests & include in Github Actions (#214)
@vsreekanti vsreekanti deleted the eng-1358-set-up-backend-integration-tests-include branch July 20, 2022 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip_integration_test Skips required integration test (only documentation/UI changes)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants