Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add dbt-cloud account list command #49

Merged
merged 5 commits into from
Apr 4, 2022
Merged

Add dbt-cloud account list command #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5f10b88
Change DbtCloudCommand no longer requires the account_id (for command…
datamie-simo Apr 4, 2022
6a6929b
Add 'dbt-cloud account list' command
datamie-simo Apr 4, 2022
d78639b
Add account_list_response.json
datamie-simo Apr 4, 2022
cb1c913
Update README
datamie-simo Apr 4, 2022
c02b082
Minor change
datamie-simo Apr 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,13 @@ jobs:
environment_count=$(cat environments.json | jq '.data | length')
[[ $environment_count > 0 ]] && exit 0 || exit 1

- name: Test 'dbt-cloud account list'
run: |
dbt-cloud account list > accounts.json
cat accounts.json | jq '.data[] | {id: .id}'
account_count=$(cat accounts.json | jq '.data | length')
[[ $account_count > 0 ]] && exit 0 || exit 1

- name: Test 'dbt-cloud metadata query'
run: dbt-cloud metadata query -f tests/data/metadata_query.graphql

Expand Down
227 changes: 145 additions & 82 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ The following environment variables are used as argument defaults:

Group | API endpoint | Command | Description |
| --- | --- | --- | --- |
| Accounts | [https://cloud.getdbt.com/api/v2/accounts/](https://docs.getdbt.com/dbt-cloud/api-v2#operation/listAccounts) | `dbt-cloud account list` | Not implemented yet |
| Accounts | [https://cloud.getdbt.com/api/v2/accounts/](https://docs.getdbt.com/dbt-cloud/api-v2#operation/listAccounts) | [dbt-cloud account list](#dbt-cloud-account-list) | Retrieves all available accounts |
| Accounts | [https://cloud.getdbt.com/api/v2/accounts/{accountId}/](https://docs.getdbt.com/dbt-cloud/api-v2#operation/getAccountById) | `dbt-cloud account get` | Not implemented yet |
| Projects | https://cloud.getdbt.com/api/v2/accounts/{accountId}/projects/ | [dbt-cloud project list](#dbt-cloud-project-list) | Returns a list of projects in the account |
| Projects | [https://cloud.getdbt.com/api/v2/accounts/{accountId}/projects/{projectId}](https://docs.getdbt.com/dbt-cloud/api-v2#operation/getProjectById) | `dbt-cloud project get` | Not implemented yet |
Expand All @@ -53,6 +53,7 @@ Group | API endpoint | Command | Description |

# Commands

* [dbt-cloud account list](#dbt-cloud-account-list)
* [dbt-cloud project list](#dbt-cloud-project-list)
* [dbt-cloud environment list](#dbt-cloud-environment-list)
* [dbt-cloud job run](#dbt-cloud-job-run)
Expand All @@ -70,6 +71,68 @@ Group | API endpoint | Command | Description |
* [dbt-cloud run get-artifact](#dbt-cloud-run-get-artifact)
* [dbt-cloud metadata query](#dbt-cloud-metadata-query)

## dbt-cloud account list
This command retrieves all available dbt Cloud accounts. For more information on the API endpoint arguments and response, run `dbt-cloud account list --help` and check out the [dbt Cloud API docs](https://docs.getdbt.com/dbt-cloud/api-v2#operation/listAccounts).

### Usage
```bash
>> dbt-cloud account list
{
"status": {
"code": 200,
"is_success": true,
"user_message": "Success!",
"developer_message": ""
},
"data": [
{
"docs_job_id": null,
"freshness_job_id": null,
"lock_reason": null,
"unlock_if_subscription_renewed": false,
"read_only_seats": 50,
"id": REDACTED,
"name": "Jaffle Shop",
"state": 1,
"plan": "team",
"pending_cancel": false,
"run_slots": 5,
"developer_seats": 4,
"queue_limit": 50,
"pod_memory_request_mebibytes": 600,
"run_duration_limit_seconds": 86400,
"enterprise_authentication_method": null,
"enterprise_login_slug": null,
"enterprise_unique_identifier": null,
"billing_email_address": "REDACTED",
"locked": false,
"develop_file_system": true,
"unlocked_at": null,
"created_at": "2021-09-06T07:41:12.146234+00:00",
"updated_at": "2022-03-07T06:05:33.350381+00:00",
"starter_repo_url": null,
"sso_reauth": false,
"git_auth_level": "team",
"identifier": "REDACTED",
"docs_job": null,
"freshness_job": null,
"enterprise_login_url": "https://cloud.getdbt.com/enterprise-login/None/"
}
],
"extra": {
"filters": {
"pk__in": [
REDACTED
]
},
"order_by": null,
"pagination": {
"count": 1,
"total_count": 1
}
}
}
```

## dbt-cloud project list
This command returns a list of projects in the account. For more information on the API endpoint arguments and response, run `dbt-cloud project list --help` and check out the [dbt Cloud API docs](https://docs.getdbt.com/dbt-cloud/api-v2#operation/listProjects).
Expand All @@ -78,97 +141,97 @@ This command returns a list of projects in the account. For more information on
```bash
>> dbt-cloud project list
{
"status": {
"code": 200,
"is_success": true,
"user_message": "Success!",
"developer_message": ""
},
"data": [
{
"name": "jaffle_shop",
"status": {
"code": 200,
"is_success": true,
"user_message": "Success!",
"developer_message": ""
},
"data": [
{
"name": "jaffle_shop",
"account_id": REDACTED,
"repository_id": REDACTED,
"connection_id": REDACTED,
"id": REDACTED,
"created_at": "2021-04-14 20:23:00.395285+00:00",
"updated_at": "2021-11-16 16:32:43.960836+00:00",
"skipped_setup": false,
"state": 1,
"dbt_project_subdirectory": null,
"connection": {
"id": REDACTED,
"account_id": REDACTED,
"repository_id": REDACTED,
"connection_id": REDACTED,
"project_id": REDACTED,
"name": "Bigquery",
"type": "bigquery",
"created_by_id": REDACTED,
"created_by_service_token_id": null,
"details": {
"project_id": "REDACTED",
"timeout_seconds": 300,
"private_key_id": "REDACTED",
"client_email": "REDACTED",
"client_id": "REDACTED",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "REDACTED",
"retries": 1,
"location": null,
"is_configured_for_oauth": false
},
"state": 1,
"created_at": "2021-11-16 16:26:01.571115+00:00",
"updated_at": "2021-11-16 16:37:42.500015+00:00"
},
"repository": {
"id": REDACTED,
"created_at": "2021-04-14 20:23:00.395285+00:00",
"updated_at": "2021-11-16 16:32:43.960836+00:00",
"skipped_setup": false,
"account_id": REDACTED,
"project_id": REDACTED,
"full_name": "REDACTED",
"remote_url": "REDACTED",
"remote_backend": "github",
"git_clone_strategy": "github_app",
"deploy_key_id": REDACTED,
"repository_credentials_id": null,
"github_installation_id": REDACTED,
"pull_request_url_template": "REDACTED",
"state": 1,
"dbt_project_subdirectory": null,
"connection": {
"id": REDACTED,
"account_id": REDACTED,
"project_id": REDACTED,
"name": "Bigquery",
"type": "bigquery",
"created_by_id": REDACTED,
"created_by_service_token_id": null,
"details": {
"project_id": "REDACTED",
"timeout_seconds": 300,
"private_key_id": "REDACTED",
"client_email": "REDACTED",
"client_id": "REDACTED",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "REDACTED",
"retries": 1,
"location": null,
"is_configured_for_oauth": false
},
"state": 1,
"created_at": "2021-11-16 16:26:01.571115+00:00",
"updated_at": "2021-11-16 16:37:42.500015+00:00"
},
"repository": {
"created_at": "2021-11-16 16:26:24.412439+00:00",
"updated_at": "2021-11-16 16:26:24.412455+00:00",
"deploy_key": {
"id": REDACTED,
"account_id": REDACTED,
"project_id": REDACTED,
"full_name": "REDACTED",
"remote_url": "REDACTED",
"remote_backend": "github",
"git_clone_strategy": "github_app",
"deploy_key_id": REDACTED,
"repository_credentials_id": null,
"github_installation_id": REDACTED,
"pull_request_url_template": "REDACTED",
"state": 1,
"created_at": "2021-11-16 16:26:24.412439+00:00",
"updated_at": "2021-11-16 16:26:24.412455+00:00",
"deploy_key": {
"id": REDACTED,
"account_id": REDACTED,
"state": 1,
"public_key": "REDACTED"
},
"github_repo": "REDACTED",
"name": "jaffle_shop",
"git_provider_id": REDACTED,
"gitlab": null,
"git_provider": null
"public_key": "REDACTED"
},
"group_permissions": [],
"docs_job_id": null,
"freshness_job_id": null,
"docs_job": null,
"freshness_job": null
}
],
"extra": {
"filters": {
"account_id": REDACTED,
"limit": 100,
"offset": 0
"github_repo": "REDACTED",
"name": "jaffle_shop",
"git_provider_id": REDACTED,
"gitlab": null,
"git_provider": null
},
"order_by": "id",
"pagination": {
"count": 1,
"total_count": 1
}
"group_permissions": [],
"docs_job_id": null,
"freshness_job_id": null,
"docs_job": null,
"freshness_job": null
}
],
"extra": {
"filters": {
"account_id": REDACTED,
"limit": 100,
"offset": 0
},
"order_by": "id",
"pagination": {
"count": 1,
"total_count": 1
}
}
}
```

## dbt-cloud environment list
Expand Down
31 changes: 22 additions & 9 deletions dbt_cloud/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
DbtCloudJobCreateCommand,
DbtCloudJobDeleteCommand,
DbtCloudJobRunCommand,
DbtCloudCommand,
DbtCloudAccountCommand,
DbtCloudRunGetCommand,
DbtCloudRunListArtifactsCommand,
DbtCloudRunGetArtifactCommand,
Expand All @@ -18,6 +18,7 @@
DbtCloudJobListCommand,
DbtCloudProjectListCommand,
DbtCloudEnvironmentListCommand,
DbtCloudAccountListCommand,
)
from dbt_cloud.demo import data_catalog
from dbt_cloud.serde import json_to_dict, dict_to_json
Expand All @@ -32,7 +33,7 @@ def execute_and_print(command, **kwargs):
return response


@click.group()
@click.group(help="The dbt Cloud command line interface.")
def dbt_cloud():
import http.client as http_client

Expand All @@ -45,27 +46,32 @@ def dbt_cloud():
http_client.HTTPConnection.debuglevel = 1


@dbt_cloud.group(help="dbt Cloud job commands")
@dbt_cloud.group(help="Interact with dbt Cloud jobs.")
def job():
pass


@dbt_cloud.group(name="run", help="dbt Cloud run commands")
@dbt_cloud.group(name="run", help="Interact with dbt Cloud job runs.")
def job_run():
pass


@dbt_cloud.group(help="dbt Cloud project commands")
@dbt_cloud.group(help="Interact with dbt Cloud projects.")
def project():
pass


@dbt_cloud.group(help="dbt Cloud environment commands")
@dbt_cloud.group(help="Interact with dbt Cloud environments.")
def environment():
pass


@dbt_cloud.group(help="dbt Cloud Metadata API commands")
@dbt_cloud.group(help="Interact with dbt Cloud accounts.")
def account():
pass


@dbt_cloud.group(help="Interact with the dbt Cloud Metadata API.")
def metadata():
pass

Expand Down Expand Up @@ -204,7 +210,7 @@ def export(file, **kwargs):


@job.command(help="Imports a dbt Cloud job from exported JSON.", name="import")
@DbtCloudCommand.click_options
@DbtCloudAccountCommand.click_options
@click.option(
"-f",
"--file",
Expand All @@ -213,7 +219,7 @@ def export(file, **kwargs):
help="Import file path.",
)
def import_job(file, **kwargs):
base_command = DbtCloudCommand.from_click_options(**kwargs)
base_command = DbtCloudAccountCommand.from_click_options(**kwargs)
job_create_kwargs = {**json_to_dict(file.read()), **base_command.dict()}
command = DbtCloudJobCreateCommand(**job_create_kwargs)
response = command.execute()
Expand Down Expand Up @@ -284,6 +290,13 @@ def list(**kwargs):
response = execute_and_print(command)


@account.command(help=DbtCloudAccountListCommand.get_description())
@DbtCloudAccountListCommand.click_options
def list(**kwargs):
command = DbtCloudAccountListCommand.from_click_options(**kwargs)
response = execute_and_print(command)


@metadata.command(help=DbtCloudMetadataQueryCommand.get_description())
@click.option(
"-f",
Expand Down
3 changes: 2 additions & 1 deletion dbt_cloud/command/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@
)
from .project import DbtCloudProjectListCommand
from .environment import DbtCloudEnvironmentListCommand
from .account import DbtCloudAccountListCommand
from .metadata import DbtCloudMetadataQueryCommand
from .command import DbtCloudCommand
from .command import DbtCloudAccountCommand
1 change: 1 addition & 0 deletions dbt_cloud/command/account/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .list import DbtCloudAccountListCommand
14 changes: 14 additions & 0 deletions dbt_cloud/command/account/list.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import requests
from dbt_cloud.command.command import DbtCloudCommand


class DbtCloudAccountListCommand(DbtCloudCommand):
"""Retrieves all available accounts."""

@property
def api_url(self) -> str:
return f"{super().api_url}/accounts/"

def execute(self) -> requests.Response:
response = requests.get(url=self.api_url, headers=self.request_headers)
return response
Loading