Skip to content

Make unattended runs possible today, and show one running - #38

Merged
ivanvyd merged 1 commit into
mainfrom
docs/service-principal-and-scheduled-pack
Aug 5, 2026
Merged

Make unattended runs possible today, and show one running#38
ivanvyd merged 1 commit into
mainfrom
docs/service-principal-and-scheduled-pack

Conversation

@ivanvyd

@ivanvyd ivanvyd commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Stacked on #37, which is stacked on #36. Review those first; GitHub retargets this as each
merges.

Unattended runs work now, with no new auth surface

databricks auth token states it plainly: "This command only works with U2M authentication. M2M
authentication using a client ID and secret is not supported."
So a service principal cannot be
brokered through the CLI, and a scheduled Question Pack was left needing someone's personal access
token.

Implementing the client-credentials flow inside LakeSpeak would make it a credential broker —
exactly what delegating authentication was meant to avoid, and what makes the security story short
enough to be credible.

The documented alternative needs no code. Databricks' M2M endpoint mints a token that the existing
DATABRICKS_TOKEN path already accepts:

export DATABRICKS_TOKEN=$(curl -sS --request POST \
  --url "$DATABRICKS_HOST/oidc/v1/token" \
  --user "$DATABRICKS_CLIENT_ID:$DATABRICKS_CLIENT_SECRET" \
  --data 'grant_type=client_credentials&scope=all-apis' \
  | jq -r .access_token)

Verified against Databricks' published M2M flow (endpoint, parameters, and the expires_in: 3600
response) rather than written from memory — that check was the point of listing it as an assumption
in the plan rather than a fact.

Four caveats travel with it in the docs, because the ones that bite are not the obvious ones:
the token lasts an hour with no refresh, so it is minted immediately before the run and no
long-running process can use it; the service principal sees only what it is granted and inherits
nothing from you; the token is never echoed; and the recipe is documented, not exercised live by
this project
.

A scheduled run you can copy

examples/github-actions/daily-brief.yml — installs the tool, mints and masks the token, runs the
pack, commits the report.

It lives under examples/ and not in .github/workflows/. A live one here would need a real
workspace's secrets in a public repository and would bill a SQL warehouse on a cron.

It pins its actions by commit, matching this repository's own workflows. A moved tag in a
workflow holding workspace credentials is someone else's code running with your secrets, and an
example that taught otherwise would be worse than no example.

It closes on the two decisions worth making before using it: a committed report puts governed data
wherever the repository is readable — an artifact with a retention policy is often better — and
exit 0 does not mean every answer was useful, because Genie can reply with a clarifying question
and that is a successful message.

Verification

224 tests pass. The docs-contract test from #36 picked up the new lakespeak pack run example
automatically and parses it — the suite went from 82 to 83 cases without anyone wiring it up, which
is the behaviour that test was added for. The workflow parses as valid YAML. All relative markdown
links across the repository resolve.

@ivanvyd
ivanvyd force-pushed the docs/positioning-after-cli-ask branch from 47c1742 to 4fc24e6 Compare August 5, 2026 18:24
Base automatically changed from docs/positioning-after-cli-ask to main August 5, 2026 18:39
`databricks auth token` states plainly that M2M is unsupported, so a
service principal cannot be brokered through the CLI. Implementing the
client-credentials flow inside LakeSpeak would make it a credential broker,
which is precisely what delegating auth was meant to avoid.

The documented alternative needs no code. Databricks' M2M endpoint mints a
token that the existing DATABRICKS_TOKEN path already accepts, so CI works
now: docs/authentication.md carries the exact call, verified against
Databricks' published flow rather than written from memory.

Four caveats travel with it, because the ones that bite are not obvious.
The token lasts an hour with no refresh, so it is minted immediately before
the run and a long-running process cannot use it. The service principal
sees only what it is granted and does not inherit the author's access. The
token is never echoed. And the recipe is documented, not exercised live by
this project -- said in the doc rather than left to be assumed.

The GitHub Actions example ships under examples/ rather than
.github/workflows/, because a live one here would need a real workspace's
secrets in a public repository and would bill a warehouse on a cron. It
pins its actions by commit for the same reason this repository's own
workflows do: a moved tag in a workflow holding workspace credentials is
someone else's code running with your secrets.

It also closes on the two decisions worth making before using it -- that a
committed report puts governed data wherever the repository is readable,
and that exit 0 does not mean every answer was useful, since Genie can
answer with a clarifying question.
@ivanvyd
ivanvyd force-pushed the docs/service-principal-and-scheduled-pack branch from aa43f19 to b0cbd12 Compare August 5, 2026 18:41
@ivanvyd
ivanvyd merged commit b934d6b into main Aug 5, 2026
10 checks passed
@ivanvyd
ivanvyd deleted the docs/service-principal-and-scheduled-pack branch August 5, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant