Prepare 19.2.0: migration extension, audit security fix, Groovy/ASM updates - #86
Open
royteeuwen wants to merge 138 commits into
Open
Prepare 19.2.0: migration extension, audit security fix, Groovy/ASM updates#86royteeuwen wants to merge 138 commits into
royteeuwen wants to merge 138 commits into
Conversation
- Bump groovy.version 4.0.31 -> 5.0.5 - Raise maven.compiler.source/target from 1.8 to 11 (Groovy 5 requires JDK 11+) - Bump exported API package versions to 20.0.0 - Update README compatibility section: 20.x requires Java 11/17/21, 19.x remains the line for Java 8 environments - Bump project version to 20.0.0-SNAPSHOT
Sling Starter ships a singleton config for org.apache.felix.hc.generalchecks.ServicesCheck (the systemalive check). Felix Configurator rejects mixing singleton + factory configs for the same base PID, so our factory config (~groovyconsole) was never materialized as an HC instance — silently failing to provide a groovyconsole-tagged health check at all. Removing the broken config. Bundle consumers that want a Groovy Console readiness check can register their own HealthCheck service in their environment if needed.
Sling Starter 14 hits an upstream NPE in Felix HTTP Jetty12 1.1.8 (WhiteboardContextHandler.getRegistry() null during a config update mid-startup) that intermittently corrupts the whiteboard registry, returning 404 for /system/health.json and any other whiteboard-managed servlets. Polling Felix HC for readiness is therefore unreliable. Switching the @BeforeAll wait to a no-op script POST against /bin/groovyconsole/post (Sling Engine routing, unaffected by the bug) plus a 15s stability window so we don't fire tests during the bundle refresh cascade. 300s overall timeout covers the slow-install variant where bundle resolution retries before the groovy host gets refreshed to 5.0.5.
Sling Starter 14 ships org.apache.felix.http.jetty12 1.1.8, which intermittently fails IT startup. Add a jetty12 feature file pinning 1.2.0 and an artifactsOverrides HIGHEST rule so it wins the clash. Remove once we move to Sling Starter 15.
Upgrade Groovy to 5.0.6 (Java 11+ required)
Modern UI (new ui.frontend module — Lit + Spectrum Web Components + Monaco, Vite): - IDE-style shell: app bar with primary Run action, resizable editor/output split, tabbed output dock (Result/Log/Trace/Table) with running-time badge and clickable stacktrace frames that jump to the source line, activity rail with History / Scheduled Jobs / Help drawers, status bar with run state, cursor position and shortcut hints (Cmd+Enter run, Cmd+S save, Cmd+K commands) - IDE-grade code assistance backed by new /bin/groovyconsole/assist/* servlets: class dictionary from OSGi bundle wiring (with auto-import), member completion via reflection incl. Groovy metaclass/GDK methods, binding and star-import context, and live compile diagnostics (parse-only, shared shell configuration with execution via the extracted GroovyShellFactory) - Folder-navigable script browser and save dialog that also work on plain Sling - Built into ui.apps at /apps/groovyconsole/spa with stable asset names Routing and configuration: - New defaultUi OSGi property (classic|modern, default classic); /groovyconsole serves the default while the classic/modern selectors always work; both UIs cross-link each other Classic UI modernization: - Replace the AEM-only ExtJS open/save dialogs with Bootstrap modals that also work on plain Sling; drop cq.wcm.edit and cq.shared clientlib dependencies (context path now injected via HTL); delete the obsolete groovyconsole-aem ExtJS dialog definitions - Fix pre-existing NPE in ScheduledJobsServlet on plain Sling and add the permission gate to ServicesListServlet Testing and infrastructure: - New ui.tests module: Playwright end-to-end suite behind -Pui-tests, booting its own Sling feature-model instance; AssistIT integration tests; CI updated
Scripts can now be executed asynchronously (POST /bin/groovyconsole/post with async=true) returning an execution id; output is polled incrementally from the new /bin/groovyconsole/stream endpoint while the script runs. The execution registry buffers output per execution, retains finished executions for ten minutes, and audits results exactly as synchronous runs do. Both UIs consume it: the modern UI shows a live output dock during the run and the classic UI appends to its output panel. Both fall back transparently to synchronous execution against older backends. Backwards compatible: without the async parameter the endpoint behaves exactly as before, keeping external clients such as the IntelliJ AEM Groovy Console plugin working unchanged. Covered by StreamingIT (including a test proving compile-only semantics and partial output before completion) and Playwright specs for both UIs.
- Send Cache-Control: no-store on the stream polling endpoint so responses are never cached by a CDN (AEM as a Cloud Service) - Degrade gracefully in both UIs when polling loses the execution (e.g. the request is routed to another cluster instance): explain that the script keeps running and point to the History panel, instead of reporting a failure - Document the async/stream API contract and the cluster affinity caveat in the README - Gate StreamingIT on actual endpoint readiness instead of only the system health check to avoid a startup race
Manual testing on the AEMaaCS SDK surfaced that browser POSTs were rejected with 403 by the Granite CSRF filter (header-authenticated clients such as curl and the IntelliJ plugin are unaffected): - The modern UI now fetches /libs/granite/csrf/token.json and sends the CSRF-Token header on POST/DELETE requests when running on AEM (skipped on plain Sling) - The classic UI's AEM clientlib declares the granite.csrf.standalone dependency, restoring the automatic token injection previously provided transitively by the removed cq.shared dependency
Extracts the shared Sling feature aggregation and launch configuration into pluginManagement so the existing 'it' profile and a new 'run' profile reuse it without duplication. mvn clean install mvn verify -Prun -pl it.tests # boots Sling on http://localhost:8080 (admin/admin) The run profile launches the same aggregated feature the integration tests use, on a fixed port (override with -Dhttp.port), and blocks in the foreground tailing the instance log; Ctrl+C stops it.
…ceholders - The output dock now selects Log (script output) by default on completion, with Result as a secondary tab — matching the classic console where the log was the primary view. Errors still jump to the Trace tab. - Widen the slide-out drawers (720 -> 960px) and keep timestamp links on one line so History/Scheduled Jobs dates no longer wrap. - Mute sp-textfield placeholders (grey italic) in the scheduler and save dialogs; Spectrum renders them in the text colour by default, making empty fields look pre-filled.
- Change the Default UI OSGi property default from classic to modern, so /groovyconsole serves the modern UI out of the box (classic still reachable via the .classic.html selector). - Document the modern UI, code assistance, streaming output, the run profile and the Default UI setting in the README, and record everything under [Unreleased] in the CHANGELOG.
on: [push, pull_request] fired both events for same-repo PR branches, running every commit twice. Restrict push to main and add a concurrency group so only one run happens per PR commit and superseded runs are cancelled.
- Order static imports before regular imports in the new/modified Groovy servlets and models (Groovy rule S8314). - Modernize the classic console.js code added for streaming and the open/save dialogs: const/let over var, optional chaining, globalThis, and a fixed regex character-class escape. - Extract a SEVERITY_ERROR constant for the repeated compile-marker severity. - Suppress S2925 on StreamingIT, where Thread.sleep is intentional remote polling (matching GroovyConsoleServiceIT). Left as-is: the Bootstrap modal role="dialog" attributes (S6819) — Bootstrap 3 modals require the role and a native <dialog> would break their show/hide.
Modern IDE-style UI, code assistance, and streaming output
Move ExecutionRegistry to the api module and make it accept any ScriptContext (plus an optional ExecutionCallback run on the worker thread), so extensions can run scripts asynchronously with their own context. AsyncScriptContext moves to api context.impl to avoid a split package now that the streaming package is exported.
Suppress class/binding/member completion inside string literals (except the
getService("...") case) and when naming a variable after def/var. Add a vitest
harness and unit tests for the completion heuristics.
Add ConsoleUiExtensionProvider and the window.GroovyConsole.registerPanel mechanism so an installed extension bundle can contribute a panel to the modern UI's activity rail, plus the supporting config/client/util plumbing.
Add the aem-groovy-console-reports extension: report definitions with an inline Groovy script and typed parameters under /conf/groovyconsole/reports, asynchronous run-once executions with persisted paginated results, CSV/XLSX export, and scheduled purging. Authorization is pure JCR access control on the report nodes (read = view/run, write = create/edit/delete) executed with the requesting user's resolver; there are no application-level access groups.
Add the reports business UI served at /apps/groovyconsole/reports.html (catalogue, parameter form, async run with polling, paginated result table, exports, history, and an inline-script editor) and the developer Reports drawer contributed to the console via the UI extension SPI. Wires the multi-entry vite build.
Add GroovyConsoleReportsIT (API, async polling, export wiring, truncation) and a Playwright reports spec. Convert the all-packages to feature-model bundles at build time with the sling-feature-converter so every bundle (incl. Groovy 5.0.6) is in the initial launch, avoiding the runtime content-package install that raced startup and timed out the IT.
Describe the reports extension in the README and CHANGELOG and add the frontend API contract in docs/reports-frontend-requirements.md: single inline script per report, JCR-permission-based access control, and asynchronous execution polling.
result.json referenced a removed 'definition' local (left over from the pure-ACL refactor), 500ing the result/pagination endpoints; restore the report lookup and reuse it for the page-size default. Under pure JCR-ACL access an anonymous user gets 404 (node not visible), so relax the IT's anonymous-view assertion to any 4xx like the execute/edit cases.
- exporter registry: use toSorted() instead of sort(false) - ResultParser: drop the redundant '!= false' literal for the exported default - ReportBrowseServlet: extract constants for the PAGE/ASSET filter and AEM primary-type literals - IT: drop the obsolete Thread.sleep stability window (no content-package install cascade now that bundles are in the launch feature)
ui.tests installs the console and reports content packages at runtime, whose bundle-refresh cascade races felix.http.jetty12 1.1.8 and corrupts the HTTP whiteboard (every UI test then fails with the page/editor never rendering). Pin jetty12 1.2.0 in the aggregated feature, as it.tests already does.
It is an implementation class, not API, so it does not belong in the api bundle. Move it to be.orbinson.aem.groovy.console.streaming.impl in the core bundle and update the ScriptPostServlet import.
The row cap ran in finishSuccess, after the script had already built the full result in memory, so it could not bound execution memory; results are stored as a gzipped binary anyway, with no JCR property-size limit. Drop capRows, the truncated flag (execution + preview + JSON + read) and the now-unused maxResultRows OSGi property, so every row is persisted. Move the result-write logic into ReportResultStore so it owns both read and write, and format dates with java.time instead of SimpleDateFormat.
Migration scripts can now be discovered from an immutable /apps path in addition to the mutable /conf path, and a startup hook auto-runs pending migrations after an AEM as a Cloud Service deployment. - scriptsBasePaths (was scriptsBasePath) defaults to /apps/groovyconsole-migration-scripts and /conf/groovyconsole/scripts/migration, searched in order; missing paths are skipped. The immutable /apps path ships with the code image, reaches publish, and is tamper-proof. It is a dedicated top-level node, not /apps/groovyconsole/..., because the console's own ui.apps package replaces the /apps/groovyconsole subtree on every deploy. - MigrationStartupHook: on activation waits for repository readiness, detects a composite node store (AEMaaCS) and enqueues a run of the pending scripts. Composite-store detection matches the AC Tool and ACM implementations (hasPermission guard + hasCapability on /apps). Configurable via autoRunOnStartup (cloudOnly default / always / never); logs the detection outcome at INFO for cloud-log verification. - MigrationScriptListener now also watches the immutable /apps scripts path so on-premises package installs can auto-run (cloud /apps changes arrive via a container swap and are covered by the startup hook instead). - README: new "Cloud vs on-premises" section and updated configuration tables. - Tests: multi-path discovery (unit + a real-instance IT proving /apps discovery), and MigrationStartupHook detection and auto-run branches.
Make deployment migrations cloud-compatible (AEMaaCS + on-prem)
Address review: keep the query-audit methods out of ReportExecutionService and off its default impl entirely, so query auditing is a self-contained, optional concern — opting out is just deleting the query-audit files, with no change to report execution. - new ReportQueryAuditService (isAvailable + audit) with its own DefaultReportQueryAuditService impl; the servlet references it instead of ReportExecutionService - ReportExecutionService / DefaultReportExecutionService reverted to carry no query-audit concern - revert the reports-api package version bump (1.0.0 is not released yet, so no minor bump is warranted)
…etection # Conflicts: # extensions/migration/bundle/src/main/groovy/be/orbinson/aem/groovy/console/migration/impl/DefaultMigrationService.groovy # ui.frontend/src/components/gc-app-bar.ts # ui.frontend/src/components/gc-app.ts # ui.frontend/src/styles/app.css
Add Oak query-index detection for Groovy scripts
…nto feature/query-audit-reports # Conflicts: # extensions/reports/bundle/pom.xml # extensions/reports/ui.frontend/src/api/reports-api.ts # extensions/reports/ui.frontend/src/components/reports/gcr-report-editor.ts # extensions/reports/ui.frontend/src/styles/reports.css
Reports: add query-audit "Audit queries" action to the report editor
The console.api package and the reports api/model packages are still on the unreleased 20.0.0 line (baseline 19.1.0), so additive changes don't need a package-version bump. Restore them to 20.0.0 / 1.0.0 and describe the /apps definitions drop-zone by what it is.
The schedule editor now offers a Frequency picker (every day / weekday / week / month / custom) with hour, minute, weekday and day-of-month inputs, a plain-language summary and the resulting cron expression. The raw cron field remains under "Custom", with the same client-side validation the server applies. Existing expressions are parsed back into the matching preset. Each distribution field now has a visible label and help text (Destination, Export format, Directory, File name, Recipients, Subject) instead of relying on placeholders, and each target is shown as its own card.
royteeuwen
force-pushed
the
feature/19.2.0
branch
from
July 20, 2026 06:07
5f70807 to
f9543bc
Compare
Add ReportDistributor.isAvailable() and list only available distributors in the distributors endpoint, so the editor never offers a destination that would fail. The filesystem distributor is available only when enabled; the email distributor only when a mail service is bound.
The page servlet appends a per-deploy version token (the bundle's last-modified time) to the stable-named entry assets, and the reports Vite build now content-hashes shared chunks. The entry is refetched after a deploy and pulls in the hashed chunks, so a redeploy shows up without clearing the browser cache, while assets stay cached within a deployment.
…duled-reports # Conflicts: # extensions/reports/bundle/pom.xml # extensions/reports/bundle/src/main/groovy/be/orbinson/aem/groovy/console/reports/servlets/ReportsServlet.groovy # extensions/reports/ui.frontend/src/api/reports-api.ts # extensions/reports/ui.frontend/src/api/reports-types.ts # extensions/reports/ui.frontend/src/components/reports/gcr-report-editor.ts # it.tests/src/test/java/be/orbinson/aem/groovy/console/it/GroovyConsoleReportsIT.java
Align testDistributorsListed with the availability gating: on the plain Sling test instance the filesystem distributor is disabled and email has no mail service, so the endpoint lists neither. Also remove a PathType import left unused after merging the query-audit browse changes.
… test The per-deploy ?v token on the module entry made the browser load reports.js twice (once as reports.js?v=…, once as the bare specifier the bundle's own imports use), so sp-theme was defined twice and the second define threw — aborting the dynamic import chain and leaving the Monaco code editor unregistered on any report whose editor also lazy-loads the path browser (e.g. a PATH parameter). Revert the ?v token and the chunk hashing back to the known-good stable-name build. Add a Playwright test that opens a report's edit view and asserts the Monaco editor actually renders, closing the gap that let this through (the existing reports specs only cover the list and run views).
Content-hash the reports entry, chunks and assets, emit a Vite manifest, and have the shared page servlet link the hashed entry (js + css) read from that manifest. A changed file gets a fresh URL and is refetched; an unchanged file keeps its URL and stays cached — busting without a query token, which previously double-loaded the entry module and broke the code editor. The console-panel entry keeps a stable name (the extension provider references it by fixed path), and the servlet falls back to the stable <name>.js/.css names when no manifest is present (e.g. migration).
The migration service serialises runs and rejects a new run with 409 while another is in progress or queued. Other migration tests in the suite can still be draining an async run when this test fires, so its single POST intermittently got a 409 (surfacing as "no response from migration run"). Retry the run until it is accepted, and mark the audited script ".always." so the accepted run re-executes and audits it even if a background run executed it first (which would otherwise skip it and drop the queryAudit).
Extend the manifest-based cache-busting to all console SPAs, so every content-hashed asset is linked by its hashed URL (cached while unchanged, refetched on change) with a graceful fallback to stable names: - add SpaManifest, a shared resolver (entry js/css by chunk name, and stable-module-URL -> hashed) used across the page servlet and console; - core modern console: hash the entry + chunks + assets and resolve the entry js, entry css and Monaco chunk css via ModernConsoleConfig, which the HTL page now links (also resolves the extension panel module URLs); - reports: also hash the console-panel entry (was stable); - migration: hash entries/chunks/assets + emit a manifest, so the shared servlet busts the migration page and the console busts its panel. Tests: it.tests assert hashed names for the reports page, reports panel, core entry, migration page and migration panel; the Playwright specs load every page (incl. the reports edit-view Monaco guard) so a manifest/ hashing regression that breaks a page is caught.
- The split Run button applied left-only rounding to the Run segment unconditionally, so without the (extension-registered) chevron the button looked square on the right. Default to a full pill; square the right edge only when the split chevron is present. - The reports/migration ui.apps nav filters replaced the shared /apps/cq/core/content/nav tree on install, wiping the core console's "AEM Groovy Console" Tools entry (and each other's). Use mode="merge" on all three nav filters so each adds its entry without deleting siblings.
Scheduler fixed-parameter values now render the same rich typed widgets as the run form (SELECT/DYNAMIC dropdowns, PATH/TAG browser, DATE picker, BOOLEAN checkbox, and a multifield for 'multiple' parameters) via a shared gcr-parameter-fields component, replacing the plain text inputs. The schedule parameterValues model now carries arrays end-to-end (ReportSchedule Map<String, Object>, servlet + service preserve List values, TS Record<string, ReportParameterValue>). Nav overlay filters switched to a two-filter pattern (merge shared nav folders, own only the leaf entry) so each package creates its Tools entry without wiping siblings.
…g is ready The migration scripts base under /conf is provisioned asynchronously and can still be absent when the console endpoint first reports ready, so the one-shot deploy in setUp intermittently 404s. Wrap it in an awaitility retry.
Two new flags on the reports configuration service (both default on): schedulingEnabled and distributionEnabled. Backend enforcement: - scheduling: DefaultReportScheduleService skips registration and unschedules all report jobs when disabled; ReportScheduledJobConsumer skips any job that still fires (authoritative runtime guard). - distribution: applyDistributions is the single sink for both scheduled and manual runs, so one guard there blocks every distribution; the manual ReportDistributeServlet returns 503 with a clear message. Frontend adapts: a new GET /bin/groovyconsole/reports/config endpoint exposes the flags; the editor hides the Schedule / Distribution sections and the run view hides 'Distribute now' when the matching feature is off (defaulting to enabled if the endpoint is unavailable).
Add scheduled reports and a pluggable distribution SPI
…le UI Base the 19.2.0 release line on current main (all extensions — migration, reports, query-audit — plus PR #96) and subtract the two things the 19.x line must not carry: - Revert Groovy 5.0.6 -> 4.0.32, ASM 9.9.1 -> 9.10.1, Java 11 -> 1.8, matching the released 19.x line (AECU / on-prem AEM compatibility). No Java 9+ or Groovy 5 APIs are used anywhere, so the downgrade is source-clean. - Keep the classic Ace-based core console UI. The modern web-components console is dropped as the core UI, but ui.frontend is retained as a shared frontend library (npm ci + source only, no core SPA built or shipped) because the reports/migration extension screens import and type-check against it. The extension screens keep their own web-component UIs. Drop the top-level Playwright ui.tests module and its CI step; add 19.x to the CI push branches. Set the whole reactor to 19.2.0-SNAPSHOT. (cherry picked from commit e5c64d2)
…patibility main exported the core API packages at 20.x, which falls outside AECU's [19,20) import range. Bring them back into 19.x, with per-package versions validated by bnd-baseline against the released 19.1.0 baseline: api 19.1.0, audit 19.2.0, configuration 19.1.0, streaming 19.2.0 (new), components 19.1.0; all others unchanged. (cherry picked from commit fb49311)
…ns for the classic 19.x line - StreamingIT: cold start briefly registers the console servlets, then a framework-level cascade (resource-resolver/repository settling) re-registers them; a single readiness hit could pass during that transient window and the non-retrying tests then hit a down /bin/groovyconsole/post.json (409). Require several consecutive readiness successes so tests start only once the instance is stable. - Drop assertions for features the 19.x line intentionally doesn't ship: the modern SPA served by default / via the .modern selector (repurposed to assert the classic console is served), and the reports panel announced in the modern console's UI-extension config. - docs: fix the manual IT launch instructions to the 'run' profile (mvn verify -Prun -pl it.tests). (cherry picked from commit f9543bc)
royteeuwen
force-pushed
the
feature/19.2.0
branch
from
July 21, 2026 13:02
f9543bc to
4e94ca1
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prepares the
19.xmaintenance line (Java 8 / Groovy 4.x) for a19.2.0release, bringing over the migration extension developed onmainwithout the Groovy 5 upgrade or any of the modern-UI/reports work it isn't compatible with.main's audit/script-servlet access-control hardening (33dc884, 29cd86c) — closes an IDOR where any authenticated user could read/delete another user's audit records, plus a scheduled-jobs permission gap and a script-save permission gap. Adapted to drop the one hunk touchingModernConsoleConfig.groovy, which doesn't exist on this line.main, adapted for this line:MigrationConsoleUiExtensionProvider,migration-console-panel.ts,gc-migration.ts) since that UI doesn't exist here. The standalone history page (/apps/groovyconsole/migrations.html) and Tools console entry are untouched.ui.frontend/src/console-shim, instead of aliasing a siblingui.frontendmodule this line doesn't ship.mockito-inline(this line's convention) instead ofmockito-core.MigrationRunOptionsgainspathanddata, mirroringAecuService.execute(path, data)— a run can be scoped to a single script or folder instead of the full configured base path, and JSON/stringdatais made available to every script in the run as thedatabinding variable (via the console's existingScriptContext/SlingBindingExtensionProviderplumbing). Wired through the HTTP API, the async job queue, and persisted on the run record (path only) for traceability.sling-feature-converter-maven-plugin+slingfeature-maven-plugin+feature-launcher-maven-plugin), scoped to the coreall+migration-allpackages, plus the jetty12 pin and a sturdier readiness-polling helper (both stability improvements, independent of Groovy version). AddedMigrationIT.19.2.0-SNAPSHOT; Groovy 4.0.31 → 4.0.32 (latest 4.x); ASM 9.7 → 9.10.1 (latest, for compatibility with newer JVMs up to Java 25 running the console — vendored for Groovy's runtime bytecode generation, no direct API dependency in this project).19.xtoo (previouslymainonly), with concurrency cancellation.Verification
mvn clean install— full reactor build green (Groovy 4.0.32 / ASM 9.10.1 / Java 8 target), including the migration frontend's Node/Vite build.mvn verify -pl it.tests -Pit— 22/22 integration tests pass (8MigrationIT+ 14GroovyConsoleServiceIT) against a real Sling Starter 14 instance with both content packages installed via the cp-converter pipeline.DefaultMigrationServiceTest,MigrationServletTest), including new coverage for path scoping (folder and single script) and data threading.