Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,9 @@ sysml2tools export "src/**/*.sysml" --format jsonl --output model.jsonl

# Include OMG standard library declarations/edges in the export
sysml2tools export model.sysml --include-stdlib

# Restrict output to a containment subtree, then narrow it with a filter expression
sysml2tools export model.sysml --target Vehicle::Engine --filter "@Deprecated"
```

See [Exporting](docs/user_guide/introduction.md#exporting) in the user guide for the full
Expand Down Expand Up @@ -263,6 +266,8 @@ sysml2tools help [lint|render|query [<query-verb>]|export]
| `--format json\|jsonl` | Output format (default: `json`) |
| `--output <file>` | Write export document to this **file** (default: stdout); differs from `render`'s `--output` dir |
| `--include-stdlib` | Include OMG standard library declarations/edges (diagnostics are never stdlib-filtered) |
| `--target <qualified-name>` | Restrict output to the containment subtree rooted at this element (before `--filter`) |
| `--filter <expr>` | Narrow output using a Phase 1 filter expression (see `render`'s `--filter`); after `--target` |

### `help` Options

Expand Down
12 changes: 9 additions & 3 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,9 +224,15 @@ compatibility-check gap is carried forward as a known limitation in the "View dy
refinements" item above.

**Done:** `export` verb — `export [--format json|jsonl] [--output <file>] [--include-stdlib]
<patterns...>` dumps the resolved semantic model (declarations, edges, diagnostics) as a
single indented JSON document or as JSON Lines, for an agent harness to index locally and run
its own queries offline. Implemented by the new `Export` subsystem (`ExportCommand`,
[--target <qualified-name>] [--filter <expr>] <patterns...>` dumps the resolved semantic
model (declarations, edges, diagnostics) as a single indented JSON document or as JSON
Lines, for an agent harness to index locally and run its own queries offline. `--target`
restricts output to a single element's containment subtree (expanding a usage/feature
target to its resolved type's subtree too), and `--filter` narrows the (optionally
`--target`-scoped) declarations/edges using the same Phase 1 filter-expression subset
`render`'s dynamic-view `--filter` uses, composing `--target`-then-`--filter` — a graceful,
non-aborting warning/diagnostic is surfaced for an unparseable `--filter` expression rather
than failing the export. Implemented by the new `Export` subsystem (`ExportCommand`,
`ExportResult`, `ExportResultSerializerContext`/`ExportLineSerializerContext`), reusing
`SysmlNode`/`SysmlEdge`/`SysmlDiagnostic` directly rather than a fourth parallel result shape.
See its design/requirements/verification documentation for the full JSON/JSONL output shape.
Expand Down
118 changes: 106 additions & 12 deletions docs/design/sysml2-tools-tool/export.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,17 @@ a targeted analysis question into a small `QueryResult`, `export` is a lossless,
dump of the whole model intended for offline/AI-assisted bulk analysis. It provides six
cooperating types:

- `ExportOptions` — an immutable record capturing `Format`, `Output`, `IncludeStdlib`, and
`Files`, parsed by `Context.Create` for one `export` invocation.
- `ExportArgumentParser` — parses `--format`, `--output`, `--include-stdlib`, and positional
file globs, rejecting any other `-`-prefixed token, mirroring
- `ExportOptions` — an immutable record capturing `Format`, `Output`, `IncludeStdlib`,
`Target`, `FilterExpression`, and `Files`, parsed by `Context.Create` for one `export`
invocation.
- `ExportArgumentParser` — parses `--format`, `--output`, `--include-stdlib`, `--target`,
`--filter`, and positional file globs, rejecting any other `-`-prefixed token, mirroring
`LintArgumentParser`/`RenderArgumentParser`'s shape.
- `ExportCommand` — the entry-point dispatcher, mirroring `LintCommand`/`RenderCommand`'s
`internal static class` shape with a `RunAsync(Context)` method and a `PrintHelp` method.
Loads the workspace, applies the `--include-stdlib` filter, builds an `ExportResult`, and
renders it as JSON or JSONL to `--output` or stdout.
Loads the workspace, applies the `--include-stdlib` filter, then `--target` subtree scoping
and `--filter` expression narrowing (see **Target Scoping** and **Filter Narrowing**
below), builds an `ExportResult`, and renders it as JSON or JSONL to `--output` or stdout.
- `ExportResult` — the envelope record (`Declarations`, `Edges`, `Diagnostics`) reusing the
existing `SysmlNode`/`SysmlEdge`/`SysmlDiagnostic` model types directly rather than
introducing a fourth parallel result shape.
Expand All @@ -39,11 +41,24 @@ cooperating types:
the workspace's own lookup key, not just an implicit array index).
2. Filters `workspace.Index.AllEdges` down to edges whose `Source` and `Target` are both
visible per `IsVisible`, producing `ExportResult.Edges`.
3. Copies `workspace.Diagnostics` verbatim into `ExportResult.Diagnostics` — diagnostics are
never stdlib-filtered (see **Stdlib Filtering** below).
4. Renders the resulting `ExportResult` as an indented JSON document (`--format json`,
3. When `--target` is supplied: resolves its containment-subtree "subject" qualified names
(see **Target Scoping** below), reporting a clean "not found" error and returning (no
export produced) when the target does not resolve to a visible declaration; otherwise
narrows `Declarations`/`Edges` (from steps 1–2) to that subtree, requiring — for edges —
both endpoints (when the source is non-null) to lie within it.
4. When `--filter` is supplied: parses it via `FilterExpressionParser.Parse`; on success,
narrows the (possibly `--target`-scoped) `Declarations` to the matched subset via
`FilterExpressionEvaluator.Evaluate`, and re-derives `Edges` to keep only edges whose
endpoints (when the source is non-null) both survive in the narrowed declaration set (see
**Filter Narrowing** below); on parse failure, appends a synthetic warning
`SysmlDiagnostic` to the diagnostics list and prints a matching console warning, falling
back to the unfiltered (but still `--target`-scoped, if applicable) result.
5. Copies `workspace.Diagnostics` (plus any synthetic `--filter`-failure diagnostic from step
4) into `ExportResult.Diagnostics` — diagnostics are never stdlib-filtered (see **Stdlib
Filtering** below).
6. Renders the resulting `ExportResult` as an indented JSON document (`--format json`,
default) or as JSONL (`--format jsonl`) — see **Output Model** below.
5. Writes the rendered text to the file named by `--output` (via `File.WriteAllText`) when
7. Writes the rendered text to the file named by `--output` (via `File.WriteAllText`) when
present, or to stdout via `context.WriteLine` otherwise.

##### Stdlib Filtering
Expand All @@ -62,6 +77,64 @@ the stdlib symbol table returned by `StdlibProvider.GetSymbolTable()` is a pre-r
consumed as-is by `WorkspaceLoader`, not re-parsed, so it can never itself be a diagnostic
source.

##### Target Scoping

`--target <qualified-name>` restricts export output to the containment subtree rooted at the
named element. `ExportCommand.ResolveTargetSubtreeSubjects` resolves the target: it returns
`null` when the name is absent from `workspace.Declarations`, or is present but fails
`IsVisible` (a standard-library declaration without `--include-stdlib`) — both cases are
reported by the caller as the same "was not found in the workspace" error, matching
`IsVisible`'s existing exclude semantics used elsewhere in this class. When the target
resolves, the "subject" set starts as `{ target }`; if the target's declaration is a
`SysmlFeatureNode` (a usage, e.g. `part myVehicle : Vehicle;`), the usage's own resolved
`SysmlEdgeKind.Typing` edge target (its type) is added to the subject set too, so a usage
target still yields useful subtree content instead of a near-empty result. A qualified name
is then "in scope" when it equals a subject or has a subject as a `"{subject}::"` prefix
(`ExportCommand.IsInTargetSubtree`).

This mirrors Core's internal `DemaConsulting.SysML2Tools.Layout.Internal.ExposeScopeResolver`
— specifically its `AddWholeSubtreeSubject` usage-to-type expansion — which the Tool project
cannot reference directly (it is `internal` to Core, and Core's `InternalsVisibleTo` list
only grants Core's own test project, not this Tool project). `ExportCommand` therefore
duplicates a small, purpose-built subset of that logic locally, following the exact same
duplication precedent already established by `IsVisible` above: `--target` only ever has a
single target and no per-target bracket filter (the standalone `--filter` option already
covers narrowing), so `ExposeScopeResolver`'s multi-target/bracket-filter/`Failures`
machinery is deliberately omitted from this smaller copy. Future changes to
`ExposeScopeResolver`'s whole-subtree-subject behavior should prompt a deliberate check of
whether this duplicate needs a matching update.

##### Filter Narrowing

`--filter <expr>` narrows the exported declarations/edges using the same Phase 1
filter-expression subset `render`'s dynamic-view `--filter` uses, reusing the public
`DemaConsulting.SysML2Tools.Filtering.FilterExpressionParser`/`FilterExpressionEvaluator`
types directly (both are `public`, so — unlike the target-subtree logic above — no
duplication is needed here). `ExportCommand.RunAsync` parses `options.FilterExpression` via
`FilterExpressionParser.Parse`; on success, it evaluates the parsed expression against the
current (possibly `--target`-scoped) declaration keys via
`FilterExpressionEvaluator.Evaluate`, narrows `Declarations` to the matched subset, and
re-derives `Edges` to keep only edges whose endpoints (when the source is non-null) both
remain in the matched set.

A parse failure does not abort the export: mirroring `GeneralViewLayoutStrategy`'s own
graceful degradation for a non-evaluatable `filter [<expr>];` statement, `ExportCommand` falls
back to the unfiltered (but still `--target`-scoped, if applicable) result, and surfaces the
failure via two channels — (a) a synthetic `SysmlDiagnostic` appended to
`ExportResult.Diagnostics`, with `FilePath = "<--filter>"` (following the same `[stdlib]…`
virtual-path convention already used for non-ordinary `FilePath` values — see **Stdlib
Filtering** above), `Line = 0`, `Column = 0`, `Severity = DiagnosticSeverity.Warning`, and a
`Message` describing the failure, so the failure is visible to offline/agent JSON/JSONL
consumers; and (b) a matching `context.WriteLine` console warning, mirroring `render`'s own
`--filter` warning-surfacing convention, for interactive visibility.
`FilterExpressionEvaluator.Evaluate` itself never fails (its `Diagnostics` list is always
empty by design), so the single failure point handled here is the parse step.

Composition order is deliberate: `--target` scopes first, `--filter` narrows second — mirroring
`GeneralViewLayoutStrategy`'s `expose`-then-`filter` pipeline exactly. With no `--target`,
`--filter` narrows the whole (stdlib-filtered) workspace, matching a view with a `filter`
statement but no `expose` statement.

#### Output Model

##### ExportResult Data Model
Expand Down Expand Up @@ -115,6 +188,12 @@ implemented directly without re-spiking.
in both the XML doc comment and `export --help` text to prevent confusion between the two
commands.
- `IncludeStdlib` (`bool`) — from `--include-stdlib`; mirrors `query`'s exact convention.
- `Target` (`string?`) — a qualified name from `--target`, restricting output to that
element's containment subtree; `null` means no target scoping (the whole stdlib-filtered
workspace). See **Target Scoping** above.
- `FilterExpression` (`string?`) — a raw Phase 1 filter-expression text from `--filter`,
narrowing the (possibly `--target`-scoped) declaration/edge set; `null` means no filtering.
See **Filter Narrowing** above.
- `Files` (`IReadOnlyList<string>`) — file glob patterns.

#### ExportCommand
Expand All @@ -132,11 +211,15 @@ implemented directly without re-spiking.
the given pattern(s)" and returns when the pattern list resolved to zero files.
5. Loads the workspace via `StdlibProvider.GetSymbolTable()` + `WorkspaceLoader.LoadAsync`;
`WriteError`s "workspace loading failed" and returns if `Workspace` is `null`.
6. Builds the filtered `ExportResult` per **Command Semantics** above.
6. Builds the filtered `ExportResult` per **Command Semantics** above, including `--target`
subtree scoping (`WriteError`s "--target '<name>' was not found in the workspace" and
returns — no export produced — when the target does not resolve) and `--filter` narrowing
(falling back to the unfiltered, `--target`-scoped result with a diagnostic and console
warning on parse failure, rather than aborting).
7. Renders as JSON or JSONL per `options.Format`.
8. Writes to `options.Output` (file) or stdout.

**`PrintHelp(Context context)`**: Prints the `export` usage line, its three options (with an
**`PrintHelp(Context context)`**: Prints the `export` usage line, its five options (with an
explicit note that `--output` names a file, not a directory), and one example invocation,
sourced from `ExportStrings`.

Expand All @@ -156,6 +239,15 @@ the rationale).
- Patterns given but none matched any files: `context.WriteError`; `Context.ExitCode`
becomes 1.
- Workspace failed to load: `context.WriteError`; `Context.ExitCode` becomes 1.
- `--target` does not resolve to a visible declaration (genuinely absent, or a
standard-library declaration without `--include-stdlib`): `context.WriteError` reporting
"--target '<name>' was not found in the workspace"; `Context.ExitCode` becomes 1; no export
is produced. Both "absent" and "stdlib-hidden" cases intentionally share this one message —
see **Target Scoping** above.
- `--filter` fails to parse/evaluate: not a hard error — falls back to the unfiltered
(`--target`-scoped, if applicable) result, with a synthetic warning `SysmlDiagnostic`
appended to the output and a matching `context.WriteLine` console warning; `Context.ExitCode`
remains 0. See **Filter Narrowing** above.

#### Dependencies

Expand Down Expand Up @@ -185,4 +277,6 @@ the rationale).
| SysML2Tools-Tool-Export-JsonEnvelope | `ExportResult`; `ExportResultSerializerContext` |
| SysML2Tools-Tool-Export-JsonlEnvelope | `Export*Line` records; `ExportLineSerializerContext` |
| SysML2Tools-Tool-Export-Help | `ExportCommand.PrintHelp`, called from `Program.RunAsync` |
| SysML2Tools-Tool-Export-Target | `ExportOptions.Target`; `ResolveTargetSubtreeSubjects`/`IsInTargetSubtree` |
| SysML2Tools-Tool-Export-Filter | `ExportOptions.FilterExpression`; `FilterExpressionParser`/`Evaluator` |
| SysML2Tools-Tool-Export-SelfTest | `Validation.RunExportSelfTestAsync` |
56 changes: 53 additions & 3 deletions docs/reqstream/sysml2-tools-tool/export.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ sections:
requirements:
- id: SysML2Tools-Tool-Export-Grammar
title: >-
The export command shall accept --format, --output, and --include-stdlib options,
plus one or more positional file glob patterns, and shall reject any other
`-`-prefixed token.
The export command shall accept --format, --output, --include-stdlib, --target, and
--filter options, plus one or more positional file glob patterns, and shall reject
any other `-`-prefixed token.
justification: |
A small, fixed flag vocabulary (mirroring the query/render commands' own per-command
argument parsers) keeps export's grammar simple and predictable, and rejecting
Expand All @@ -27,9 +27,13 @@ sections:
- ExportArgumentParser_FormatFlag_CapturesRawValue
- ExportArgumentParser_OutputFlag_CapturesValue
- ExportArgumentParser_IncludeStdlibFlag_SetsTrue
- ExportArgumentParser_TargetFlag_CapturesValue
- ExportArgumentParser_FilterFlag_CapturesValue
- ExportArgumentParser_UnrecognizedFlag_ThrowsArgumentException
- ExportArgumentParser_FormatFlagMissingValue_ThrowsArgumentException
- ExportArgumentParser_OutputFlagMissingValue_ThrowsArgumentException
- ExportArgumentParser_TargetFlagMissingValue_ThrowsArgumentException
- ExportArgumentParser_FilterFlagMissingValue_ThrowsArgumentException

- id: SysML2Tools-Tool-Export-Format
title: >-
Expand Down Expand Up @@ -131,6 +135,52 @@ sections:
tests:
- ExportSubsystem_ExportHelp_PrintsHelpWithoutThrowing

- id: SysML2Tools-Tool-Export-Target
title: >-
The export command shall accept --target <qualified-name> restricting output to the
containment subtree rooted at that element (expanding a usage/feature target to also
include its resolved type's subtree), reporting a clear "not found" error — with no
export produced — when the name does not resolve to a visible declaration (either
genuinely absent, or a standard-library declaration without --include-stdlib).
justification: |
A single-target containment-subtree scope lets an agent/CLI consumer request a
focused slice of a large workspace instead of always exporting the whole model,
mirroring the subtree-scoping semantics already established by render's `expose`
statement handling (Core's ExposeScopeResolver), applied here as a small, purpose-
built duplicate since the Tool project cannot reference that Core-internal type.
tests:
- ExportSubsystem_TargetFlag_RestrictsToSubtree
- ExportSubsystem_TargetFlag_IncludesEdgesWithBothEndpointsInSubtree
- ExportSubsystem_TargetFlag_UnresolvedName_ReportsNotFoundError
- ExportSubsystem_TargetFlag_StdlibTargetWithoutIncludeStdlib_ReportsNotFoundError
- ExportSubsystem_TargetFlag_StdlibTargetWithIncludeStdlib_Succeeds
- Context_Create_ExportCommand_WithTarget_SetsTarget
- ExportIntegration_RealFixture_TargetAndFilter_ProducesScopedJson

- id: SysML2Tools-Tool-Export-Filter
title: >-
The export command shall accept --filter <expr> narrowing exported declarations and
edges using the Phase 1 filter-expression subset (Core.Filtering), applied after
--target scoping (narrowing the target-scoped set further when both are supplied, or
the whole stdlib-filtered workspace when --target is absent); an unparsable or
unsupported --filter expression shall not abort the export — it shall fall back to
the unfiltered (but still target-scoped, if applicable) result, surfaced via both a
synthetic warning diagnostic in the output and a console warning.
justification: |
Reusing the public Core.Filtering FilterExpressionParser/FilterExpressionEvaluator
directly (no duplication needed, unlike --target's subtree logic) keeps export's
filter semantics identical to render's `--filter`; falling back to the unfiltered
result on parse failure (rather than aborting) mirrors GeneralViewLayoutStrategy's
own graceful degradation for a non-evaluatable `filter [<expr>];` statement, so a
typo in the expression never turns a working export invocation into a hard failure.
tests:
- ExportSubsystem_FilterFlag_NarrowsDeclarations
- ExportSubsystem_FilterFlag_WithoutTarget_NarrowsWholeWorkspace
- ExportSubsystem_TargetAndFilter_ComposeTargetFirstThenFilter
- ExportSubsystem_FilterFlag_UnsupportedConstruct_AddsDiagnosticAndWarns
- Context_Create_ExportCommand_WithFilter_SetsFilterExpression
- ExportIntegration_RealFixture_TargetAndFilter_ProducesScopedJson

- id: SysML2Tools-Tool-Export-SelfTest
title: >-
The self-validation suite (--validate) shall include an export self-test that loads
Expand Down
Loading
Loading