diff --git a/docs/adr/0002-crdt-based-layout-system.md b/docs/adr/0002-crdt-based-layout-system.md new file mode 100644 index 0000000000..a937b76064 --- /dev/null +++ b/docs/adr/0002-crdt-based-layout-system.md @@ -0,0 +1,111 @@ +# 2. CRDT-Based Layout System + +Date: 2024-08-16 + +## Status + +Accepted + +## Context + +ComfyUI's node graph editor faces fundamental architectural limitations that prevent us from achieving our product goals: + +### The Problem + +In the current system, each node manages its own position directly within LiteGraph. This creates several critical issues: + +1. **Performance Degradation**: Every UI update requires traversing the entire graph to detect changes. With graphs containing 100+ nodes, this polling-based approach causes visible lag during interactions. + +2. **Snap-Back Hell**: Multiple systems (LiteGraph canvas, Vue widgets, drag handlers) fight over node positions. Users experience frustrating "snap-back" where nodes jump between positions during drag operations. + +3. **No Collaboration Path**: Direct mutation of node positions makes real-time collaboration impossible. There's no way to merge concurrent edits from multiple users without conflicts. + +4. **Limited Renderer Options**: Position data is tightly coupled to LiteGraph's canvas renderer, blocking us from implementing WebGL rendering for large graphs or accessibility-focused DOM rendering. + +5. **Missing Features**: Without a proper event system, we can't implement undo/redo, animation systems, or viewport culling efficiently. + +### Why Now? + +- User complaints about performance with large workflows are increasing +- The AI art community expects real-time collaboration (see Figma, Miro) +- Accessibility requirements demand alternative rendering modes +- The technical debt is compounding with each new feature + +## Decision + +We will implement a centralized layout tree using CRDT (Conflict-free Replicated Data Types) as the single source of truth for all spatial data. + +### Key Design Choices + +1. **CRDT-Based Layout Tree**: Use Yjs to maintain a centralized tree structure that owns all node positions, sizes, and spatial relationships. + +2. **Command Pattern**: Every position change is an explicit command/operation rather than direct mutation. This enables: + - Precise operation history for undo/redo + - Automatic conflict resolution for concurrent edits + - Event stream for observers without polling + +3. **Unidirectional Data Flow**: + ``` + User Input → Layout Commands → CRDT Tree → Renderers + ``` + LiteGraph becomes a pure renderer that receives position updates, never mutates them. + +4. **Spatial Indexing**: The tree structure naturally supports a QuadTree spatial index for O(log n) viewport queries instead of O(n) full scans. + +### Why CRDT? + +CRDTs solve our core problems elegantly: +- **Local-First**: Works perfectly for single-user while being collaboration-ready +- **Automatic Conflict Resolution**: No more snap-back from competing updates +- **Event-Driven**: Changes propagate through observers, not polling +- **Memory Efficient**: Only changed portions of the tree are updated + +### Implementation Approach + +Phase 1: Build alongside existing system +- Layout tree observes LiteGraph changes initially +- Gradually migrate interactions to command pattern +- Maintain full backwards compatibility + +Phase 2: Invert control +- Layout tree becomes source of truth +- LiteGraph receives updates via one-way sync +- Enable alternative renderers + +## Consequences + +### Positive + +- **10x Performance**: Viewport culling and spatial indexing eliminate full graph traversals +- **Multiplayer Ready**: CRDT foundation enables real-time collaboration without architecture changes +- **Undo/Redo**: Command pattern makes history trivial to implement +- **Renderer Flexibility**: Clean separation allows WebGL, DOM, or hybrid rendering +- **Developer Experience**: Clear data flow and event system simplify debugging + +### Negative + +- **Learning Curve**: Team needs to understand CRDT concepts and command pattern +- **Migration Complexity**: Existing code must be carefully migrated to new system +- **Initial Memory Overhead**: ~30KB for Yjs library + operation history storage + +### Mitigations + +- Provide clear migration guides and examples +- Build compatibility layer for gradual migration +- Implement operation history pruning for long-running sessions + +## Notes + +This architecture aligns with modern state management patterns seen in Figma, Linear, and other collaborative tools. The investment in CRDT infrastructure pays dividends across multiple feature areas and positions ComfyUI as a modern, collaborative AI workflow tool. + +The command pattern also opens doors for: +- Macro recording and playback +- Automated testing of UI interactions +- Remote control via API +- AI-assisted layout optimization + +## References + +- [Yjs Documentation](https://docs.yjs.dev/) +- [CRDTs: The Hard Parts](https://martin.kleppmann.com/2020/07/06/crdt-hard-parts-hydra.html) +- [Figma's Multiplayer Technology](https://www.figma.com/blog/how-figmas-multiplayer-technology-works/) \ No newline at end of file diff --git a/docs/adr/README.md b/docs/adr/README.md index 67ef529de1..90e25f7da1 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -11,6 +11,7 @@ An Architecture Decision Record captures an important architectural decision mad | ADR | Title | Status | Date | |-----|-------|--------|------| | [0001](0001-merge-litegraph-into-frontend.md) | Merge LiteGraph.js into ComfyUI Frontend | Accepted | 2025-08-05 | +| [0002](0002-crdt-based-layout-system.md) | CRDT-Based Layout System | Accepted | 2024-08-16 | ## Creating a New ADR diff --git a/package-lock.json b/package-lock.json index d2dcae5ade..a41189d283 100644 --- a/package-lock.json +++ b/package-lock.json @@ -49,6 +49,7 @@ "vue-i18n": "^9.14.3", "vue-router": "^4.4.3", "vuefire": "^3.2.1", + "yjs": "^13.6.27", "zod": "^3.23.8", "zod-validation-error": "^3.3.0" }, @@ -10103,6 +10104,15 @@ "integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==", "dev": true }, + "node_modules/isomorphic.js": { + "version": "0.2.5", + "resolved": "https://registry.npmjs.org/isomorphic.js/-/isomorphic.js-0.2.5.tgz", + "integrity": "sha512-PIeMbHqMt4DnUP3MA/Flc0HElYjMXArsw1qwJZcm9sqR8mq3l8NYizFMty0pWwE/tzIGH3EKK5+jes5mAr85yw==", + "funding": { + "type": "GitHub Sponsors ❤", + "url": "https://github.com/sponsors/dmonad" + } + }, "node_modules/jackspeak": { "version": "3.4.0", "resolved": "https://registry.npmjs.org/jackspeak/-/jackspeak-3.4.0.tgz", @@ -10906,6 +10916,26 @@ "deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.", "dev": true }, + "node_modules/lib0": { + "version": "0.2.114", + "resolved": "https://registry.npmjs.org/lib0/-/lib0-0.2.114.tgz", + "integrity": "sha512-gcxmNFzA4hv8UYi8j43uPlQ7CGcyMJ2KQb5kZASw6SnAKAf10hK12i2fjrS3Cl/ugZa5Ui6WwIu1/6MIXiHttQ==", + "dependencies": { + "isomorphic.js": "^0.2.4" + }, + "bin": { + "0ecdsa-generate-keypair": "bin/0ecdsa-generate-keypair.js", + "0gentesthtml": "bin/gentesthtml.js", + "0serve": "bin/0serve.js" + }, + "engines": { + "node": ">=16" + }, + "funding": { + "type": "GitHub Sponsors ❤", + "url": "https://github.com/sponsors/dmonad" + } + }, "node_modules/lie": { "version": "3.3.0", "resolved": "https://registry.npmjs.org/lie/-/lie-3.3.0.tgz", @@ -17917,6 +17947,22 @@ "node": ">=8" } }, + "node_modules/yjs": { + "version": "13.6.27", + "resolved": "https://registry.npmjs.org/yjs/-/yjs-13.6.27.tgz", + "integrity": "sha512-OIDwaflOaq4wC6YlPBy2L6ceKeKuF7DeTxx+jPzv1FHn9tCZ0ZwSRnUBxD05E3yed46fv/FWJbvR+Ud7x0L7zw==", + "dependencies": { + "lib0": "^0.2.99" + }, + "engines": { + "node": ">=16.0.0", + "npm": ">=8.0.0" + }, + "funding": { + "type": "GitHub Sponsors ❤", + "url": "https://github.com/sponsors/dmonad" + } + }, "node_modules/yn": { "version": "3.1.1", "resolved": "https://registry.npmjs.org/yn/-/yn-3.1.1.tgz", diff --git a/package.json b/package.json index c188015cc6..4894cef377 100644 --- a/package.json +++ b/package.json @@ -115,6 +115,7 @@ "vue-i18n": "^9.14.3", "vue-router": "^4.4.3", "vuefire": "^3.2.1", + "yjs": "^13.6.27", "zod": "^3.23.8", "zod-validation-error": "^3.3.0" } diff --git a/src/adapters/layoutAdapter.ts b/src/adapters/layoutAdapter.ts new file mode 100644 index 0000000000..abc50bc29f --- /dev/null +++ b/src/adapters/layoutAdapter.ts @@ -0,0 +1,82 @@ +/** + * Layout Adapter Interface + * + * Abstracts the underlying CRDT implementation to allow for different + * backends (Yjs, Automerge, etc.) and easier testing. + */ +import type { LayoutOperation } from '@/types/layoutOperations' +import type { NodeId, NodeLayout } from '@/types/layoutTypes' + +/** + * Change event emitted by the adapter + */ +export interface AdapterChange { + /** Type of change */ + type: 'set' | 'delete' | 'clear' + /** Affected node IDs */ + nodeIds: NodeId[] + /** Actor who made the change */ + actor?: string +} + +/** + * Layout adapter interface for CRDT abstraction + */ +export interface LayoutAdapter { + /** + * Set a node's layout data + */ + setNode(nodeId: NodeId, layout: NodeLayout): void + + /** + * Get a node's layout data + */ + getNode(nodeId: NodeId): NodeLayout | null + + /** + * Delete a node + */ + deleteNode(nodeId: NodeId): void + + /** + * Get all nodes + */ + getAllNodes(): Map + + /** + * Clear all nodes + */ + clear(): void + + /** + * Add an operation to the log + */ + addOperation(operation: LayoutOperation): void + + /** + * Get operations since a timestamp + */ + getOperationsSince(timestamp: number): LayoutOperation[] + + /** + * Get operations by a specific actor + */ + getOperationsByActor(actor: string): LayoutOperation[] + + /** + * Subscribe to changes + */ + subscribe(callback: (change: AdapterChange) => void): () => void + + /** + * Transaction support for atomic updates + */ + transaction(fn: () => void, actor?: string): void + + /** + * Network sync methods (for future use) + */ + getStateVector(): Uint8Array + getStateAsUpdate(): Uint8Array + applyUpdate(update: Uint8Array): void +} diff --git a/src/adapters/mockLayoutAdapter.ts b/src/adapters/mockLayoutAdapter.ts new file mode 100644 index 0000000000..1657575bc4 --- /dev/null +++ b/src/adapters/mockLayoutAdapter.ts @@ -0,0 +1,137 @@ +/** + * Mock Layout Adapter + * + * Simple in-memory implementation for testing without CRDT overhead. + */ +import type { LayoutOperation } from '@/types/layoutOperations' +import type { NodeId, NodeLayout } from '@/types/layoutTypes' + +import type { AdapterChange, LayoutAdapter } from './layoutAdapter' + +/** + * Mock implementation for testing + */ +export class MockLayoutAdapter implements LayoutAdapter { + private nodes = new Map() + private operations: LayoutOperation[] = [] + private changeCallbacks = new Set<(change: AdapterChange) => void>() + private currentActor?: string + + setNode(nodeId: NodeId, layout: NodeLayout): void { + this.nodes.set(nodeId, { ...layout }) + this.notifyChange({ + type: 'set', + nodeIds: [nodeId], + actor: this.currentActor + }) + } + + getNode(nodeId: NodeId): NodeLayout | null { + const layout = this.nodes.get(nodeId) + return layout ? { ...layout } : null + } + + deleteNode(nodeId: NodeId): void { + const existed = this.nodes.delete(nodeId) + if (existed) { + this.notifyChange({ + type: 'delete', + nodeIds: [nodeId], + actor: this.currentActor + }) + } + } + + getAllNodes(): Map { + // Return a copy to prevent external mutations + const copy = new Map() + for (const [id, layout] of this.nodes) { + copy.set(id, { ...layout }) + } + return copy + } + + clear(): void { + const nodeIds = Array.from(this.nodes.keys()) + this.nodes.clear() + this.operations = [] + + if (nodeIds.length > 0) { + this.notifyChange({ + type: 'clear', + nodeIds, + actor: this.currentActor + }) + } + } + + addOperation(operation: LayoutOperation): void { + this.operations.push({ ...operation }) + } + + getOperationsSince(timestamp: number): LayoutOperation[] { + return this.operations + .filter((op) => op.timestamp > timestamp) + .map((op) => ({ ...op })) + } + + getOperationsByActor(actor: string): LayoutOperation[] { + return this.operations + .filter((op) => op.actor === actor) + .map((op) => ({ ...op })) + } + + subscribe(callback: (change: AdapterChange) => void): () => void { + this.changeCallbacks.add(callback) + return () => this.changeCallbacks.delete(callback) + } + + transaction(fn: () => void, actor?: string): void { + const previousActor = this.currentActor + this.currentActor = actor + try { + fn() + } finally { + this.currentActor = previousActor + } + } + + // Mock network sync methods + getStateVector(): Uint8Array { + return new Uint8Array([1, 2, 3]) // Mock data + } + + getStateAsUpdate(): Uint8Array { + // Simple serialization for testing + const json = JSON.stringify({ + nodes: Array.from(this.nodes.entries()), + operations: this.operations + }) + return new TextEncoder().encode(json) + } + + applyUpdate(update: Uint8Array): void { + // Simple deserialization for testing + const json = new TextDecoder().decode(update) + const data = JSON.parse(json) as { + nodes: Array<[NodeId, NodeLayout]> + operations: LayoutOperation[] + } + + this.nodes.clear() + for (const [id, layout] of data.nodes) { + this.nodes.set(id, layout) + } + this.operations = data.operations + } + + private notifyChange(change: AdapterChange): void { + this.changeCallbacks.forEach((callback) => { + try { + callback(change) + } catch (error) { + console.error('Error in mock adapter change callback:', error) + } + }) + } +} diff --git a/src/adapters/yjsLayoutAdapter.ts b/src/adapters/yjsLayoutAdapter.ts new file mode 100644 index 0000000000..d0a9ee4974 --- /dev/null +++ b/src/adapters/yjsLayoutAdapter.ts @@ -0,0 +1,202 @@ +/** + * Yjs Layout Adapter + * + * Implements the LayoutAdapter interface using Yjs as the CRDT backend. + * Provides efficient local state management with future collaboration support. + */ +import * as Y from 'yjs' + +import type { LayoutOperation } from '@/types/layoutOperations' +import type { Bounds, NodeId, NodeLayout, Point } from '@/types/layoutTypes' + +import type { AdapterChange, LayoutAdapter } from './layoutAdapter' + +/** + * Yjs implementation of the layout adapter + */ +export class YjsLayoutAdapter implements LayoutAdapter { + private ydoc: Y.Doc + private ynodes: Y.Map> + private yoperations: Y.Array + private changeCallbacks = new Set<(change: AdapterChange) => void>() + + constructor() { + this.ydoc = new Y.Doc() + this.ynodes = this.ydoc.getMap('nodes') + this.yoperations = this.ydoc.getArray('operations') + + // Set up change observation + this.ynodes.observe((event, transaction) => { + const change: AdapterChange = { + type: 'set', // Yjs doesn't distinguish set/delete in observe + nodeIds: [], + actor: transaction.origin as string | undefined + } + + // Collect affected node IDs + event.changes.keys.forEach((changeType, key) => { + change.nodeIds.push(key) + if (changeType.action === 'delete') { + change.type = 'delete' + } + }) + + // Notify subscribers + this.notifyChange(change) + }) + } + + /** + * Set a node's layout data + */ + setNode(nodeId: NodeId, layout: NodeLayout): void { + const ynode = this.layoutToYNode(layout) + this.ynodes.set(nodeId, ynode) + } + + /** + * Get a node's layout data + */ + getNode(nodeId: NodeId): NodeLayout | null { + const ynode = this.ynodes.get(nodeId) + return ynode ? this.yNodeToLayout(ynode) : null + } + + /** + * Delete a node + */ + deleteNode(nodeId: NodeId): void { + this.ynodes.delete(nodeId) + } + + /** + * Get all nodes + */ + getAllNodes(): Map { + const result = new Map() + for (const [nodeId] of this.ynodes) { + const ynode = this.ynodes.get(nodeId) + if (ynode) { + result.set(nodeId, this.yNodeToLayout(ynode)) + } + } + return result + } + + /** + * Clear all nodes + */ + clear(): void { + this.ynodes.clear() + } + + /** + * Add an operation to the log + */ + addOperation(operation: LayoutOperation): void { + this.yoperations.push([operation]) + } + + /** + * Get operations since a timestamp + */ + getOperationsSince(timestamp: number): LayoutOperation[] { + const operations: LayoutOperation[] = [] + this.yoperations.forEach((op) => { + if (op && op.timestamp > timestamp) { + operations.push(op) + } + }) + return operations + } + + /** + * Get operations by a specific actor + */ + getOperationsByActor(actor: string): LayoutOperation[] { + const operations: LayoutOperation[] = [] + this.yoperations.forEach((op) => { + if (op && op.actor === actor) { + operations.push(op) + } + }) + return operations + } + + /** + * Subscribe to changes + */ + subscribe(callback: (change: AdapterChange) => void): () => void { + this.changeCallbacks.add(callback) + return () => this.changeCallbacks.delete(callback) + } + + /** + * Transaction support for atomic updates + */ + transaction(fn: () => void, actor?: string): void { + this.ydoc.transact(fn, actor) + } + + /** + * Get the current state vector for sync + */ + getStateVector(): Uint8Array { + return Y.encodeStateVector(this.ydoc) + } + + /** + * Get state as update for sending to peers + */ + getStateAsUpdate(): Uint8Array { + return Y.encodeStateAsUpdate(this.ydoc) + } + + /** + * Apply updates from remote peers + */ + applyUpdate(update: Uint8Array): void { + Y.applyUpdate(this.ydoc, update) + } + + /** + * Convert layout to Yjs structure + */ + private layoutToYNode(layout: NodeLayout): Y.Map { + const ynode = new Y.Map() + ynode.set('id', layout.id) + ynode.set('position', layout.position) + ynode.set('size', layout.size) + ynode.set('zIndex', layout.zIndex) + ynode.set('visible', layout.visible) + ynode.set('bounds', layout.bounds) + return ynode + } + + /** + * Convert Yjs structure to layout + */ + private yNodeToLayout(ynode: Y.Map): NodeLayout { + return { + id: ynode.get('id') as string, + position: ynode.get('position') as Point, + size: ynode.get('size') as { width: number; height: number }, + zIndex: ynode.get('zIndex') as number, + visible: ynode.get('visible') as boolean, + bounds: ynode.get('bounds') as Bounds + } + } + + /** + * Notify all change subscribers + */ + private notifyChange(change: AdapterChange): void { + this.changeCallbacks.forEach((callback) => { + try { + callback(change) + } catch (error) { + console.error('Error in adapter change callback:', error) + } + }) + } +} diff --git a/src/components/graph/GraphCanvas.vue b/src/components/graph/GraphCanvas.vue index f4579cce0b..2089e6f516 100644 --- a/src/components/graph/GraphCanvas.vue +++ b/src/components/graph/GraphCanvas.vue @@ -133,6 +133,8 @@ import type { NodeState, VueNodeData } from '@/composables/graph/useGraphNodeManager' +import { useLayout } from '@/composables/graph/useLayout' +import { useLayoutSync } from '@/composables/graph/useLayoutSync' import { useNodeBadge } from '@/composables/node/useNodeBadge' import { useCanvasDrop } from '@/composables/useCanvasDrop' import { useContextMenuTranslation } from '@/composables/useContextMenuTranslation' @@ -157,6 +159,7 @@ import { useWorkflowService } from '@/services/workflowService' import { useCommandStore } from '@/stores/commandStore' import { useExecutionStore } from '@/stores/executionStore' import { useCanvasStore } from '@/stores/graphStore' +import { layoutStore } from '@/stores/layoutStore' import { useNodeDefStore } from '@/stores/nodeDefStore' import { useSettingStore } from '@/stores/settingStore' import { useToastStore } from '@/stores/toastStore' @@ -174,6 +177,7 @@ const workspaceStore = useWorkspaceStore() const canvasStore = useCanvasStore() const executionStore = useExecutionStore() const toastStore = useToastStore() +const { mutations: layoutMutations } = useLayout() const betaMenuEnabled = computed( () => settingStore.get('Comfy.UseNewMenu') !== 'Disabled' ) @@ -316,6 +320,19 @@ const initializeNodeManager = () => { nodeSizes.value = nodeManager.nodeSizes detectChangesInRAF = nodeManager.detectChangesInRAF Object.assign(performanceMetrics, nodeManager.performanceMetrics) + + // Initialize layout system with existing nodes + const nodes = comfyApp.graph._nodes.map((node: any) => ({ + id: node.id.toString(), + pos: node.pos, + size: node.size + })) + layoutStore.initializeFromLiteGraph(nodes) + + // Initialize layout sync (one-way: Layout Store → LiteGraph) + const { startSync } = useLayoutSync() + startSync(canvasStore.canvas) + // Force computed properties to re-evaluate nodeDataTrigger.value++ } @@ -493,6 +510,13 @@ const handleNodeSelect = (event: PointerEvent, nodeData: VueNodeData) => { } canvasStore.canvas.selectNode(node) + + // Bring node to front when clicked (similar to LiteGraph behavior) + // Skip if node is pinned + if (!node.flags?.pinned) { + layoutMutations.setSource('vue') + layoutMutations.bringNodeToFront(nodeData.id) + } node.selected = true canvasStore.updateSelectedItems() diff --git a/src/components/graph/vueNodes/LGraphNode.vue b/src/components/graph/vueNodes/LGraphNode.vue index b9e195853d..3aa80a6946 100644 --- a/src/components/graph/vueNodes/LGraphNode.vue +++ b/src/components/graph/vueNodes/LGraphNode.vue @@ -5,6 +5,7 @@
diff --git a/src/composables/graph/README.md b/src/composables/graph/README.md new file mode 100644 index 0000000000..e5eeb64d46 --- /dev/null +++ b/src/composables/graph/README.md @@ -0,0 +1,211 @@ +# Graph Composables - Reactive Layout System + +This directory contains composables for the reactive layout system, enabling Vue nodes to handle their own interactions while maintaining synchronization with LiteGraph. + +## Composable Architecture + +```mermaid +graph TB + subgraph "Composables" + URL[useReactiveLayout
- Singleton Management
- Service Access] + UVNI[useVueNodeInteraction
- Node Dragging
- CSS Transforms] + ULGS[useLiteGraphSync
- Bidirectional Sync
- Position Updates] + end + + subgraph "Services" + LT[ReactiveLayoutTree] + HT[ReactiveHitTester] + end + + subgraph "Components" + GC[GraphCanvas] + VN[Vue Nodes] + TP[TransformPane] + end + + URL --> LT + URL --> HT + UVNI --> URL + ULGS --> URL + + GC --> ULGS + VN --> UVNI + TP --> URL + + +## Interaction Flow + +```mermaid +sequenceDiagram + participant User + participant VueNode + participant UVNI as useVueNodeInteraction + participant LT as LayoutTree + participant LG as LiteGraph + + User->>VueNode: pointerdown + VueNode->>UVNI: startDrag(event) + UVNI->>UVNI: Set drag state + UVNI->>UVNI: Capture pointer + + User->>VueNode: pointermove + VueNode->>UVNI: handleDrag(event) + UVNI->>UVNI: Calculate delta + UVNI->>VueNode: Update CSS transform + Note over VueNode: Visual feedback only + + User->>VueNode: pointerup + VueNode->>UVNI: endDrag(event) + UVNI->>LT: updateNodePosition(finalPos) + LT->>LG: Trigger reactive sync + LG->>LG: Update canvas +``` + +## useReactiveLayout + +Singleton management for the reactive layout system. + +```mermaid +classDiagram + class useReactiveLayout { + +layoutTree: ComputedRef~ReactiveLayoutTree~ + +hitTester: ComputedRef~ReactiveHitTester~ + +nodePositions: ComputedRef~Map~ + +nodeBounds: ComputedRef~Map~ + +selectedNodes: ComputedRef~Set~ + -initialize(): void + } + + class Singleton { + <> + Shared across all components + } + + useReactiveLayout --> Singleton : implements +``` + +## useVueNodeInteraction + +Handles individual node interactions with CSS transforms. + +```mermaid +flowchart LR + subgraph "Drag State" + DS[isDragging
dragDelta
dragStartPos] + end + + subgraph "Event Handlers" + SD[startDrag] + HD[handleDrag] + ED[endDrag] + end + + subgraph "Computed Styles" + NS[nodeStyle
- position
- dimensions
- z-index] + DGS[dragStyle
- transform
- transition] + end + + SD --> DS + HD --> DS + ED --> DS + + DS --> NS + DS --> DGS +``` + +### Transform Calculation + +```mermaid +graph TB + subgraph "Mouse Delta" + MD[event.clientX/Y - startMouse] + end + + subgraph "Canvas Transform" + CT[screenToCanvas conversion] + end + + subgraph "Drag Delta" + DD[Canvas-space delta] + end + + subgraph "CSS Transform" + CSS[translate(deltaX, deltaY)] + end + + MD --> CT + CT --> DD + DD --> CSS +``` + +## useLiteGraphSync + +Bidirectional synchronization between LiteGraph and the reactive layout tree. + +```mermaid +stateDiagram-v2 + [*] --> Initialize + + Initialize --> SyncFromLiteGraph + SyncFromLiteGraph --> WatchLayoutTree + + state WatchLayoutTree { + [*] --> Listening + Listening --> PositionChanged: Layout tree update + PositionChanged --> UpdateLiteGraph + UpdateLiteGraph --> TriggerRedraw + TriggerRedraw --> Listening + } + + state SyncFromLiteGraph { + [*] --> ReadNodes + ReadNodes --> UpdateLayoutTree + UpdateLayoutTree --> [*] + } +``` + +## Integration Example + +```typescript +// In GraphCanvas.vue +const { initializeSync } = useLiteGraphSync() +onMounted(() => { + initializeSync() // Start bidirectional sync +}) + +// In LGraphNode.vue +const { + isDragging, + startDrag, + handleDrag, + endDrag, + dragStyle, + updatePosition +} = useVueNodeInteraction(props.nodeData.id) + +// Template +
+``` + +## Performance Considerations + +1. **CSS Transforms During Drag**: No layout recalculation, GPU accelerated +2. **Batch Position Updates**: Layout tree updates trigger single LiteGraph sync +3. **Reactive Efficiency**: Vue's computed properties cache results +4. **Spatial Indexing**: QuadTree integration for fast hit testing + +## Future Migration Path + +Currently: Vue nodes use CSS transforms, commit to layout tree on drag end +Future: Each renderer owns complete interaction handling and layout state \ No newline at end of file diff --git a/src/composables/graph/useGraphNodeManager.ts b/src/composables/graph/useGraphNodeManager.ts index d6304bdcca..9cd76c9d57 100644 --- a/src/composables/graph/useGraphNodeManager.ts +++ b/src/composables/graph/useGraphNodeManager.ts @@ -4,6 +4,7 @@ */ import { nextTick, reactive, readonly } from 'vue' +import { layoutMutations } from '@/services/layoutMutations' import type { WidgetValue } from '@/types/simplifiedWidget' import type { SpatialIndexDebugInfo } from '@/types/spatialIndex' @@ -482,6 +483,11 @@ export const useGraphNodeManager = (graph: LGraph): GraphNodeManager => { currentPos.y !== node.pos[1] ) { nodePositions.set(id, { x: node.pos[0], y: node.pos[1] }) + + // Push position change to layout store + // Source is already set to 'canvas' in detectChangesInRAF + void layoutMutations.moveNode(id, { x: node.pos[0], y: node.pos[1] }) + return true } return false @@ -499,6 +505,14 @@ export const useGraphNodeManager = (graph: LGraph): GraphNodeManager => { currentSize.height !== node.size[1] ) { nodeSizes.set(id, { width: node.size[0], height: node.size[1] }) + + // Push size change to layout store + // Source is already set to 'canvas' in detectChangesInRAF + void layoutMutations.resizeNode(id, { + width: node.size[0], + height: node.size[1] + }) + return true } return false @@ -549,6 +563,9 @@ export const useGraphNodeManager = (graph: LGraph): GraphNodeManager => { let positionUpdates = 0 let sizeUpdates = 0 + // Set source for all canvas-driven updates + layoutMutations.setSource('canvas') + // Process each node for changes for (const node of graph._nodes) { const id = String(node.id) @@ -606,6 +623,15 @@ export const useGraphNodeManager = (graph: LGraph): GraphNodeManager => { } spatialIndex.insert(id, bounds, id) + // Add node to layout store + layoutMutations.setSource('canvas') + void layoutMutations.createNode(id, { + position: { x: node.pos[0], y: node.pos[1] }, + size: { width: node.size[0], height: node.size[1] }, + zIndex: node.order || 0, + visible: true + }) + // Call original callback if provided if (originalCallback) { void originalCallback(node) @@ -624,6 +650,10 @@ export const useGraphNodeManager = (graph: LGraph): GraphNodeManager => { // Remove from spatial index spatialIndex.remove(id) + // Remove node from layout store + layoutMutations.setSource('canvas') + void layoutMutations.deleteNode(id) + // Clean up all tracking references nodeRefs.delete(id) vueNodeData.delete(id) diff --git a/src/composables/graph/useLayout.ts b/src/composables/graph/useLayout.ts new file mode 100644 index 0000000000..0199a8db23 --- /dev/null +++ b/src/composables/graph/useLayout.ts @@ -0,0 +1,31 @@ +/** + * Main composable for accessing the layout system + * + * Provides unified access to the layout store and mutation API. + */ +import { layoutMutations } from '@/services/layoutMutations' +import { layoutStore } from '@/stores/layoutStore' +import type { Bounds, NodeId, Point } from '@/types/layoutTypes' + +/** + * Main composable for accessing the layout system + */ +export function useLayout() { + return { + // Store access + store: layoutStore, + + // Mutation API + mutations: layoutMutations, + + // Reactive accessors + getNodeLayoutRef: (nodeId: NodeId) => layoutStore.getNodeLayoutRef(nodeId), + getAllNodes: () => layoutStore.getAllNodes(), + getNodesInBounds: (bounds: Bounds) => layoutStore.getNodesInBounds(bounds), + + // Non-reactive queries (for performance) + queryNodeAtPoint: (point: Point) => layoutStore.queryNodeAtPoint(point), + queryNodesInBounds: (bounds: Bounds) => + layoutStore.queryNodesInBounds(bounds) + } +} diff --git a/src/composables/graph/useLayoutSync.ts b/src/composables/graph/useLayoutSync.ts new file mode 100644 index 0000000000..4d9e113cf4 --- /dev/null +++ b/src/composables/graph/useLayoutSync.ts @@ -0,0 +1,97 @@ +/** + * Composable for syncing LiteGraph with the Layout system + * + * Implements one-way sync from Layout Store to LiteGraph. + * The layout store is the single source of truth. + */ +import log from 'loglevel' +import { onUnmounted } from 'vue' + +import { layoutStore } from '@/stores/layoutStore' + +// Create a logger for layout debugging +const logger = log.getLogger('layout') +// In dev mode, always show debug logs +if (import.meta.env.DEV) { + logger.setLevel('debug') +} + +/** + * Composable for syncing LiteGraph with the Layout system + * This replaces the bidirectional sync with a one-way sync + */ +export function useLayoutSync() { + let unsubscribe: (() => void) | null = null + + /** + * Start syncing from Layout system to LiteGraph + * This is one-way: Layout → LiteGraph only + */ + function startSync(canvas: any) { + if (!canvas?.graph) return + + // Subscribe to layout changes + unsubscribe = layoutStore.onChange((change) => { + logger.debug('Layout sync received change:', { + source: change.source, + nodeIds: change.nodeIds, + type: change.type + }) + + // Apply changes to LiteGraph regardless of source + // The layout store is the single source of truth + for (const nodeId of change.nodeIds) { + const layout = layoutStore.getNodeLayoutRef(nodeId).value + if (!layout) continue + + const liteNode = canvas.graph.getNodeById(parseInt(nodeId)) + if (!liteNode) continue + + // Update position if changed + if ( + liteNode.pos[0] !== layout.position.x || + liteNode.pos[1] !== layout.position.y + ) { + logger.debug(`Updating LiteGraph node ${nodeId} position:`, { + from: { x: liteNode.pos[0], y: liteNode.pos[1] }, + to: layout.position + }) + liteNode.pos[0] = layout.position.x + liteNode.pos[1] = layout.position.y + } + + // Update size if changed + if ( + liteNode.size[0] !== layout.size.width || + liteNode.size[1] !== layout.size.height + ) { + liteNode.size[0] = layout.size.width + liteNode.size[1] = layout.size.height + } + } + + // Trigger single redraw for all changes + canvas.setDirty(true, true) + }) + } + + /** + * Stop syncing + */ + function stopSync() { + if (unsubscribe) { + unsubscribe() + unsubscribe = null + } + } + + // Auto-cleanup on unmount + onUnmounted(() => { + stopSync() + }) + + return { + startSync, + stopSync + } +} diff --git a/src/composables/graph/useNodeChangeDetection.ts b/src/composables/graph/useNodeChangeDetection.ts new file mode 100644 index 0000000000..54748893a2 --- /dev/null +++ b/src/composables/graph/useNodeChangeDetection.ts @@ -0,0 +1,180 @@ +/** + * Node Change Detection + * + * RAF-based change detection for node positions and sizes. + * Syncs LiteGraph changes to the layout system. + */ +import { reactive } from 'vue' + +import type { LGraph, LGraphNode } from '@/lib/litegraph/src/litegraph' +import { layoutMutations } from '@/services/layoutMutations' + +export interface ChangeDetectionMetrics { + updateTime: number + positionUpdates: number + sizeUpdates: number + rafUpdateCount: number +} + +/** + * Change detection for node geometry + */ +export function useNodeChangeDetection(graph: LGraph) { + const metrics = reactive({ + updateTime: 0, + positionUpdates: 0, + sizeUpdates: 0, + rafUpdateCount: 0 + }) + + // Track last known positions/sizes + const lastSnapshot = new Map< + string, + { pos: [number, number]; size: [number, number] } + >() + + /** + * Detects position changes for a single node + */ + const detectPositionChanges = ( + node: LGraphNode, + nodePositions: Map + ): boolean => { + const id = String(node.id) + const currentPos = nodePositions.get(id) + + if ( + !currentPos || + currentPos.x !== node.pos[0] || + currentPos.y !== node.pos[1] + ) { + nodePositions.set(id, { x: node.pos[0], y: node.pos[1] }) + + // Push position change to layout store + void layoutMutations.moveNode(id, { x: node.pos[0], y: node.pos[1] }) + + return true + } + return false + } + + /** + * Detects size changes for a single node + */ + const detectSizeChanges = ( + node: LGraphNode, + nodeSizes: Map + ): boolean => { + const id = String(node.id) + const currentSize = nodeSizes.get(id) + + if ( + !currentSize || + currentSize.width !== node.size[0] || + currentSize.height !== node.size[1] + ) { + nodeSizes.set(id, { width: node.size[0], height: node.size[1] }) + + // Push size change to layout store + void layoutMutations.resizeNode(id, { + width: node.size[0], + height: node.size[1] + }) + + return true + } + return false + } + + /** + * Main RAF change detection function + */ + const detectChanges = ( + nodePositions: Map, + nodeSizes: Map, + onSpatialChange?: (node: LGraphNode, id: string) => void + ) => { + const startTime = performance.now() + + if (!graph?._nodes) return + + let positionUpdates = 0 + let sizeUpdates = 0 + + // Set source for all canvas-driven updates + layoutMutations.setSource('canvas') + + // Process each node for changes + for (const node of graph._nodes) { + const id = String(node.id) + + const posChanged = detectPositionChanges(node, nodePositions) + const sizeChanged = detectSizeChanges(node, nodeSizes) + + if (posChanged) positionUpdates++ + if (sizeChanged) sizeUpdates++ + + // Notify spatial change if needed + if ((posChanged || sizeChanged) && onSpatialChange) { + onSpatialChange(node, id) + } + } + + // Update metrics + const endTime = performance.now() + metrics.updateTime = endTime - startTime + metrics.positionUpdates = positionUpdates + metrics.sizeUpdates = sizeUpdates + + if (positionUpdates > 0 || sizeUpdates > 0) { + metrics.rafUpdateCount++ + } + } + + /** + * Take a snapshot of current node positions/sizes + */ + const takeSnapshot = () => { + if (!graph?._nodes) return + + lastSnapshot.clear() + for (const node of graph._nodes) { + lastSnapshot.set(String(node.id), { + pos: [node.pos[0], node.pos[1]], + size: [node.size[0], node.size[1]] + }) + } + } + + /** + * Check if any nodes have changed since last snapshot + */ + const hasChangedSinceSnapshot = (): boolean => { + if (!graph?._nodes) return false + + for (const node of graph._nodes) { + const id = String(node.id) + const last = lastSnapshot.get(id) + if (!last) continue + + if ( + last.pos[0] !== node.pos[0] || + last.pos[1] !== node.pos[1] || + last.size[0] !== node.size[0] || + last.size[1] !== node.size[1] + ) { + return true + } + } + return false + } + + return { + metrics, + detectChanges, + detectPositionChanges, + detectSizeChanges, + takeSnapshot, + hasChangedSinceSnapshot + } +} diff --git a/src/composables/graph/useNodeLayout.ts b/src/composables/graph/useNodeLayout.ts new file mode 100644 index 0000000000..05cc467516 --- /dev/null +++ b/src/composables/graph/useNodeLayout.ts @@ -0,0 +1,199 @@ +/** + * Composable for individual Vue node components + * + * Uses customRef for shared write access with Canvas renderer. + * Provides dragging functionality and reactive layout state. + */ +import log from 'loglevel' +import { computed, inject } from 'vue' + +import { layoutMutations } from '@/services/layoutMutations' +import { layoutStore } from '@/stores/layoutStore' +import type { Point } from '@/types/layoutTypes' + +// Create a logger for layout debugging +const logger = log.getLogger('layout') +// In dev mode, always show debug logs +if (import.meta.env.DEV) { + logger.setLevel('debug') +} + +/** + * Composable for individual Vue node components + * Uses customRef for shared write access with Canvas renderer + */ +export function useNodeLayout(nodeId: string) { + const store = layoutStore + const mutations = layoutMutations + + // Get transform utilities from TransformPane if available + const transformState = inject('transformState') as + | { + canvasToScreen: (point: Point) => Point + screenToCanvas: (point: Point) => Point + } + | undefined + + // Get the customRef for this node (shared write access) + const layoutRef = store.getNodeLayoutRef(nodeId) + + logger.debug(`useNodeLayout initialized for node ${nodeId}`, { + hasLayout: !!layoutRef.value, + initialPosition: layoutRef.value?.position + }) + + // Computed properties for easy access + const position = computed(() => { + const layout = layoutRef.value + const pos = layout?.position ?? { x: 0, y: 0 } + logger.debug(`Node ${nodeId} position computed:`, { + pos, + hasLayout: !!layout, + layoutRefValue: layout + }) + return pos + }) + const size = computed( + () => layoutRef.value?.size ?? { width: 200, height: 100 } + ) + const bounds = computed( + () => + layoutRef.value?.bounds ?? { + x: position.value.x, + y: position.value.y, + width: size.value.width, + height: size.value.height + } + ) + const isVisible = computed(() => layoutRef.value?.visible ?? true) + const zIndex = computed(() => layoutRef.value?.zIndex ?? 0) + + // Drag state + let isDragging = false + let dragStartPos: Point | null = null + let dragStartMouse: Point | null = null + + /** + * Start dragging the node + */ + function startDrag(event: PointerEvent) { + if (!layoutRef.value) return + + isDragging = true + dragStartPos = { ...position.value } + dragStartMouse = { x: event.clientX, y: event.clientY } + + // Set mutation source + mutations.setSource('vue') + + // Capture pointer + const target = event.target as HTMLElement + target.setPointerCapture(event.pointerId) + } + + /** + * Handle drag movement + */ + const handleDrag = (event: PointerEvent) => { + if (!isDragging || !dragStartPos || !dragStartMouse || !transformState) { + logger.debug(`Drag skipped for node ${nodeId}:`, { + isDragging, + hasDragStartPos: !!dragStartPos, + hasDragStartMouse: !!dragStartMouse, + hasTransformState: !!transformState + }) + return + } + + // Calculate mouse delta in screen coordinates + const mouseDelta = { + x: event.clientX - dragStartMouse.x, + y: event.clientY - dragStartMouse.y + } + + // Convert to canvas coordinates + const canvasOrigin = transformState.screenToCanvas({ x: 0, y: 0 }) + const canvasWithDelta = transformState.screenToCanvas(mouseDelta) + const canvasDelta = { + x: canvasWithDelta.x - canvasOrigin.x, + y: canvasWithDelta.y - canvasOrigin.y + } + + // Calculate new position + const newPosition = { + x: dragStartPos.x + canvasDelta.x, + y: dragStartPos.y + canvasDelta.y + } + + logger.debug(`Dragging node ${nodeId}:`, { + mouseDelta, + canvasDelta, + newPosition, + currentLayoutPos: layoutRef.value?.position + }) + + // Apply mutation through the layout system + mutations.moveNode(nodeId, newPosition) + } + + /** + * End dragging + */ + function endDrag(event: PointerEvent) { + if (!isDragging) return + + isDragging = false + dragStartPos = null + dragStartMouse = null + + // Release pointer + const target = event.target as HTMLElement + target.releasePointerCapture(event.pointerId) + } + + /** + * Update node position directly (without drag) + */ + function moveTo(position: Point) { + mutations.setSource('vue') + mutations.moveNode(nodeId, position) + } + + /** + * Update node size + */ + function resize(newSize: { width: number; height: number }) { + mutations.setSource('vue') + mutations.resizeNode(nodeId, newSize) + } + + return { + // Reactive state (via customRef) + layoutRef, + position, + size, + bounds, + isVisible, + zIndex, + + // Mutations + moveTo, + resize, + + // Drag handlers + startDrag, + handleDrag, + endDrag, + + // Computed styles for Vue templates + nodeStyle: computed(() => ({ + position: 'absolute' as const, + left: `${position.value.x}px`, + top: `${position.value.y}px`, + width: `${size.value.width}px`, + height: `${size.value.height}px`, + zIndex: zIndex.value, + cursor: isDragging ? 'grabbing' : 'grab' + })) + } +} diff --git a/src/composables/graph/useNodeState.ts b/src/composables/graph/useNodeState.ts new file mode 100644 index 0000000000..eed71f9154 --- /dev/null +++ b/src/composables/graph/useNodeState.ts @@ -0,0 +1,260 @@ +/** + * Node State Management + * + * Manages node visibility, dirty state, and other UI state. + * Provides reactive state for Vue components. + */ +import { nextTick, reactive, readonly } from 'vue' + +import { PERFORMANCE_CONFIG } from '@/constants/layout' +import type { LGraphNode } from '@/lib/litegraph/src/litegraph' + +import type { SafeWidgetData, VueNodeData, WidgetValue } from './useNodeWidgets' + +export interface NodeState { + visible: boolean + dirty: boolean + lastUpdate: number + culled: boolean +} + +export interface NodeMetadata { + lastRenderTime: number + cachedBounds: DOMRect | null + lodLevel: 'high' | 'medium' | 'low' +} + +/** + * Extract safe Vue data from LiteGraph node + */ +export function extractVueNodeData( + node: LGraphNode, + widgets?: SafeWidgetData[] +): VueNodeData { + return { + id: String(node.id), + title: node.title || 'Untitled', + type: node.type || 'Unknown', + mode: node.mode || 0, + selected: node.selected || false, + executing: false, // Will be updated separately based on execution state + widgets, + inputs: node.inputs ? [...node.inputs] : undefined, + outputs: node.outputs ? [...node.outputs] : undefined, + flags: node.flags ? { ...node.flags } : undefined + } +} + +/** + * Node state management composable + */ +export function useNodeState() { + // Reactive state maps + const vueNodeData = reactive(new Map()) + const nodeState = reactive(new Map()) + const nodePositions = reactive(new Map()) + const nodeSizes = reactive( + new Map() + ) + + // Non-reactive node references + const nodeRefs = new Map() + + // WeakMap for heavy metadata that auto-GCs + const nodeMetadata = new WeakMap() + + // Update batching + const pendingUpdates = new Set() + const criticalUpdates = new Set() + const lowPriorityUpdates = new Set() + let updateScheduled = false + let batchTimeoutId: number | null = null + + /** + * Attach metadata to a node + */ + const attachMetadata = (node: LGraphNode) => { + nodeMetadata.set(node, { + lastRenderTime: performance.now(), + cachedBounds: null, + lodLevel: 'high' + }) + } + + /** + * Get access to original LiteGraph node + */ + const getNode = (id: string): LGraphNode | undefined => { + return nodeRefs.get(id) + } + + /** + * Schedule an update for a node + */ + const scheduleUpdate = ( + nodeId?: string, + priority: 'critical' | 'normal' | 'low' = 'normal' + ) => { + if (nodeId) { + const state = nodeState.get(nodeId) + if (state) state.dirty = true + + // Priority queuing + if (priority === 'critical') { + criticalUpdates.add(nodeId) + flush() // Immediate flush for critical updates + return + } else if (priority === 'low') { + lowPriorityUpdates.add(nodeId) + } else { + pendingUpdates.add(nodeId) + } + } + + if (!updateScheduled) { + updateScheduled = true + + // Adaptive batching strategy + if (pendingUpdates.size > 10) { + // Many updates - batch in nextTick + void nextTick(() => flush()) + } else { + // Few updates - small delay for more batching + batchTimeoutId = window.setTimeout( + () => flush(), + PERFORMANCE_CONFIG.BATCH_UPDATE_DELAY + ) + } + } + } + + /** + * Flush all pending updates + */ + const flush = () => { + if (batchTimeoutId !== null) { + clearTimeout(batchTimeoutId) + batchTimeoutId = null + } + + // Clear all pending updates + criticalUpdates.clear() + pendingUpdates.clear() + lowPriorityUpdates.clear() + updateScheduled = false + + // Trigger any additional update logic here + } + + /** + * Initialize node state + */ + const initializeNode = (node: LGraphNode, vueData: VueNodeData): void => { + const id = String(node.id) + + // Store references + nodeRefs.set(id, node) + vueNodeData.set(id, vueData) + + // Initialize state + nodeState.set(id, { + visible: true, + dirty: false, + lastUpdate: performance.now(), + culled: false + }) + + // Initialize position and size + nodePositions.set(id, { x: node.pos[0], y: node.pos[1] }) + nodeSizes.set(id, { width: node.size[0], height: node.size[1] }) + + // Attach metadata + attachMetadata(node) + } + + /** + * Clean up node state + */ + const cleanupNode = (nodeId: string): void => { + nodeRefs.delete(nodeId) + vueNodeData.delete(nodeId) + nodeState.delete(nodeId) + nodePositions.delete(nodeId) + nodeSizes.delete(nodeId) + } + + /** + * Update node property + */ + const updateNodeProperty = ( + nodeId: string, + property: string, + value: unknown + ): void => { + const currentData = vueNodeData.get(nodeId) + if (!currentData) return + + if (property === 'title') { + vueNodeData.set(nodeId, { + ...currentData, + title: String(value) + }) + } else if (property === 'flags.collapsed') { + vueNodeData.set(nodeId, { + ...currentData, + flags: { + ...currentData.flags, + collapsed: Boolean(value) + } + }) + } + } + + /** + * Update widget state + */ + const updateWidgetState = ( + nodeId: string, + widgetName: string, + value: unknown + ): void => { + const currentData = vueNodeData.get(nodeId) + if (!currentData?.widgets) return + + const updatedWidgets = currentData.widgets.map((w) => + w.name === widgetName ? { ...w, value: value as WidgetValue } : w + ) + vueNodeData.set(nodeId, { + ...currentData, + widgets: updatedWidgets + }) + } + + return { + // State maps (read-only) + vueNodeData: readonly(vueNodeData) as ReadonlyMap, + nodeState: readonly(nodeState) as ReadonlyMap, + nodePositions: readonly(nodePositions) as ReadonlyMap< + string, + { x: number; y: number } + >, + nodeSizes: readonly(nodeSizes) as ReadonlyMap< + string, + { width: number; height: number } + >, + + // Methods + getNode, + attachMetadata, + scheduleUpdate, + flush, + initializeNode, + cleanupNode, + updateNodeProperty, + updateWidgetState, + + // Mutable access for internal use + _mutableNodePositions: nodePositions, + _mutableNodeSizes: nodeSizes + } +} diff --git a/src/composables/graph/useNodeWidgets.ts b/src/composables/graph/useNodeWidgets.ts new file mode 100644 index 0000000000..260e26d4dc --- /dev/null +++ b/src/composables/graph/useNodeWidgets.ts @@ -0,0 +1,182 @@ +/** + * Node Widget Management + * + * Handles widget state synchronization between LiteGraph and Vue. + * Provides wrapped callbacks to maintain consistency. + */ +import type { LGraphNode } from '@/lib/litegraph/src/litegraph' +import type { WidgetValue } from '@/types/simplifiedWidget' + +export type { WidgetValue } + +export interface SafeWidgetData { + name: string + type: string + value: WidgetValue + options?: Record + callback?: ((value: unknown) => void) | undefined +} + +export interface VueNodeData { + id: string + title: string + type: string + mode: number + selected: boolean + executing: boolean + widgets?: SafeWidgetData[] + inputs?: unknown[] + outputs?: unknown[] + flags?: { + collapsed?: boolean + } +} + +/** + * Validates that a value is a valid WidgetValue type + */ +export function validateWidgetValue(value: unknown): WidgetValue { + if (value === null || value === undefined || value === void 0) { + return undefined + } + if ( + typeof value === 'string' || + typeof value === 'number' || + typeof value === 'boolean' + ) { + return value + } + if (typeof value === 'object') { + // Check if it's a File array + if (Array.isArray(value) && value.every((item) => item instanceof File)) { + return value as File[] + } + // Otherwise it's a generic object + return value as object + } + // If none of the above, return undefined + console.warn(`Invalid widget value type: ${typeof value}`, value) + return undefined +} + +/** + * Extract safe widget data from LiteGraph widgets + */ +export function extractWidgetData( + widgets?: any[] +): SafeWidgetData[] | undefined { + if (!widgets) return undefined + + return widgets.map((widget) => { + try { + let value = widget.value + + // For combo widgets, if value is undefined, use the first option as default + if ( + value === undefined && + widget.type === 'combo' && + widget.options?.values && + Array.isArray(widget.options.values) && + widget.options.values.length > 0 + ) { + value = widget.options.values[0] + } + + return { + name: widget.name, + type: widget.type, + value: validateWidgetValue(value), + options: widget.options ? { ...widget.options } : undefined, + callback: widget.callback + } + } catch (error) { + return { + name: widget.name || 'unknown', + type: widget.type || 'text', + value: undefined, + options: undefined, + callback: undefined + } + } + }) +} + +/** + * Widget callback management for LiteGraph/Vue sync + */ +export function useNodeWidgets() { + /** + * Creates a wrapped callback for a widget that maintains LiteGraph/Vue sync + */ + const createWrappedCallback = ( + widget: { value?: unknown; name: string }, + originalCallback: ((value: unknown) => void) | undefined, + nodeId: string, + onUpdate: (nodeId: string, widgetName: string, value: unknown) => void + ) => { + let updateInProgress = false + + return (value: unknown) => { + if (updateInProgress) return + updateInProgress = true + + try { + // Validate that the value is of an acceptable type + if ( + value !== null && + value !== undefined && + typeof value !== 'string' && + typeof value !== 'number' && + typeof value !== 'boolean' && + typeof value !== 'object' + ) { + console.warn(`Invalid widget value type: ${typeof value}`) + updateInProgress = false + return + } + + // Always update widget.value to ensure sync + widget.value = value + + // Call the original callback if it exists + if (originalCallback) { + originalCallback.call(widget, value) + } + + // Update Vue state to maintain synchronization + onUpdate(nodeId, widget.name, value) + } finally { + updateInProgress = false + } + } + } + + /** + * Sets up widget callbacks for a node + */ + const setupNodeWidgetCallbacks = ( + node: LGraphNode, + onUpdate: (nodeId: string, widgetName: string, value: unknown) => void + ) => { + if (!node.widgets) return + + const nodeId = String(node.id) + + node.widgets.forEach((widget) => { + const originalCallback = widget.callback + widget.callback = createWrappedCallback( + widget, + originalCallback, + nodeId, + onUpdate + ) + }) + } + + return { + validateWidgetValue, + extractWidgetData, + createWrappedCallback, + setupNodeWidgetCallbacks + } +} diff --git a/src/constants/layout.ts b/src/constants/layout.ts new file mode 100644 index 0000000000..128982e80e --- /dev/null +++ b/src/constants/layout.ts @@ -0,0 +1,73 @@ +/** + * Layout System Constants + * + * Centralized configuration values for the layout system. + * These values control spatial indexing, performance, and behavior. + */ + +/** + * QuadTree configuration for spatial indexing + */ +export const QUADTREE_CONFIG = { + /** Default bounds for the QuadTree - covers a large canvas area */ + DEFAULT_BOUNDS: { + x: -10000, + y: -10000, + width: 20000, + height: 20000 + }, + /** Maximum tree depth to prevent excessive subdivision */ + MAX_DEPTH: 6, + /** Maximum items per node before subdivision */ + MAX_ITEMS_PER_NODE: 4 +} as const + +/** + * Performance and optimization settings + */ +export const PERFORMANCE_CONFIG = { + /** RAF-based change detection interval (roughly 60fps) */ + CHANGE_DETECTION_INTERVAL: 16, + /** Spatial query cache TTL in milliseconds */ + SPATIAL_CACHE_TTL: 1000, + /** Maximum cache size for spatial queries */ + SPATIAL_CACHE_MAX_SIZE: 100, + /** Batch update delay in milliseconds */ + BATCH_UPDATE_DELAY: 4 +} as const + +/** + * Default values for node layout + */ +export const NODE_DEFAULTS = { + /** Default node size when not specified */ + SIZE: { width: 200, height: 100 }, + /** Default z-index for new nodes */ + Z_INDEX: 0, + /** Default visibility state */ + VISIBLE: true +} as const + +/** + * Debug and development settings + */ +export const DEBUG_CONFIG = { + /** LocalStorage key for enabling layout debug mode */ + LAYOUT_DEBUG_KEY: 'layout-debug', + /** Logger name for layout system */ + LOGGER_NAME: 'layout', + /** Logger name for layout store */ + STORE_LOGGER_NAME: 'layout-store' +} as const + +/** + * Actor and source identifiers + */ +export const ACTOR_CONFIG = { + /** Prefix for auto-generated actor IDs */ + USER_PREFIX: 'user-', + /** Length of random suffix for actor IDs */ + ID_LENGTH: 9, + /** Default source when not specified */ + DEFAULT_SOURCE: 'external' as const +} as const diff --git a/src/lib/litegraph/src/LGraphNode.ts b/src/lib/litegraph/src/LGraphNode.ts index eaebc3ff80..3cb0a7d641 100644 --- a/src/lib/litegraph/src/LGraphNode.ts +++ b/src/lib/litegraph/src/LGraphNode.ts @@ -1967,6 +1967,14 @@ export class LGraphNode move(deltaX: number, deltaY: number): void { if (this.pinned) return + // If Vue nodes mode is enabled, skip LiteGraph's direct position update + // The layout store will handle the movement and sync back to LiteGraph + if (LiteGraph.vueNodesMode) { + // Vue nodes handle their own dragging through the layout store + // This prevents the snap-back issue from conflicting position updates + return + } + this.pos[0] += deltaX this.pos[1] += deltaY } diff --git a/src/services/layoutMutations.ts b/src/services/layoutMutations.ts new file mode 100644 index 0000000000..6a8d143853 --- /dev/null +++ b/src/services/layoutMutations.ts @@ -0,0 +1,150 @@ +/** + * Layout Mutations - Simplified Direct Operations + * + * Provides a clean API for layout operations that are CRDT-ready. + * Operations are synchronous and applied directly to the store. + */ +import { layoutStore } from '@/stores/layoutStore' +import type { + LayoutMutations, + NodeId, + NodeLayout, + Point, + Size +} from '@/types/layoutTypes' + +class LayoutMutationsImpl implements LayoutMutations { + /** + * Set the current mutation source + */ + setSource(source: 'canvas' | 'vue' | 'external'): void { + layoutStore.setSource(source) + } + + /** + * Set the current actor (for CRDT) + */ + setActor(actor: string): void { + layoutStore.setActor(actor) + } + + /** + * Move a node to a new position + */ + moveNode(nodeId: NodeId, position: Point): void { + const existing = layoutStore.getNodeLayoutRef(nodeId).value + if (!existing) return + + layoutStore.applyOperation({ + type: 'moveNode', + nodeId, + position, + previousPosition: existing.position, + timestamp: Date.now(), + source: layoutStore.getCurrentSource(), + actor: layoutStore.getCurrentActor() + }) + } + + /** + * Resize a node + */ + resizeNode(nodeId: NodeId, size: Size): void { + const existing = layoutStore.getNodeLayoutRef(nodeId).value + if (!existing) return + + layoutStore.applyOperation({ + type: 'resizeNode', + nodeId, + size, + previousSize: existing.size, + timestamp: Date.now(), + source: layoutStore.getCurrentSource(), + actor: layoutStore.getCurrentActor() + }) + } + + /** + * Set node z-index + */ + setNodeZIndex(nodeId: NodeId, zIndex: number): void { + const existing = layoutStore.getNodeLayoutRef(nodeId).value + if (!existing) return + + layoutStore.applyOperation({ + type: 'setNodeZIndex', + nodeId, + zIndex, + previousZIndex: existing.zIndex, + timestamp: Date.now(), + source: layoutStore.getCurrentSource(), + actor: layoutStore.getCurrentActor() + }) + } + + /** + * Create a new node + */ + createNode(nodeId: NodeId, layout: Partial): void { + const fullLayout: NodeLayout = { + id: nodeId, + position: layout.position ?? { x: 0, y: 0 }, + size: layout.size ?? { width: 200, height: 100 }, + zIndex: layout.zIndex ?? 0, + visible: layout.visible ?? true, + bounds: { + x: layout.position?.x ?? 0, + y: layout.position?.y ?? 0, + width: layout.size?.width ?? 200, + height: layout.size?.height ?? 100 + } + } + + layoutStore.applyOperation({ + type: 'createNode', + nodeId, + layout: fullLayout, + timestamp: Date.now(), + source: layoutStore.getCurrentSource(), + actor: layoutStore.getCurrentActor() + }) + } + + /** + * Delete a node + */ + deleteNode(nodeId: NodeId): void { + const existing = layoutStore.getNodeLayoutRef(nodeId).value + if (!existing) return + + layoutStore.applyOperation({ + type: 'deleteNode', + nodeId, + previousLayout: existing, + timestamp: Date.now(), + source: layoutStore.getCurrentSource(), + actor: layoutStore.getCurrentActor() + }) + } + + /** + * Bring a node to the front (highest z-index) + */ + bringNodeToFront(nodeId: NodeId): void { + // Get all nodes to find the highest z-index + const allNodes = layoutStore.getAllNodes().value + let maxZIndex = 0 + + for (const [, layout] of allNodes) { + if (layout.zIndex > maxZIndex) { + maxZIndex = layout.zIndex + } + } + + // Set this node's z-index to be one higher than the current max + this.setNodeZIndex(nodeId, maxZIndex + 1) + } +} + +// Create singleton instance +export const layoutMutations = new LayoutMutationsImpl() diff --git a/src/services/spatialIndexManager.ts b/src/services/spatialIndexManager.ts new file mode 100644 index 0000000000..874530761f --- /dev/null +++ b/src/services/spatialIndexManager.ts @@ -0,0 +1,166 @@ +/** + * Spatial Index Manager + * + * Manages spatial indexing for efficient node queries based on bounds. + * Uses QuadTree for fast spatial lookups with caching for performance. + */ +import { PERFORMANCE_CONFIG, QUADTREE_CONFIG } from '@/constants/layout' +import type { Bounds, NodeId } from '@/types/layoutTypes' +import { QuadTree } from '@/utils/spatial/QuadTree' + +/** + * Cache entry for spatial queries + */ +interface CacheEntry { + result: NodeId[] + timestamp: number +} + +/** + * Spatial index manager using QuadTree + */ +export class SpatialIndexManager { + private quadTree: QuadTree + private queryCache: Map + private cacheSize = 0 + + constructor(bounds?: Bounds) { + this.quadTree = new QuadTree( + bounds ?? QUADTREE_CONFIG.DEFAULT_BOUNDS, + { + maxDepth: QUADTREE_CONFIG.MAX_DEPTH, + maxItemsPerNode: QUADTREE_CONFIG.MAX_ITEMS_PER_NODE + } + ) + this.queryCache = new Map() + } + + /** + * Insert a node into the spatial index + */ + insert(nodeId: NodeId, bounds: Bounds): void { + this.quadTree.insert(nodeId, bounds, nodeId) + this.invalidateCache() + } + + /** + * Update a node's bounds in the spatial index + */ + update(nodeId: NodeId, bounds: Bounds): void { + this.quadTree.update(nodeId, bounds) + this.invalidateCache() + } + + /** + * Remove a node from the spatial index + */ + remove(nodeId: NodeId): void { + this.quadTree.remove(nodeId) + this.invalidateCache() + } + + /** + * Query nodes within the given bounds + */ + query(bounds: Bounds): NodeId[] { + const cacheKey = this.getCacheKey(bounds) + const cached = this.queryCache.get(cacheKey) + + // Check cache validity + if (cached) { + const age = Date.now() - cached.timestamp + if (age < PERFORMANCE_CONFIG.SPATIAL_CACHE_TTL) { + return cached.result + } + // Remove stale entry + this.queryCache.delete(cacheKey) + this.cacheSize-- + } + + // Perform query + const result = this.quadTree.query(bounds) + + // Cache result + this.addToCache(cacheKey, result) + + return result + } + + /** + * Clear all nodes from the spatial index + */ + clear(): void { + this.quadTree.clear() + this.invalidateCache() + } + + /** + * Get the current size of the index + */ + get size(): number { + return this.quadTree.size + } + + /** + * Get debug information about the spatial index + */ + getDebugInfo() { + return { + quadTreeInfo: this.quadTree.getDebugInfo(), + cacheSize: this.cacheSize, + cacheEntries: this.queryCache.size + } + } + + /** + * Generate cache key for bounds + */ + private getCacheKey(bounds: Bounds): string { + return `${bounds.x},${bounds.y},${bounds.width},${bounds.height}` + } + + /** + * Add result to cache with LRU eviction + */ + private addToCache(key: string, result: NodeId[]): void { + // Evict oldest entries if cache is full + if (this.cacheSize >= PERFORMANCE_CONFIG.SPATIAL_CACHE_MAX_SIZE) { + const oldestKey = this.findOldestCacheEntry() + if (oldestKey) { + this.queryCache.delete(oldestKey) + this.cacheSize-- + } + } + + this.queryCache.set(key, { + result, + timestamp: Date.now() + }) + this.cacheSize++ + } + + /** + * Find oldest cache entry for LRU eviction + */ + private findOldestCacheEntry(): string | null { + let oldestKey: string | null = null + let oldestTime = Infinity + + for (const [key, entry] of this.queryCache) { + if (entry.timestamp < oldestTime) { + oldestTime = entry.timestamp + oldestKey = key + } + } + + return oldestKey + } + + /** + * Invalidate all cached queries + */ + private invalidateCache(): void { + this.queryCache.clear() + this.cacheSize = 0 + } +} diff --git a/src/stores/layoutStore.ts b/src/stores/layoutStore.ts new file mode 100644 index 0000000000..039ee1c95d --- /dev/null +++ b/src/stores/layoutStore.ts @@ -0,0 +1,665 @@ +/** + * Layout Store - Single Source of Truth + * + * Uses Yjs for efficient local state management and future collaboration. + * CRDT ensures conflict-free operations for both single and multi-user scenarios. + */ +import log from 'loglevel' +import { type ComputedRef, type Ref, computed, customRef } from 'vue' +import * as Y from 'yjs' + +import { ACTOR_CONFIG, DEBUG_CONFIG } from '@/constants/layout' +import { SpatialIndexManager } from '@/services/spatialIndexManager' +import type { + CreateNodeOperation, + DeleteNodeOperation, + LayoutOperation, + MoveNodeOperation, + ResizeNodeOperation, + SetNodeZIndexOperation +} from '@/types/layoutOperations' +import type { + Bounds, + LayoutChange, + LayoutStore, + NodeId, + NodeLayout, + Point +} from '@/types/layoutTypes' + +// Create logger for layout store +const logger = log.getLogger(DEBUG_CONFIG.STORE_LOGGER_NAME) +// In dev mode, always show debug logs +if (import.meta.env.DEV) { + logger.setLevel('debug') +} + +class LayoutStoreImpl implements LayoutStore { + // Yjs document and shared data structures + private ydoc = new Y.Doc() + private ynodes: Y.Map> // Maps nodeId -> Y.Map containing NodeLayout data + private yoperations: Y.Array // Operation log + + // Vue reactivity layer + private version = 0 + private currentSource: 'canvas' | 'vue' | 'external' = + ACTOR_CONFIG.DEFAULT_SOURCE + private currentActor = `${ACTOR_CONFIG.USER_PREFIX}${Math.random() + .toString(36) + .substr(2, ACTOR_CONFIG.ID_LENGTH)}` + + // Change listeners + private changeListeners = new Set<(change: LayoutChange) => void>() + + // CustomRef cache and trigger functions + private nodeRefs = new Map>() + private nodeTriggers = new Map void>() + + // Spatial index manager + private spatialIndex: SpatialIndexManager + + constructor() { + // Initialize Yjs data structures + this.ynodes = this.ydoc.getMap('nodes') + this.yoperations = this.ydoc.getArray('operations') + + // Initialize spatial index manager + this.spatialIndex = new SpatialIndexManager() + + // Listen for Yjs changes and trigger Vue reactivity + this.ynodes.observe((event) => { + this.version++ + + // Trigger all affected node refs + event.changes.keys.forEach((_change, key) => { + const trigger = this.nodeTriggers.get(key) + if (trigger) { + logger.debug(`Yjs change detected for node ${key}, triggering ref`) + trigger() + } + }) + }) + + // Debug: Log layout operations + if (localStorage.getItem(DEBUG_CONFIG.LAYOUT_DEBUG_KEY) === 'true') { + this.yoperations.observe((event) => { + const operations: LayoutOperation[] = [] + event.changes.added.forEach((item) => { + const content = item.content.getContent() + if (Array.isArray(content) && content.length > 0) { + operations.push(content[0] as LayoutOperation) + } + }) + console.log('Layout Operation:', operations) + }) + } + } + + /** + * Get or create a customRef for a node layout + */ + getNodeLayoutRef(nodeId: NodeId): Ref { + let nodeRef = this.nodeRefs.get(nodeId) + + if (!nodeRef) { + logger.debug(`Creating new layout ref for node ${nodeId}`) + + nodeRef = customRef((track, trigger) => { + // Store the trigger so we can call it when Yjs changes + this.nodeTriggers.set(nodeId, trigger) + + return { + get: () => { + track() + const ynode = this.ynodes.get(nodeId) + const layout = ynode ? this.yNodeToLayout(ynode) : null + logger.debug(`Layout ref GET for node ${nodeId}:`, { + position: layout?.position, + hasYnode: !!ynode, + version: this.version + }) + return layout + }, + set: (newLayout: NodeLayout | null) => { + if (newLayout === null) { + // Delete operation + const existing = this.ynodes.get(nodeId) + if (existing) { + this.applyOperation({ + type: 'deleteNode', + nodeId, + timestamp: Date.now(), + source: this.currentSource, + actor: this.currentActor, + previousLayout: this.yNodeToLayout(existing) + }) + } + } else { + // Update operation - detect what changed + const existing = this.ynodes.get(nodeId) + if (!existing) { + // Create operation + this.applyOperation({ + type: 'createNode', + nodeId, + layout: newLayout, + timestamp: Date.now(), + source: this.currentSource, + actor: this.currentActor + }) + } else { + const existingLayout = this.yNodeToLayout(existing) + + // Check what properties changed + if ( + existingLayout.position.x !== newLayout.position.x || + existingLayout.position.y !== newLayout.position.y + ) { + this.applyOperation({ + type: 'moveNode', + nodeId, + position: newLayout.position, + previousPosition: existingLayout.position, + timestamp: Date.now(), + source: this.currentSource, + actor: this.currentActor + }) + } + if ( + existingLayout.size.width !== newLayout.size.width || + existingLayout.size.height !== newLayout.size.height + ) { + this.applyOperation({ + type: 'resizeNode', + nodeId, + size: newLayout.size, + previousSize: existingLayout.size, + timestamp: Date.now(), + source: this.currentSource, + actor: this.currentActor + }) + } + if (existingLayout.zIndex !== newLayout.zIndex) { + this.applyOperation({ + type: 'setNodeZIndex', + nodeId, + zIndex: newLayout.zIndex, + previousZIndex: existingLayout.zIndex, + timestamp: Date.now(), + source: this.currentSource, + actor: this.currentActor + }) + } + } + } + logger.debug(`Layout ref SET triggering for node ${nodeId}`) + trigger() + } + } + }) + + this.nodeRefs.set(nodeId, nodeRef) + } + + return nodeRef + } + + /** + * Get nodes within bounds (reactive) + */ + getNodesInBounds(bounds: Bounds): ComputedRef { + return computed(() => { + // Touch version for reactivity + void this.version + + const result: NodeId[] = [] + for (const [nodeId] of this.ynodes) { + const ynode = this.ynodes.get(nodeId) + if (ynode) { + const layout = this.yNodeToLayout(ynode) + if (layout && this.boundsIntersect(layout.bounds, bounds)) { + result.push(nodeId) + } + } + } + return result + }) + } + + /** + * Get all nodes as a reactive map + */ + getAllNodes(): ComputedRef> { + return computed(() => { + // Touch version for reactivity + void this.version + + const result = new Map() + for (const [nodeId] of this.ynodes) { + const ynode = this.ynodes.get(nodeId) + if (ynode) { + const layout = this.yNodeToLayout(ynode) + if (layout) { + result.set(nodeId, layout) + } + } + } + return result + }) + } + + /** + * Get current version for change detection + */ + getVersion(): ComputedRef { + return computed(() => this.version) + } + + /** + * Query node at point (non-reactive for performance) + */ + queryNodeAtPoint(point: Point): NodeId | null { + const nodes: Array<[NodeId, NodeLayout]> = [] + + for (const [nodeId] of this.ynodes) { + const ynode = this.ynodes.get(nodeId) + if (ynode) { + const layout = this.yNodeToLayout(ynode) + if (layout) { + nodes.push([nodeId, layout]) + } + } + } + + // Sort by zIndex (top to bottom) + nodes.sort(([, a], [, b]) => b.zIndex - a.zIndex) + + for (const [nodeId, layout] of nodes) { + if (this.pointInBounds(point, layout.bounds)) { + return nodeId + } + } + + return null + } + + /** + * Query nodes in bounds (non-reactive for performance) + */ + queryNodesInBounds(bounds: Bounds): NodeId[] { + return this.spatialIndex.query(bounds) + } + + /** + * Apply a layout operation using Yjs transactions + */ + applyOperation(operation: LayoutOperation): void { + logger.debug(`applyOperation called:`, { + type: operation.type, + nodeId: operation.nodeId, + operation + }) + + // Create change object outside transaction so we can use it after + const change: LayoutChange = { + type: 'update', + nodeIds: [], + timestamp: operation.timestamp, + source: operation.source, + operation + } + + // Use Yjs transaction for atomic updates + this.ydoc.transact(() => { + // Add operation to log + this.yoperations.push([operation]) + + // Apply the operation + this.applyOperationInTransaction(operation, change) + }, this.currentActor) + + // Post-transaction updates + this.finalizeOperation(change) + } + + /** + * Apply operation within a transaction + */ + private applyOperationInTransaction( + operation: LayoutOperation, + change: LayoutChange + ): void { + switch (operation.type) { + case 'moveNode': + this.handleMoveNode(operation as MoveNodeOperation, change) + break + case 'resizeNode': + this.handleResizeNode(operation as ResizeNodeOperation, change) + break + case 'setNodeZIndex': + this.handleSetNodeZIndex(operation as SetNodeZIndexOperation, change) + break + case 'createNode': + this.handleCreateNode(operation as CreateNodeOperation, change) + break + case 'deleteNode': + this.handleDeleteNode(operation as DeleteNodeOperation, change) + break + } + } + + /** + * Finalize operation after transaction + */ + private finalizeOperation(change: LayoutChange): void { + // Update version + this.version++ + + // Manually trigger affected node refs after transaction + // This is needed because Yjs observers don't fire for property changes + change.nodeIds.forEach((nodeId) => { + const trigger = this.nodeTriggers.get(nodeId) + if (trigger) { + logger.debug( + `Manually triggering ref for node ${nodeId} after operation` + ) + trigger() + } + }) + + // Notify listeners (after transaction completes) + setTimeout(() => this.notifyChange(change), 0) + } + + /** + * Subscribe to layout changes + */ + onChange(callback: (change: LayoutChange) => void): () => void { + this.changeListeners.add(callback) + return () => this.changeListeners.delete(callback) + } + + /** + * Set the current operation source + */ + setSource(source: 'canvas' | 'vue' | 'external'): void { + this.currentSource = source + } + + /** + * Set the current actor (for CRDT) + */ + setActor(actor: string): void { + this.currentActor = actor + } + + /** + * Get the current operation source + */ + getCurrentSource(): 'canvas' | 'vue' | 'external' { + return this.currentSource + } + + /** + * Get the current actor + */ + getCurrentActor(): string { + return this.currentActor + } + + /** + * Initialize store with existing nodes + */ + initializeFromLiteGraph( + nodes: Array<{ id: string; pos: [number, number]; size: [number, number] }> + ): void { + logger.debug('Initializing layout store from LiteGraph', { + nodeCount: nodes.length, + nodes: nodes.map((n) => ({ id: n.id, pos: n.pos })) + }) + + this.ydoc.transact(() => { + this.ynodes.clear() + this.nodeRefs.clear() + this.nodeTriggers.clear() + this.spatialIndex.clear() + + nodes.forEach((node, index) => { + const layout: NodeLayout = { + id: node.id.toString(), + position: { x: node.pos[0], y: node.pos[1] }, + size: { width: node.size[0], height: node.size[1] }, + zIndex: index, + visible: true, + bounds: { + x: node.pos[0], + y: node.pos[1], + width: node.size[0], + height: node.size[1] + } + } + + this.ynodes.set(layout.id, this.layoutToYNode(layout)) + + // Add to spatial index + this.spatialIndex.insert(layout.id, layout.bounds) + + logger.debug( + `Initialized node ${layout.id} at position:`, + layout.position + ) + }) + }, 'initialization') + + logger.debug('Layout store initialization complete', { + totalNodes: this.ynodes.size + }) + } + + // Operation handlers + private handleMoveNode( + operation: MoveNodeOperation, + change: LayoutChange + ): void { + const ynode = this.ynodes.get(operation.nodeId) + if (!ynode) { + logger.warn(`No ynode found for ${operation.nodeId}`) + return + } + + logger.debug(`Moving node ${operation.nodeId}`, operation.position) + + const size = ynode.get('size') as { width: number; height: number } + ynode.set('position', operation.position) + this.updateNodeBounds(ynode, operation.position, size) + + // Update spatial index + this.spatialIndex.update(operation.nodeId, { + x: operation.position.x, + y: operation.position.y, + width: size.width, + height: size.height + }) + + change.nodeIds.push(operation.nodeId) + } + + private handleResizeNode( + operation: ResizeNodeOperation, + change: LayoutChange + ): void { + const ynode = this.ynodes.get(operation.nodeId) + if (!ynode) return + + const position = ynode.get('position') as Point + ynode.set('size', operation.size) + this.updateNodeBounds(ynode, position, operation.size) + + // Update spatial index + this.spatialIndex.update(operation.nodeId, { + x: position.x, + y: position.y, + width: operation.size.width, + height: operation.size.height + }) + + change.nodeIds.push(operation.nodeId) + } + + private handleSetNodeZIndex( + operation: SetNodeZIndexOperation, + change: LayoutChange + ): void { + const ynode = this.ynodes.get(operation.nodeId) + if (!ynode) return + + ynode.set('zIndex', operation.zIndex) + change.nodeIds.push(operation.nodeId) + } + + private handleCreateNode( + operation: CreateNodeOperation, + change: LayoutChange + ): void { + const ynode = this.layoutToYNode(operation.layout) + this.ynodes.set(operation.nodeId, ynode) + + // Add to spatial index + this.spatialIndex.insert(operation.nodeId, operation.layout.bounds) + + change.type = 'create' + change.nodeIds.push(operation.nodeId) + } + + private handleDeleteNode( + operation: DeleteNodeOperation, + change: LayoutChange + ): void { + if (!this.ynodes.has(operation.nodeId)) return + + this.ynodes.delete(operation.nodeId) + this.nodeRefs.delete(operation.nodeId) + this.nodeTriggers.delete(operation.nodeId) + + // Remove from spatial index + this.spatialIndex.remove(operation.nodeId) + + change.type = 'delete' + change.nodeIds.push(operation.nodeId) + } + + /** + * Update node bounds helper + */ + private updateNodeBounds( + ynode: Y.Map, + position: Point, + size: { width: number; height: number } + ): void { + ynode.set('bounds', { + x: position.x, + y: position.y, + width: size.width, + height: size.height + }) + } + + // Helper methods + private layoutToYNode(layout: NodeLayout): Y.Map { + const ynode = new Y.Map() + ynode.set('id', layout.id) + ynode.set('position', layout.position) + ynode.set('size', layout.size) + ynode.set('zIndex', layout.zIndex) + ynode.set('visible', layout.visible) + ynode.set('bounds', layout.bounds) + return ynode + } + + private yNodeToLayout(ynode: Y.Map): NodeLayout { + return { + id: ynode.get('id') as string, + position: ynode.get('position') as Point, + size: ynode.get('size') as { width: number; height: number }, + zIndex: ynode.get('zIndex') as number, + visible: ynode.get('visible') as boolean, + bounds: ynode.get('bounds') as Bounds + } + } + + private notifyChange(change: LayoutChange): void { + this.changeListeners.forEach((listener) => { + try { + listener(change) + } catch (error) { + console.error('Error in layout change listener:', error) + } + }) + } + + private pointInBounds(point: Point, bounds: Bounds): boolean { + return ( + point.x >= bounds.x && + point.x <= bounds.x + bounds.width && + point.y >= bounds.y && + point.y <= bounds.y + bounds.height + ) + } + + private boundsIntersect(a: Bounds, b: Bounds): boolean { + return !( + a.x + a.width < b.x || + b.x + b.width < a.x || + a.y + a.height < b.y || + b.y + b.height < a.y + ) + } + + // CRDT-specific methods + getOperationsSince(timestamp: number): LayoutOperation[] { + const operations: LayoutOperation[] = [] + this.yoperations.forEach((op) => { + if (op && op.timestamp > timestamp) { + operations.push(op) + } + }) + return operations + } + + getOperationsByActor(actor: string): LayoutOperation[] { + const operations: LayoutOperation[] = [] + this.yoperations.forEach((op) => { + if (op && op.actor === actor) { + operations.push(op) + } + }) + return operations + } + + /** + * Get the Yjs document for network sync (future feature) + */ + getYDoc(): Y.Doc { + return this.ydoc + } + + /** + * Apply updates from remote peers (future feature) + */ + applyUpdate(update: Uint8Array): void { + Y.applyUpdate(this.ydoc, update) + } + + /** + * Get state as update for sending to peers (future feature) + */ + getStateAsUpdate(): Uint8Array { + return Y.encodeStateAsUpdate(this.ydoc) + } +} + +// Create singleton instance +export const layoutStore = new LayoutStoreImpl() + +// Export types for convenience +export type { LayoutStore } from '@/types/layoutTypes' diff --git a/src/types/layoutOperations.ts b/src/types/layoutOperations.ts new file mode 100644 index 0000000000..23fe18158e --- /dev/null +++ b/src/types/layoutOperations.ts @@ -0,0 +1,168 @@ +/** + * Layout Operation Types + * + * Defines the operation interface for the CRDT-based layout system. + * Each operation is immutable and contains all information needed for: + * - Application (forward) + * - Undo/redo (reverse) + * - Conflict resolution (CRDT) + * - Debugging (actor, timestamp, source) + */ +import type { NodeId, NodeLayout, Point } from './layoutTypes' + +/** + * Base operation interface that all operations extend + */ +export interface BaseOperation { + /** Unique operation ID for deduplication */ + id?: string + /** Timestamp for ordering operations */ + timestamp: number + /** Actor who performed the operation (for CRDT) */ + actor: string + /** Source system that initiated the operation */ + source: 'canvas' | 'vue' | 'external' + /** Node this operation affects */ + nodeId: NodeId +} + +/** + * Operation type discriminator for type narrowing + */ +export type OperationType = + | 'moveNode' + | 'resizeNode' + | 'setNodeZIndex' + | 'createNode' + | 'deleteNode' + | 'setNodeVisibility' + | 'batchUpdate' + +/** + * Move node operation + */ +export interface MoveNodeOperation extends BaseOperation { + type: 'moveNode' + position: Point + previousPosition: Point +} + +/** + * Resize node operation + */ +export interface ResizeNodeOperation extends BaseOperation { + type: 'resizeNode' + size: { width: number; height: number } + previousSize: { width: number; height: number } +} + +/** + * Set node z-index operation + */ +export interface SetNodeZIndexOperation extends BaseOperation { + type: 'setNodeZIndex' + zIndex: number + previousZIndex: number +} + +/** + * Create node operation + */ +export interface CreateNodeOperation extends BaseOperation { + type: 'createNode' + layout: NodeLayout +} + +/** + * Delete node operation + */ +export interface DeleteNodeOperation extends BaseOperation { + type: 'deleteNode' + previousLayout: NodeLayout +} + +/** + * Set node visibility operation + */ +export interface SetNodeVisibilityOperation extends BaseOperation { + type: 'setNodeVisibility' + visible: boolean + previousVisible: boolean +} + +/** + * Batch update operation for atomic multi-property changes + */ +export interface BatchUpdateOperation extends BaseOperation { + type: 'batchUpdate' + updates: Partial + previousValues: Partial +} + +/** + * Union of all operation types + */ +export type LayoutOperation = + | MoveNodeOperation + | ResizeNodeOperation + | SetNodeZIndexOperation + | CreateNodeOperation + | DeleteNodeOperation + | SetNodeVisibilityOperation + | BatchUpdateOperation + +/** + * Type guards for operations + */ +export const isBaseOperation = (op: unknown): op is BaseOperation => { + return ( + typeof op === 'object' && + op !== null && + 'timestamp' in op && + 'actor' in op && + 'source' in op && + 'nodeId' in op + ) +} + +export const isMoveNodeOperation = ( + op: LayoutOperation +): op is MoveNodeOperation => op.type === 'moveNode' + +export const isResizeNodeOperation = ( + op: LayoutOperation +): op is ResizeNodeOperation => op.type === 'resizeNode' + +export const isCreateNodeOperation = ( + op: LayoutOperation +): op is CreateNodeOperation => op.type === 'createNode' + +export const isDeleteNodeOperation = ( + op: LayoutOperation +): op is DeleteNodeOperation => op.type === 'deleteNode' + +/** + * Operation application interface + */ +export interface OperationApplicator< + T extends LayoutOperation = LayoutOperation +> { + canApply(operation: T): boolean + apply(operation: T): void + reverse(operation: T): void +} + +/** + * Operation serialization for network/storage + */ +export interface OperationSerializer { + serialize(operation: LayoutOperation): string + deserialize(data: string): LayoutOperation +} + +/** + * Conflict resolution strategy + */ +export interface ConflictResolver { + resolve(op1: LayoutOperation, op2: LayoutOperation): LayoutOperation[] +} diff --git a/src/types/layoutTypes.ts b/src/types/layoutTypes.ts new file mode 100644 index 0000000000..541874bb9a --- /dev/null +++ b/src/types/layoutTypes.ts @@ -0,0 +1,204 @@ +/** + * Layout System - Type Definitions + * + * This file contains all type definitions for the layout system + * that manages node positions, bounds, and spatial data. + */ +import type { ComputedRef, Ref } from 'vue' + +import type { LayoutOperation } from './layoutOperations' + +// Basic geometric types +export interface Point { + x: number + y: number +} + +export interface Size { + width: number + height: number +} + +export interface Bounds { + x: number + y: number + width: number + height: number +} + +// ID types for type safety +export type NodeId = string +export type SlotId = string +export type ConnectionId = string + +// Layout data structures +export interface NodeLayout { + id: NodeId + position: Point + size: Size + zIndex: number + visible: boolean + // Computed bounds for hit testing + bounds: Bounds +} + +export interface SlotLayout { + id: SlotId + nodeId: NodeId + position: Point // Relative to node + type: 'input' | 'output' + index: number +} + +export interface ConnectionLayout { + id: ConnectionId + sourceSlot: SlotId + targetSlot: SlotId + // Control points for curved connections + controlPoints?: Point[] +} + +// Mutation types +export type LayoutMutationType = + | 'moveNode' + | 'resizeNode' + | 'setNodeZIndex' + | 'createNode' + | 'deleteNode' + | 'batch' + +export interface LayoutMutation { + type: LayoutMutationType + timestamp: number + source: 'canvas' | 'vue' | 'external' +} + +export interface MoveNodeMutation extends LayoutMutation { + type: 'moveNode' + nodeId: NodeId + position: Point + previousPosition?: Point +} + +export interface ResizeNodeMutation extends LayoutMutation { + type: 'resizeNode' + nodeId: NodeId + size: Size + previousSize?: Size +} + +export interface SetNodeZIndexMutation extends LayoutMutation { + type: 'setNodeZIndex' + nodeId: NodeId + zIndex: number + previousZIndex?: number +} + +export interface CreateNodeMutation extends LayoutMutation { + type: 'createNode' + nodeId: NodeId + layout: NodeLayout +} + +export interface DeleteNodeMutation extends LayoutMutation { + type: 'deleteNode' + nodeId: NodeId + previousLayout?: NodeLayout +} + +export interface BatchMutation extends LayoutMutation { + type: 'batch' + mutations: AnyLayoutMutation[] +} + +// Union type for all mutations +export type AnyLayoutMutation = + | MoveNodeMutation + | ResizeNodeMutation + | SetNodeZIndexMutation + | CreateNodeMutation + | DeleteNodeMutation + | BatchMutation + +// Change notification types +export interface LayoutChange { + type: 'create' | 'update' | 'delete' + nodeIds: NodeId[] + timestamp: number + source: 'canvas' | 'vue' | 'external' + operation: LayoutOperation +} + +// Store interfaces +export interface LayoutStore { + // CustomRef accessors for shared write access + getNodeLayoutRef(nodeId: NodeId): Ref + getNodesInBounds(bounds: Bounds): ComputedRef + getAllNodes(): ComputedRef> + getVersion(): ComputedRef + + // Spatial queries (non-reactive) + queryNodeAtPoint(point: Point): NodeId | null + queryNodesInBounds(bounds: Bounds): NodeId[] + + // Direct mutation API (CRDT-ready) + applyOperation(operation: LayoutOperation): void + + // Change subscription + onChange(callback: (change: LayoutChange) => void): () => void + + // Initialization + initializeFromLiteGraph( + nodes: Array<{ id: string; pos: [number, number]; size: [number, number] }> + ): void + + // Source and actor management + setSource(source: 'canvas' | 'vue' | 'external'): void + setActor(actor: string): void + getCurrentSource(): 'canvas' | 'vue' | 'external' + getCurrentActor(): string +} + +// Re-export operation types from dedicated operations file +export type { + LayoutOperation as AnyLayoutOperation, + BaseOperation, + MoveNodeOperation, + ResizeNodeOperation, + SetNodeZIndexOperation, + CreateNodeOperation, + DeleteNodeOperation, + SetNodeVisibilityOperation, + BatchUpdateOperation, + OperationType, + OperationApplicator, + OperationSerializer, + ConflictResolver +} from './layoutOperations' + +// Simplified mutation API +export interface LayoutMutations { + // Single node operations (synchronous, CRDT-ready) + moveNode(nodeId: NodeId, position: Point): void + resizeNode(nodeId: NodeId, size: Size): void + setNodeZIndex(nodeId: NodeId, zIndex: number): void + + // Lifecycle operations + createNode(nodeId: NodeId, layout: Partial): void + deleteNode(nodeId: NodeId): void + + // Stacking operations + bringNodeToFront(nodeId: NodeId): void + + // Source tracking + setSource(source: 'canvas' | 'vue' | 'external'): void + setActor(actor: string): void // For CRDT +} + +// CRDT-ready operation log (for future CRDT integration) +export interface OperationLog { + operations: LayoutOperation[] + addOperation(operation: LayoutOperation): void + getOperationsSince(timestamp: number): LayoutOperation[] + getOperationsByActor(actor: string): LayoutOperation[] +} diff --git a/tests-ui/tests/stores/layoutStore.test.ts b/tests-ui/tests/stores/layoutStore.test.ts new file mode 100644 index 0000000000..93ad887d21 --- /dev/null +++ b/tests-ui/tests/stores/layoutStore.test.ts @@ -0,0 +1,249 @@ +import { beforeEach, describe, expect, it } from 'vitest' + +import { layoutStore } from '@/stores/layoutStore' +import type { NodeLayout } from '@/types/layoutTypes' + +describe('layoutStore CRDT operations', () => { + beforeEach(() => { + // Clear the store before each test + layoutStore.initializeFromLiteGraph([]) + }) + // Helper to create test node data + const createTestNode = (id: string): NodeLayout => ({ + id, + position: { x: 100, y: 100 }, + size: { width: 200, height: 100 }, + zIndex: 0, + visible: true, + bounds: { x: 100, y: 100, width: 200, height: 100 } + }) + + it('should create and retrieve nodes', () => { + const nodeId = 'test-node-1' + const layout = createTestNode(nodeId) + + // Create node + layoutStore.setSource('external') + layoutStore.applyOperation({ + type: 'createNode', + nodeId, + layout, + timestamp: Date.now(), + source: 'external', + actor: 'test' + }) + + // Retrieve node + const nodeRef = layoutStore.getNodeLayoutRef(nodeId) + expect(nodeRef.value).toEqual(layout) + }) + + it('should move nodes', () => { + const nodeId = 'test-node-2' + const layout = createTestNode(nodeId) + + // Create node first + layoutStore.applyOperation({ + type: 'createNode', + nodeId, + layout, + timestamp: Date.now(), + source: 'external', + actor: 'test' + }) + + // Move node + const newPosition = { x: 200, y: 300 } + layoutStore.applyOperation({ + type: 'moveNode', + nodeId, + position: newPosition, + previousPosition: layout.position, + timestamp: Date.now(), + source: 'vue', + actor: 'test' + }) + + // Verify position updated + const nodeRef = layoutStore.getNodeLayoutRef(nodeId) + expect(nodeRef.value?.position).toEqual(newPosition) + }) + + it('should resize nodes', () => { + const nodeId = 'test-node-3' + const layout = createTestNode(nodeId) + + // Create node + layoutStore.applyOperation({ + type: 'createNode', + nodeId, + layout, + timestamp: Date.now(), + source: 'external', + actor: 'test' + }) + + // Resize node + const newSize = { width: 300, height: 150 } + layoutStore.applyOperation({ + type: 'resizeNode', + nodeId, + size: newSize, + previousSize: layout.size, + timestamp: Date.now(), + source: 'canvas', + actor: 'test' + }) + + // Verify size updated + const nodeRef = layoutStore.getNodeLayoutRef(nodeId) + expect(nodeRef.value?.size).toEqual(newSize) + }) + + it('should delete nodes', () => { + const nodeId = 'test-node-4' + const layout = createTestNode(nodeId) + + // Create node + layoutStore.applyOperation({ + type: 'createNode', + nodeId, + layout, + timestamp: Date.now(), + source: 'external', + actor: 'test' + }) + + // Delete node + layoutStore.applyOperation({ + type: 'deleteNode', + nodeId, + previousLayout: layout, + timestamp: Date.now(), + source: 'external', + actor: 'test' + }) + + // Verify node deleted + const nodeRef = layoutStore.getNodeLayoutRef(nodeId) + expect(nodeRef.value).toBeNull() + }) + + it('should handle source and actor tracking', async () => { + const nodeId = 'test-node-5' + const layout = createTestNode(nodeId) + + // Set source and actor + layoutStore.setSource('vue') + layoutStore.setActor('user-123') + + // Track change notifications AFTER setting source/actor + const changes: any[] = [] + const unsubscribe = layoutStore.onChange((change) => { + changes.push(change) + }) + + // Create node + layoutStore.applyOperation({ + type: 'createNode', + nodeId, + layout, + timestamp: Date.now(), + source: layoutStore.getCurrentSource(), + actor: layoutStore.getCurrentActor() + }) + + // Wait for async notification + await new Promise((resolve) => setTimeout(resolve, 50)) + + expect(changes.length).toBeGreaterThanOrEqual(1) + const lastChange = changes[changes.length - 1] + expect(lastChange.source).toBe('vue') + expect(lastChange.operation.actor).toBe('user-123') + + unsubscribe() + }) + + it('should query nodes by spatial bounds', () => { + const nodes = [ + { id: 'node-a', position: { x: 0, y: 0 } }, + { id: 'node-b', position: { x: 100, y: 100 } }, + { id: 'node-c', position: { x: 250, y: 250 } } + ] + + // Create nodes with proper bounds + nodes.forEach(({ id, position }) => { + const layout: NodeLayout = { + ...createTestNode(id), + position, + bounds: { + x: position.x, + y: position.y, + width: 200, + height: 100 + } + } + layoutStore.applyOperation({ + type: 'createNode', + nodeId: id, + layout, + timestamp: Date.now(), + source: 'external', + actor: 'test' + }) + }) + + // Query nodes in bounds + const nodesInBounds = layoutStore.queryNodesInBounds({ + x: 50, + y: 50, + width: 200, + height: 200 + }) + + // node-a: (0,0) to (200,100) - overlaps with query bounds (50,50) to (250,250) + // node-b: (100,100) to (300,200) - overlaps with query bounds + // node-c: (250,250) to (450,350) - touches corner of query bounds + expect(nodesInBounds).toContain('node-a') + expect(nodesInBounds).toContain('node-b') + expect(nodesInBounds).toContain('node-c') + }) + + it('should maintain operation history', () => { + const nodeId = 'test-node-history' + const layout = createTestNode(nodeId) + const startTime = Date.now() + + // Create node + layoutStore.applyOperation({ + type: 'createNode', + nodeId, + layout, + timestamp: startTime, + source: 'external', + actor: 'test-actor' + }) + + // Move node + layoutStore.applyOperation({ + type: 'moveNode', + nodeId, + position: { x: 150, y: 150 }, + previousPosition: { x: 100, y: 100 }, + timestamp: startTime + 100, + source: 'vue', + actor: 'test-actor' + }) + + // Get operations by actor + const operations = layoutStore.getOperationsByActor('test-actor') + expect(operations.length).toBeGreaterThanOrEqual(2) + expect(operations[0].type).toBe('createNode') + expect(operations[1].type).toBe('moveNode') + + // Get operations since timestamp + const recentOps = layoutStore.getOperationsSince(startTime + 50) + expect(recentOps.length).toBeGreaterThanOrEqual(1) + expect(recentOps[0].type).toBe('moveNode') + }) +})