diff --git a/crates/oxc_allocator/README.md b/crates/oxc_allocator/README.md new file mode 100644 index 0000000000000..07ec7465c5988 --- /dev/null +++ b/crates/oxc_allocator/README.md @@ -0,0 +1,29 @@ +# Oxc Allocator + +A high-performance memory allocator using bump-based arena allocation for fast AST node creation. + +## Overview + +Oxc uses a bump-based memory arena for faster AST allocations. This crate provides an [`Allocator`] that manages memory efficiently by allocating objects in a single contiguous memory region, avoiding the overhead of individual heap allocations. + +## Key Features + +- **Bump allocation**: Fast allocation by simply incrementing a pointer +- **Arena-based memory management**: All allocations live for the same lifetime +- **Zero-copy data structures**: Efficient `Box`, `Vec`, `String`, and `HashMap` implementations +- **Optimal for AST operations**: Perfect for parse-transform-emit workflows + +## Architecture + +The allocator is designed specifically for AST processing workflows where: + +1. A large number of nodes are allocated during parsing +2. All nodes have the same lifetime (tied to the AST) +3. Memory is released all at once when the AST is dropped + +This approach is significantly faster than using the system allocator for AST operations. + +## Features + +- `serialize` - Enables serialization support for `Box` and `Vec` with `serde` +- `from_raw_parts` - Adds unsafe `from_raw_parts` method (not recommended for general use) diff --git a/crates/oxc_ast/README.md b/crates/oxc_ast/README.md new file mode 100644 index 0000000000000..b1b582aba9b70 --- /dev/null +++ b/crates/oxc_ast/README.md @@ -0,0 +1,35 @@ +# Oxc AST + +Abstract Syntax Tree definitions for JavaScript, TypeScript, and JSX. + +## Overview + +This crate provides comprehensive AST (Abstract Syntax Tree) node definitions that support the full spectrum of JavaScript and TypeScript syntax, including JSX. The AST closely follows ECMAScript specifications while providing ergonomic APIs for manipulation. + +## Key Features + +- **Complete language support**: JavaScript, TypeScript, JSX, and TSX +- **ECMAScript compliant**: AST structure follows official specifications +- **Memory efficient**: Designed to work with `oxc_allocator` for fast allocation +- **Visitor patterns**: Integrates with `oxc_ast_visit` for traversal +- **Type-safe**: Leverages Rust's type system for correctness + +## AST Design Principles + +The AST design differs from estree in several important ways: + +- **Explicit identifiers**: Uses specific types like `BindingIdentifier`, `IdentifierReference`, and `IdentifierName` instead of generic `Identifier` +- **Precise assignment targets**: `AssignmentExpression.left` uses `AssignmentTarget` instead of generic `Pattern` +- **Literal specialization**: Replaces generic `Literal` with `BooleanLiteral`, `NumericLiteral`, `StringLiteral`, etc. +- **Evaluation order**: Field order follows ECMAScript evaluation order for consistency + +## Architecture + +The AST is designed for: + +- **Parse**: Efficient construction during parsing +- **Transform**: Easy manipulation during transpilation +- **Visit**: Systematic traversal for analysis +- **Codegen**: Clean conversion back to source code + +All AST nodes are allocated in an arena (`oxc_allocator`) for optimal performance. diff --git a/crates/oxc_ast_macros/README.md b/crates/oxc_ast_macros/README.md new file mode 100644 index 0000000000000..d2f60d8a4d076 --- /dev/null +++ b/crates/oxc_ast_macros/README.md @@ -0,0 +1,32 @@ +# Oxc AST Macros + +Procedural macros for generating AST-related code and ensuring memory layout consistency. + +## Overview + +This crate provides procedural macros that generate boilerplate code for AST nodes, ensuring consistent memory layout and providing derived traits automatically. + +## Key Features + +- **`#[ast]` attribute**: Marks types as AST nodes and generates required traits +- **Memory layout control**: Ensures `#[repr(C)]` for predictable memory layout +- **Trait derivation**: Automatically derives common traits like `Debug`, `Clone`, etc. +- **Code generation**: Reduces boilerplate and ensures consistency across AST types + +## What the `#[ast]` Macro Does + +1. **Adds `#[repr(C)]`**: Ensures predictable memory layout across platforms +2. **Marker for tooling**: Identifies types as AST nodes for code generation tools +3. **Trait derivation**: Automatically implements common traits +4. **Consistency**: Ensures all AST nodes follow the same patterns + +## Architecture + +This macro system enables: + +- **Maintainable AST**: Reduces boilerplate across hundreds of AST types +- **Consistent layout**: Critical for performance and correctness +- **Tool integration**: Allows `oxc_ast_tools` to generate visitor code +- **Type safety**: Ensures all AST nodes have required traits + +The macros are designed to be transparent and generate minimal, efficient code. diff --git a/crates/oxc_ast_visit/README.md b/crates/oxc_ast_visit/README.md new file mode 100644 index 0000000000000..09391f2a891ab --- /dev/null +++ b/crates/oxc_ast_visit/README.md @@ -0,0 +1,25 @@ +# Oxc AST Visit + +Visitor pattern implementations for traversing and mutating AST nodes. + +## Overview + +This crate provides visitor traits and implementations for systematically traversing AST nodes. It supports both immutable visitation (`Visit`) and mutable transformation (`VisitMut`). + +## Key Features + +- **`Visit` trait**: Immutable AST traversal for analysis +- **`VisitMut` trait**: Mutable AST traversal for transformations +- **Generated implementations**: Most visitor code is auto-generated for consistency +- **UTF-8 to UTF-16 conversion**: Special visitors for span conversion + +## Architecture + +The visitor pattern is designed for: + +- **Analysis**: Static analysis, linting, and code inspection +- **Transformation**: Code modification and transpilation +- **Consistency**: Systematic traversal of all AST nodes +- **Performance**: Efficient traversal with minimal overhead + +Most visitor implementations are generated by `oxc_ast_tools` to ensure all AST nodes are covered and traversal is consistent. diff --git a/crates/oxc_cfg/README.md b/crates/oxc_cfg/README.md new file mode 100644 index 0000000000000..ee72e44ec4f7f --- /dev/null +++ b/crates/oxc_cfg/README.md @@ -0,0 +1,40 @@ +# Oxc CFG + +Control Flow Graph construction and analysis for JavaScript and TypeScript. + +## Overview + +This crate provides data structures and algorithms for building and analyzing Control Flow Graphs (CFGs) from AST nodes. CFGs are essential for advanced static analysis, optimization, and understanding program flow. + +## Key Features + +- **CFG construction**: Build control flow graphs from AST nodes +- **Block-based representation**: Organizes code into basic blocks +- **Graph analysis**: Traverse and analyze control flow patterns +- **DOT export**: Visualize CFGs using Graphviz dot format +- **Visitor integration**: Works with oxc visitor patterns + +## Architecture + +### Basic Blocks + +The CFG organizes code into basic blocks - sequences of instructions with: + +- Single entry point (first instruction) +- Single exit point (last instruction) +- No internal branches or jumps + +### Graph Structure + +- **Nodes**: Basic blocks containing instructions +- **Edges**: Control flow between blocks (conditional, unconditional, exception) +- **Entry/Exit**: Special blocks for function entry and exit points + +### Analysis Applications + +- **Dead code elimination**: Find unreachable blocks +- **Data flow analysis**: Track variable usage across control paths +- **Loop detection**: Identify back edges and loop structures +- **Path analysis**: Enumerate possible execution paths + +The CFG integrates with semantic analysis to provide comprehensive program understanding. diff --git a/crates/oxc_codegen/README.md b/crates/oxc_codegen/README.md new file mode 100644 index 0000000000000..607bd2190d79d --- /dev/null +++ b/crates/oxc_codegen/README.md @@ -0,0 +1,33 @@ +# Oxc Codegen + +High-performance code generation from AST back to JavaScript/TypeScript source code. + +## Overview + +This crate converts AST nodes back into source code strings, supporting JavaScript, TypeScript, and JSX. It's designed for speed and correctness, producing clean, readable output. + +## Key Features + +- **Fast code generation**: Optimized for performance with minimal allocations +- **Source map support**: Generate accurate source maps during output +- **Configurable formatting**: Control whitespace, semicolons, and other formatting options +- **Comment preservation**: Maintain comments during code generation +- **Binary expression optimization**: Intelligent parentheses insertion + +## Architecture + +### Code Generation Pipeline + +1. **AST Traversal**: Walk through AST nodes systematically +2. **Token Generation**: Convert nodes to appropriate tokens/strings +3. **Formatting**: Apply whitespace, indentation, and style rules +4. **Source Mapping**: Track original source positions if enabled + +### Design Principles + +- **Correctness**: Generated code must be functionally equivalent to original +- **Performance**: Minimize string allocations and copying +- **Readability**: Produce clean, well-formatted output +- **Fidelity**: Preserve semantic meaning and behavior + +The codegen is adapted from esbuild's approach, optimized for Rust and oxc's AST structure. diff --git a/crates/oxc_data_structures/README.md b/crates/oxc_data_structures/README.md new file mode 100644 index 0000000000000..8f5531156afc2 --- /dev/null +++ b/crates/oxc_data_structures/README.md @@ -0,0 +1,26 @@ +# Oxc Data Structures + +Common data structures and utilities used across oxc crates. + +## Overview + +This crate provides specialized data structures and utilities that are used throughout the oxc toolchain. These structures are optimized for the specific needs of compiler and tooling workloads. + +## Key Features + +- **Code buffer**: Efficient string building with segment tracking +- **Inline strings**: Memory-efficient string storage for short strings +- **Pointer extensions**: Utilities for safe pointer manipulation +- **Slice iterators**: Enhanced iteration capabilities for slices +- **Rope data structure**: Efficient text manipulation for large documents + +## Architecture + +These data structures are designed with specific compiler requirements in mind: + +- **Performance**: Optimized for common patterns in parsing and code generation +- **Memory efficiency**: Minimize allocations and memory overhead +- **Safety**: Provide safe abstractions over potentially unsafe operations +- **Ergonomics**: Easy to use APIs that integrate well with other oxc components + +The structures complement Rust's standard library with domain-specific optimizations for JavaScript/TypeScript tooling. diff --git a/crates/oxc_diagnostics/README.md b/crates/oxc_diagnostics/README.md new file mode 100644 index 0000000000000..e4bbccaa48e26 --- /dev/null +++ b/crates/oxc_diagnostics/README.md @@ -0,0 +1,33 @@ +# Oxc Diagnostics + +Error reporting and diagnostic utilities for JavaScript and TypeScript tooling. + +## Overview + +This crate provides comprehensive error handling and diagnostic reporting capabilities. It implements the [miette] diagnostic trait, making it compatible with other Rust diagnostic tooling while providing specialized features for JavaScript/TypeScript errors. + +## Key Features + +- **Rich diagnostics**: Detailed error messages with source context +- **Source highlighting**: Syntax-highlighted error locations +- **Multiple error support**: Collect and report multiple errors at once +- **Miette integration**: Compatible with the miette diagnostic ecosystem +- **Severity levels**: Support for errors, warnings, and informational messages + +## Architecture + +### Diagnostic Components + +- **Message**: Primary error/warning description +- **Labels**: Highlight specific source locations +- **Help text**: Suggestions for fixing the issue +- **Source context**: Display relevant source code sections + +### Error Flow + +1. **Creation**: Tools create `OxcDiagnostic` instances for problems +2. **Collection**: `DiagnosticService` aggregates diagnostics +3. **Formatting**: Rich terminal output with colors and context +4. **Reporting**: Display to users in IDE, CLI, or other interfaces + +The diagnostic system ensures consistent, high-quality error reporting across all oxc tools. diff --git a/crates/oxc_estree/README.md b/crates/oxc_estree/README.md new file mode 100644 index 0000000000000..71bf9ef7d6a1f --- /dev/null +++ b/crates/oxc_estree/README.md @@ -0,0 +1,38 @@ +# Oxc ESTree + +ESTree compatibility layer for serialization and interoperability. + +## Overview + +This crate provides compatibility with the ESTree AST specification, primarily for serialization purposes. It enables oxc AST nodes to be serialized to and from JSON in ESTree format, facilitating interoperability with other JavaScript tools. + +## Key Features + +- **ESTree compatibility**: Convert oxc AST to/from standard ESTree format +- **Serialization support**: JSON serialization/deserialization via serde +- **Tool interoperability**: Enable integration with ESTree-based tools +- **Optional feature**: Only enabled when `serialize` feature is active + +## Usage + +## Architecture + +### ESTree Specification + +ESTree is a community standard for representing JavaScript AST nodes. This crate ensures oxc's AST can be represented in this standard format while maintaining compatibility with the broader JavaScript tooling ecosystem. + +### Design Principles + +- **Compatibility**: Full compatibility with ESTree specification +- **Optional overhead**: Only included when serialization is needed +- **Type safety**: Maintains Rust's type safety during conversion +- **Performance**: Efficient serialization with minimal overhead + +### Use Cases + +- **IDE integration**: Language servers communicating via JSON +- **Tool interoperability**: Working with Babel, ESLint, and other ESTree tools +- **Data exchange**: Transferring AST data between different systems +- **Debugging**: Human-readable AST representation + +When the `serialize` feature is disabled, this crate provides only a placeholder trait to support derive macros without overhead. diff --git a/crates/oxc_isolated_declarations/README.md b/crates/oxc_isolated_declarations/README.md new file mode 100644 index 0000000000000..c5d73829de3ba --- /dev/null +++ b/crates/oxc_isolated_declarations/README.md @@ -0,0 +1,40 @@ +# Oxc Isolated Declarations + +TypeScript isolated declarations transformer for generating `.d.ts` files. + +## Overview + +This crate implements TypeScript's isolated declarations feature, which generates TypeScript declaration files (`.d.ts`) from source code without requiring full type checking. This enables faster builds and better incremental compilation. + +## Key Features + +- **Fast declaration generation**: Generate `.d.ts` files without full type checking +- **TypeScript 5.5+ compatibility**: Implements the latest isolated declarations specification +- **Incremental builds**: Enables faster TypeScript compilation workflows +- **Comprehensive support**: Handles classes, functions, interfaces, and complex types + +## Architecture + +### Isolated Declarations Concept + +Isolated declarations allow generating TypeScript declaration files without full type inference by requiring that: + +- All exported functions have explicit return types +- All exported variables have explicit types +- Type information is locally available + +### Implementation Details + +- **AST transformation**: Convert implementation AST to declaration AST +- **Type extraction**: Extract and preserve type information +- **Export analysis**: Determine what needs to be included in declarations +- **Error reporting**: Provide helpful diagnostics for missing type annotations + +### Benefits + +- **Faster builds**: No full type checking required +- **Incremental compilation**: Each file can be processed independently +- **Parallel processing**: Multiple files can be processed simultaneously +- **Simplified tooling**: Easier to integrate into build systems + +This implementation follows the TypeScript compiler's approach while leveraging oxc's performance-oriented architecture. diff --git a/crates/oxc_linter/README.md b/crates/oxc_linter/README.md new file mode 100644 index 0000000000000..cdcd47a7068d4 --- /dev/null +++ b/crates/oxc_linter/README.md @@ -0,0 +1,40 @@ +# Oxc Linter + +High-performance ESLint-compatible linter for JavaScript and TypeScript. + +## Overview + +This crate provides a fast, ESLint-compatible linting engine with comprehensive rule support. It leverages oxc's high-performance AST and semantic analysis to deliver significantly faster linting than traditional tools. + +## Key Features + +- **ESLint compatibility**: Drop-in replacement for most ESLint workflows +- **High performance**: 50-100x faster than ESLint on large codebases +- **Comprehensive rules**: Supports most popular ESLint rules and plugins +- **TypeScript support**: Full TypeScript linting with type-aware rules +- **Plugin system**: Extensible architecture for custom rules + +## Architecture + +### Rule System + +- **Rule categories**: Correctness, Suspicious, Pedantic, Style, etc. +- **Configurable severity**: Error, Warning, or Off for each rule +- **Auto-fixing**: Many rules can automatically fix issues +- **Plugin support**: Load external rule collections + +### Performance Design + +- **Single AST pass**: Most rules run in a single traversal +- **Efficient analysis**: Leverages oxc's semantic analysis +- **Parallel execution**: Process multiple files concurrently +- **Memory efficiency**: Uses arena allocation for optimal performance + +### ESLint Compatibility + +- **Configuration format**: Supports ESLint config files +- **Rule parity**: Implements behavior-compatible versions of popular rules +- **Plugin ecosystem**: Compatible with many ESLint plugins +- **Migration path**: Easy transition from ESLint to oxlint + +The linter is designed to be both a standalone tool and a library component for integration into other development tools. diff --git a/crates/oxc_macros/README.md b/crates/oxc_macros/README.md new file mode 100644 index 0000000000000..8cf3df3a48ff0 --- /dev/null +++ b/crates/oxc_macros/README.md @@ -0,0 +1,38 @@ +# Oxc Macros + +Procedural macros for declaring lint rules and other oxc components. + +## Overview + +This crate provides procedural macros that simplify the declaration and implementation of lint rules and other oxc components. These macros reduce boilerplate code and ensure consistent patterns across the codebase. + +## Key Features + +- **`declare_oxc_lint!`**: Macro for declaring lint rules with metadata +- **Rule documentation**: Auto-generates documentation for website +- **Category management**: Organize rules into logical categories +- **Boilerplate reduction**: Eliminates repetitive rule declaration code + +## Architecture + +### Macro System + +The macros generate: + +- Rule struct definitions +- Metadata for documentation generation +- Registration code for the rule system +- Consistent interfaces across all rules + +### Documentation Generation + +Rule documentation is automatically extracted and used to build the oxc website documentation, ensuring that rule descriptions stay in sync with implementation. + +### Benefits + +- **Consistency**: All rules follow the same declaration pattern +- **Documentation**: Automatic documentation generation +- **Type safety**: Compile-time verification of rule metadata +- **Maintainability**: Centralized rule management + +This macro system enables rapid development of new lint rules while maintaining high quality and consistency. diff --git a/crates/oxc_mangler/README.md b/crates/oxc_mangler/README.md new file mode 100644 index 0000000000000..9430c187b224a --- /dev/null +++ b/crates/oxc_mangler/README.md @@ -0,0 +1,53 @@ +# Oxc Mangler + +Variable name mangling for JavaScript minification. + +## Overview + +This crate provides variable name mangling capabilities for JavaScript minification. It shortens variable names to reduce bundle size while preserving program semantics and ensuring gzip-friendly output. + +## Key Features + +- **Intelligent mangling**: Shortens variable names while avoiding conflicts +- **Scope-aware**: Respects JavaScript scoping rules +- **Gzip optimization**: Generates names that compress well +- **Configurable**: Options to preserve specific names or patterns +- **Base54 encoding**: Efficient character usage for maximum compression + +## Architecture + +### Mangling Strategy + +1. **Symbol Analysis**: Identify all variables and their scopes +2. **Frequency Analysis**: Count usage to optimize for common variables +3. **Name Generation**: Generate short names using base54 encoding +4. **Conflict Resolution**: Ensure no naming conflicts across scopes +5. **Integration**: Update symbol table with new names + +### Base54 Encoding + +Uses a character set optimized for JavaScript identifiers: + +- First character: `a-zA-Z_$` (54 options) +- Subsequent characters: `a-zA-Z0-9_$` (64 options) +- Generates shortest possible names: `a`, `b`, ..., `aa`, `ab`, etc. + +### Gzip Optimization + +The mangling algorithm considers: + +- **Character frequency**: Prefer characters that compress well +- **Repetition patterns**: Generate names that create gzip-friendly patterns +- **Context awareness**: Consider surrounding code when choosing names + +### Integration with Minifier + +The mangler works as part of the broader minification pipeline: + +1. **Parse**: Build AST from source code +2. **Analyze**: Perform semantic analysis +3. **Mangle**: Shorten variable names +4. **Transform**: Apply other minification transforms +5. **Generate**: Output minified code + +This approach ensures maximum size reduction while maintaining correctness. diff --git a/crates/oxc_napi/README.md b/crates/oxc_napi/README.md new file mode 100644 index 0000000000000..3cbc4c8145e1b --- /dev/null +++ b/crates/oxc_napi/README.md @@ -0,0 +1,51 @@ +# Oxc NAPI + +Node.js Native API bindings for oxc tools. + +## Overview + +This crate provides Node.js bindings for oxc tools, enabling JavaScript/TypeScript applications to use oxc's high-performance parsing, linting, and transformation capabilities through native Node.js modules. + +## Key Features + +- **Native performance**: Direct access to oxc's Rust implementation +- **UTF-16 compatibility**: Automatic span conversion for JavaScript strings +- **Error handling**: Proper JavaScript error propagation +- **Comment preservation**: Maintains comments during processing +- **Type safety**: TypeScript definitions for all APIs + +## Architecture + +### UTF-8 to UTF-16 Conversion + +JavaScript uses UTF-16 string encoding, while Rust uses UTF-8. This crate handles the conversion: + +- **Span conversion**: Updates all source positions to UTF-16 offsets +- **Comment handling**: Preserves comment positions during conversion +- **Error mapping**: Ensures error positions are correct in JavaScript + +### Node.js Integration + +The bindings are designed for: + +- **npm packages**: Used by `@oxc-project/` npm packages +- **Build tools**: Integration with Webpack, Vite, and other bundlers +- **Editor support**: Language server and editor extension features +- **CLI tools**: Command-line interfaces for Node.js environments + +### Error Handling + +Provides JavaScript-friendly error handling: + +- **OxcError**: Rust errors converted to JavaScript exceptions +- **Diagnostic integration**: Rich error messages with source context +- **Stack traces**: Proper error propagation to JavaScript + +### Performance Considerations + +- **Zero-copy**: Minimizes data copying between Rust and JavaScript +- **Efficient conversion**: Optimized UTF-8 to UTF-16 conversion +- **Memory management**: Proper cleanup of Rust resources +- **Async support**: Non-blocking operations where appropriate + +This crate enables the broader JavaScript ecosystem to benefit from oxc's performance while maintaining familiar JavaScript APIs. diff --git a/crates/oxc_parser/README.md b/crates/oxc_parser/README.md new file mode 100644 index 0000000000000..319d4091b5ead --- /dev/null +++ b/crates/oxc_parser/README.md @@ -0,0 +1,42 @@ +# Oxc Parser + +High-performance JavaScript and TypeScript parser with comprehensive language support. + +## Overview + +This crate provides a fast, spec-compliant parser for JavaScript and TypeScript that produces a complete Abstract Syntax Tree (AST). It supports all modern language features including JSX, TSX, and the latest ECMAScript proposals. + +## Key Features + +- **Complete language support**: JavaScript, TypeScript, JSX, and TSX +- **Latest ECMAScript**: All stable ECMAScript features plus stage 3+ proposals +- **High performance**: Significantly faster than traditional parsers +- **Error recovery**: Continues parsing after errors to provide complete AST +- **Comprehensive AST**: Detailed node information with accurate source positions + +## Architecture + +### Parser Design + +- **Recursive descent**: Traditional recursive descent parser architecture +- **Error recovery**: Sophisticated error recovery for IDE-friendly parsing +- **Memory efficient**: Uses arena allocation for optimal performance +- **Streaming**: Processes source text in a single pass + +### Language Support + +- **JavaScript**: Full ECMAScript 2024+ support +- **TypeScript**: Complete TypeScript syntax including decorators +- **JSX/TSX**: React JSX and TypeScript JSX syntax +- **Proposals**: Stage 3 decorators and other advancing proposals + +### AST Structure + +The parser produces an AST that closely follows ECMAScript specifications: + +- **Accurate positions**: Every node has precise source location information +- **Complete information**: Preserves all syntactic details including trivia +- **Type-safe**: Leverages Rust's type system for correctness +- **Visitor-friendly**: Designed for easy traversal and transformation + +The parser is designed as the foundation for all other oxc tools, providing the high-quality AST needed for analysis, transformation, and code generation. diff --git a/crates/oxc_semantic/README.md b/crates/oxc_semantic/README.md new file mode 100644 index 0000000000000..4cb6ee5e3ce13 --- /dev/null +++ b/crates/oxc_semantic/README.md @@ -0,0 +1,56 @@ +# Oxc Semantic + +Comprehensive semantic analysis for JavaScript and TypeScript programs. + +## Overview + +This crate performs semantic analysis on JavaScript and TypeScript ASTs, building symbol tables, scope trees, and control flow graphs. It provides the foundation for advanced static analysis, linting, and transformation tools. + +## Key Features + +- **Symbol resolution**: Build complete symbol tables with binding information +- **Scope analysis**: Construct scope trees following ECMAScript scoping rules +- **Reference tracking**: Track all variable references and their relationships +- **Control flow**: Optional control flow graph construction +- **JSDoc parsing**: Extract and parse JSDoc comments +- **Module analysis**: Analyze import/export relationships + +## Architecture + +### Semantic Analysis Pipeline + +1. **AST Traversal**: Visit all nodes to collect declarations +2. **Scope Building**: Construct scope tree following language rules +3. **Symbol Resolution**: Create symbol table with binding information +4. **Reference Analysis**: Track all identifier references +5. **Control Flow**: Optionally build control flow graphs +6. **JSDoc Processing**: Parse and attach documentation + +### Key Data Structures + +#### Symbol Table + +- **Symbols**: All declared identifiers (variables, functions, classes, etc.) +- **References**: All uses of identifiers +- **Bindings**: Association between names and symbols in scopes + +#### Scope Tree + +- **Scope hierarchy**: Nested scopes following language semantics +- **Binding resolution**: How identifiers resolve to declarations +- **Closure analysis**: Capture and usage patterns + +#### Control Flow Graph + +- **Basic blocks**: Sequences of statements with single entry/exit +- **Flow edges**: Conditional and unconditional control flow +- **Exception handling**: Try/catch/finally flow modeling + +### Applications + +- **Linting**: Detect unused variables, undefined references, etc. +- **Transformation**: Safe variable renaming and scope analysis +- **Analysis**: Dead code detection, dependency analysis +- **IDE features**: Go-to-definition, find references, refactoring + +The semantic analyzer provides the deep program understanding needed for sophisticated JavaScript and TypeScript tooling. diff --git a/crates/oxc_span/README.md b/crates/oxc_span/README.md new file mode 100644 index 0000000000000..e54a083bc4169 --- /dev/null +++ b/crates/oxc_span/README.md @@ -0,0 +1,51 @@ +# Oxc Span + +Source position utilities and span management for precise error reporting. + +## Overview + +This crate provides essential utilities for managing source positions and spans in JavaScript and TypeScript code. It enables precise error reporting, source mapping, and location-aware transformations across all oxc tools. + +## Key Features + +- **Precise positions**: Accurate byte-level source positions +- **Span utilities**: Source ranges with start and end positions +- **Source type detection**: Automatic language and module detection +- **Atom interning**: Efficient string storage for identifiers +- **Content comparison**: Semantic equality for AST nodes + +## Architecture + +### Span System + +- **Byte positions**: All positions are UTF-8 byte offsets +- **Inclusive ranges**: Spans include start position, exclude end position +- **Efficiency**: Uses u32 for positions, supporting files up to 4GB +- **Precision**: Enables precise error highlighting and source maps + +### Source Type System + +The source type encodes important metadata: + +- **Language**: JavaScript vs TypeScript +- **Module system**: ESM vs Script +- **JSX support**: JSX vs plain syntax +- **Variant**: Standard vs definition files (.d.ts) + +### Atom Interning + +Atoms provide memory-efficient string storage: + +- **Deduplication**: Identical strings share memory +- **Fast comparison**: Pointer equality for identical content +- **Compact representation**: Reduced memory usage for identifiers + +### Integration Points + +- **Parser**: Creates spans during tokenization and parsing +- **Semantic**: Tracks spans for all symbols and references +- **Linter**: Uses spans for precise error reporting +- **Codegen**: Maintains spans for source map generation +- **Transformer**: Preserves spans during AST manipulation + +This crate provides the foundation for precise source location tracking throughout the oxc toolchain. diff --git a/crates/oxc_syntax/README.md b/crates/oxc_syntax/README.md new file mode 100644 index 0000000000000..ace759496b6de --- /dev/null +++ b/crates/oxc_syntax/README.md @@ -0,0 +1,63 @@ +# Oxc Syntax + +Common JavaScript and TypeScript syntax definitions and utilities. + +## Overview + +This crate provides shared syntax definitions, constants, and utilities used across the oxc toolchain. It contains fundamental types and enums that represent JavaScript and TypeScript language constructs. + +## Key Features + +- **Syntax constants**: Keywords, operators, and language tokens +- **AST utilities**: Common patterns and helper functions for AST manipulation +- **Identifier validation**: ECMAScript identifier checking and utilities +- **Operator precedence**: Precedence tables for expression parsing +- **Module records**: Import/export relationship tracking + +## Key Components + +## Architecture + +### Design Principles + +- **Shared definitions**: Avoid duplication across oxc crates +- **Type safety**: Use newtypes for different kinds of IDs +- **Performance**: Efficient representations for common operations +- **Standards compliance**: Follow ECMAScript specifications + +### Core Components + +#### Language Constants + +- **Keywords**: All JavaScript/TypeScript reserved words +- **Operators**: Binary, unary, and assignment operators +- **Tokens**: Punctuation and special symbols + +#### AST Utilities + +- **Node flags**: Metadata about AST nodes (computed properties, etc.) +- **Traversal helpers**: Common patterns for walking AST trees +- **Type guards**: Runtime type checking for AST nodes + +#### Semantic Types + +- **IDs**: Type-safe identifiers for nodes, symbols, scopes, references +- **Flags**: Bitfield metadata for semantic entities +- **Relationships**: Parent-child and reference relationships + +#### Module System + +- **Import/Export tracking**: Comprehensive module dependency analysis +- **Resolution**: Module specifier resolution utilities +- **Metadata**: Module type and format information + +### Integration + +This crate serves as the foundation for: + +- **Parser**: Uses syntax definitions during tokenization +- **Semantic**: Leverages ID types and flags for analysis +- **Linter**: References operator precedence and keyword tables +- **Transformer**: Uses module records for import/export handling + +The syntax crate ensures consistency and type safety across all oxc components. diff --git a/crates/oxc_transformer_plugins/README.md b/crates/oxc_transformer_plugins/README.md new file mode 100644 index 0000000000000..e5d3c32ac863e --- /dev/null +++ b/crates/oxc_transformer_plugins/README.md @@ -0,0 +1,56 @@ +# Oxc Transformer Plugins + +Specialized transformation plugins for common JavaScript and TypeScript patterns. + +## Overview + +This crate provides specific transformation plugins that can be used independently or as part of the broader oxc transformation pipeline. These plugins handle common transformation needs like global variable injection and module format conversion. + +## Key Features + +- **Global variable injection**: Add global variables to modules +- **Module runner transforms**: Transform modules for different execution environments +- **Global defines replacement**: Replace compile-time constants +- **Composable plugins**: Use individually or combine multiple plugins + +## Architecture + +### Plugin System + +Each plugin implements the transformation logic for a specific use case: + +- **Focused responsibility**: Each plugin handles one type of transformation +- **Traverse integration**: Uses oxc's traversal system for efficient AST walking +- **Composable**: Multiple plugins can be applied in sequence +- **Context-aware**: Access to semantic information when needed + +### Common Use Cases + +#### Build-time Optimizations + +- **Dead code elimination**: Remove unreachable code paths +- **Constant folding**: Replace compile-time constants +- **Environment variables**: Inject environment-specific values + +#### Module System Compatibility + +- **Polyfill injection**: Add necessary polyfills for target environments +- **Global shimming**: Provide Node.js globals in browser environments +- **Format conversion**: Transform between module formats + +#### Development Tools + +- **Debug injection**: Add development-time debugging code +- **Hot module replacement**: Support for HMR systems +- **Testing utilities**: Inject test-specific globals + +### Integration Points + +These plugins integrate with: + +- **oxc_transformer**: Core transformation infrastructure +- **oxc_traverse**: AST traversal and mutation +- **Build tools**: Webpack, Vite, Rollup, and other bundlers +- **CLI tools**: Direct usage in command-line transformations + +The plugin architecture enables extensible, composable transformations while maintaining high performance through oxc's efficient traversal system. diff --git a/crates/oxc_traverse/README.md b/crates/oxc_traverse/README.md new file mode 100644 index 0000000000000..5739a13e39a7e --- /dev/null +++ b/crates/oxc_traverse/README.md @@ -0,0 +1,66 @@ +# Oxc Traverse + +Advanced AST traversal with parent context and efficient tree navigation. + +## Overview + +This crate provides sophisticated AST traversal capabilities that allow visitors to read up the tree from any node. Unlike traditional visitors that only provide downward traversal, oxc_traverse enables accessing parent nodes and sibling contexts during traversal. + +## Key Features + +- **Parent context access**: Read parent nodes during traversal +- **Efficient navigation**: Navigate up and down the AST tree +- **Memory safety**: Statically prevents aliasing violations +- **Traverse context**: Rich context information during traversal +- **Generated visitors**: Most traversal code is auto-generated for consistency + +## Architecture + +### Traversal Context + +The `TraverseCtx` provides rich information during traversal: + +- **Parent stack**: Complete chain of parent nodes +- **Scope context**: Current scope and scope hierarchy +- **Symbol information**: Access to semantic analysis results +- **AST utilities**: Helper methods for common operations + +### Memory Safety Design + +The traversal system prevents Rust aliasing violations through: + +- **Controlled access**: Only safe references are provided to visitors +- **Stack-based parents**: Parent information without direct references +- **Immutable ancestors**: Read-only access to ancestor nodes +- **Mutable current**: Safe mutable access to current node + +### Code Generation + +Most traversal code is generated to ensure: + +- **Complete coverage**: All AST nodes have traversal methods +- **Consistency**: Uniform traversal patterns across node types +- **Performance**: Optimized traversal with minimal overhead +- **Maintainability**: Automatic updates when AST changes + +### Use Cases + +#### Static Analysis + +- **Linting**: Check code patterns with parent context +- **Dependency analysis**: Track imports/exports with scope awareness +- **Security analysis**: Detect dangerous patterns in context + +#### Code Transformation + +- **Transpilation**: Transform syntax with contextual awareness +- **Optimization**: Apply optimizations based on usage patterns +- **Refactoring**: Safe code modifications with full context + +#### Code Generation + +- **Template processing**: Generate code with contextual information +- **Macro expansion**: Expand macros with scope awareness +- **AST construction**: Build new AST nodes with proper context + +The traverse system enables sophisticated transformations that would be difficult or impossible with traditional visitor patterns, while maintaining Rust's safety guarantees.