Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
08a179a
feat: add support for `getLocFromIndex` and `getIndexFromLoc`
lumirlumir May 24, 2025
c0b4cc4
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir May 28, 2025
5f6fb42
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir Jun 5, 2025
adc573f
wip: complete `getLocFromIndex`
lumirlumir Jun 5, 2025
8def91f
wip: resolve ts error
lumirlumir Jun 5, 2025
b9f5a17
wip: add tests for types
lumirlumir Jun 5, 2025
9c2c153
wip: complete tests for `getLocFromIndex`
lumirlumir Jun 7, 2025
7655663
wip: update `README.md`
lumirlumir Jun 7, 2025
bdb8f11
wip: add `@public`
lumirlumir Jun 7, 2025
b970ccc
wip: add more test cases
lumirlumir Jun 7, 2025
1e319ee
wip: add type tests for `getIndexFromLoc`
lumirlumir Jun 7, 2025
953a16c
wip: update `getIndexFromLoc`
lumirlumir Jun 7, 2025
4977cb2
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir Jun 7, 2025
e1df65c
wip: add more test cases for `getLocFromIndex`
lumirlumir Jun 8, 2025
bc963b3
Merge branch 'feat-add-support-for-getlocfromindex-and-getindexfromlo…
lumirlumir Jun 8, 2025
c31843f
wip: complete `getIndexFromLoc`
lumirlumir Jun 8, 2025
a831324
wip: add more test cases
lumirlumir Jun 8, 2025
eaa212d
wip: add more tests
lumirlumir Jun 8, 2025
478b8d5
wip: update error message
lumirlumir Jun 16, 2025
498402f
wip: more detailed test cases
lumirlumir Jun 16, 2025
fbd565e
fix: improve error messages for column range validation
lumirlumir Jun 16, 2025
e4b789b
wip: remove `@ts-ignore`
lumirlumir Jun 16, 2025
bc5c6de
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir Jun 16, 2025
4844c96
wip: retrieve `lineStart` and `columnStart` from AST
lumirlumir Jun 19, 2025
c0f6bab
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir Jun 22, 2025
ed7dffc
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir Jul 2, 2025
bd21283
wip: lazy caculation
lumirlumir Jul 2, 2025
b7a4abf
wip: refactor `findLineNumberBinarySearch`
lumirlumir Jul 3, 2025
3ef01a9
wip: refactor `#lines`
lumirlumir Jul 3, 2025
2098f91
wip: refactor `#setLineColumnStart`
lumirlumir Jul 3, 2025
c4e80dd
wip: freeze `#lines` and add test cases
lumirlumir Jul 3, 2025
82591a6
wip: fix CI
lumirlumir Jul 3, 2025
1a5022e
wip: complete refactor (maybe?)
lumirlumir Jul 3, 2025
3c5fad7
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir Jul 4, 2025
2d9d12b
wip: add more test cases
lumirlumir Jul 4, 2025
c70890a
wip: add more test cases
lumirlumir Jul 4, 2025
59b3f2e
fix: wrong console logging
lumirlumir Jul 4, 2025
bb13197
wip: cleanup
lumirlumir Jul 4, 2025
1750960
wip: lazily caculate `#lines`
lumirlumir Jul 4, 2025
2cef189
wip: create `#rootNodeLoc`
lumirlumir Jul 4, 2025
1d1022a
wip: remove `#lineStart`
lumirlumir Jul 4, 2025
0e4cac0
wip: cleanup
lumirlumir Jul 4, 2025
1c0531c
wip: remove `@ts-expect-error`
lumirlumir Jul 4, 2025
1e3ba06
wip: refactor `rootNodeLoc`
lumirlumir Jul 5, 2025
bf02009
wip: refactor `getLocFromIndex` and add update test cases
lumirlumir Jul 5, 2025
aebf8d2
wip: add more test cases
lumirlumir Jul 5, 2025
710f945
wip: complete `getLocFromIndex`
lumirlumir Jul 5, 2025
23510e0
wip: modify comments
lumirlumir Jul 5, 2025
7a6e57f
wip: update test cases
lumirlumir Jul 5, 2025
0d98c6b
wip: remove unnecessary test
lumirlumir Jul 5, 2025
2fe8cd5
wip: refactor line ending calculation
lumirlumir Jul 5, 2025
75e156f
wip: add early return logic
lumirlumir Jul 5, 2025
b1e5249
wip: refactor `#ensureLineStartIndicesFromLoc`
lumirlumir Jul 5, 2025
9d1f541
wip: cleanup using destructuring
lumirlumir Jul 9, 2025
123118c
wip: calculate lines together
lumirlumir Jul 9, 2025
91f3b8e
wip: calculate lines together
lumirlumir Jul 9, 2025
98691ad
wip: refactor code to reduce redundency
lumirlumir Jul 9, 2025
ae465b6
wip: refactor `#ensureLineStartIndicesFromLoc`
lumirlumir Jul 10, 2025
8afa6ac
wip: refactor `#ensureLineStartIndicesFromLoc`
lumirlumir Jul 10, 2025
ffb9b54
wip: rename var
lumirlumir Jul 10, 2025
13014b4
wip: rename to `#fineNextLine`
lumirlumir Jul 11, 2025
3d3448a
wip: refactor `additonalLinesNeeded`
lumirlumir Jul 11, 2025
1296ace
wip: replace `structuredClone` with `RegExp`
lumirlumir Jul 11, 2025
af494ed
Merge branch 'main' into feat-add-support-for-getlocfromindex-and-get…
lumirlumir Aug 2, 2025
0651a8f
wip: resolve issues with `y` flag
lumirlumir Aug 2, 2025
9131119
Merge branch 'feat-add-support-for-getlocfromindex-and-getindexfromlo…
lumirlumir Aug 2, 2025
92f1094
wip: add notes in `README.md`
lumirlumir Aug 2, 2025
0ae05bf
wip: add test case
lumirlumir Aug 2, 2025
842dfb8
fix: multi-character line break sequence
lumirlumir Aug 2, 2025
df39059
wip: save calculation by using short-circuit mechanism
lumirlumir Aug 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/plugin-kit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,8 @@ The `TextSourceCodeBase` class is intended to be a base class that has several o

- `lines` - an array of text lines that is created automatically when the constructor is called.
- `getLoc(node)` - gets the location of a node. Works for nodes that have the ESLint-style `loc` property and nodes that have the Unist-style [`position` property](https://github.com/syntax-tree/unist?tab=readme-ov-file#position). If you're using an AST with a different location format, you'll still need to implement this method yourself.
- `getLocFromIndex(index)` - Converts a source text index into a `{ line: number, column: number }` pair. (For this method to work, the root node should always cover the entire source code text, and the `getLoc()` method needs to be implemented correctly.)
- `getIndexFromLoc(loc)` - Converts a `{ line: number, column: number }` pair into a source text index. (For this method to work, the root node should always cover the entire source code text, and the `getLoc()` method needs to be implemented correctly.)
- `getRange(node)` - gets the range of a node within the source text. Works for nodes that have the ESLint-style `range` property and nodes that have the Unist-style [`position` property](https://github.com/syntax-tree/unist?tab=readme-ov-file#position). If you're using an AST with a different range format, you'll still need to implement this method yourself.
- `getText(nodeOrToken, charsBefore, charsAfter)` - gets the source text for the given node or token that has range information attached. Optionally, can return additional characters before and after the given node or token. As long as `getRange()` is properly implemented, this method will just work.
- `getAncestors(node)` - returns the ancestry of the node. In order for this to work, you must implement the `getParent()` method yourself.
Expand Down
270 changes: 266 additions & 4 deletions packages/plugin-kit/src/source-code.js
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,32 @@ function hasPosStyleRange(node) {
return "position" in node;
}

/**
* Performs binary search to find the line number containing a given target index.
* Returns the lower bound - the index of the first element greater than the target.
* **Please note that the `lineStartIndices` should be sorted in ascending order**.
* - Time Complexity: O(log n) - Significantly faster than linear search for large files.
* @param {number[]} lineStartIndices Sorted array of line start indices.
* @param {number} targetIndex The target index to find the line number for.
* @returns {number} The line number for the target index.
*/
function findLineNumberBinarySearch(lineStartIndices, targetIndex) {
let low = 0;
let high = lineStartIndices.length - 1;

while (low < high) {
const mid = ((low + high) / 2) | 0; // Use bitwise OR to floor the division.

if (targetIndex < lineStartIndices[mid]) {
high = mid;
} else {
low = mid + 1;
}
}

return low;
}

//-----------------------------------------------------------------------------
// Exports
//-----------------------------------------------------------------------------
Expand Down Expand Up @@ -216,15 +242,27 @@ export class Directive {

/**
* Source Code Base Object
* @template {SourceCodeBaseTypeOptions & {SyntaxElementWithLoc: object}} [Options=SourceCodeBaseTypeOptions & {SyntaxElementWithLoc: object}]
* @template {SourceCodeBaseTypeOptions & {RootNode: object, SyntaxElementWithLoc: object}} [Options=SourceCodeBaseTypeOptions & {RootNode: object, SyntaxElementWithLoc: object}]
* @implements {TextSourceCode<Options>}
*/
export class TextSourceCodeBase {
/**
* The lines of text in the source code.
* @type {Array<string>}
*/
#lines;
#lines = [];

/**
* The indices of the start of each line in the source code.
* @type {Array<number>}
*/
#lineStartIndices = [0];

/**
* The pattern to match lineEndings in the source code.
* @type {RegExp}
*/
#lineEndingPattern;

/**
* The AST of the source code.
Expand All @@ -243,12 +281,105 @@ export class TextSourceCodeBase {
* @param {Object} options The options for the instance.
* @param {string} options.text The source code text.
* @param {Options['RootNode']} options.ast The root AST node.
* @param {RegExp} [options.lineEndingPattern] The pattern to match lineEndings in the source code.
* @param {RegExp} [options.lineEndingPattern] The pattern to match lineEndings in the source code. Defaults to `/\r?\n/u`.
*/
constructor({ text, ast, lineEndingPattern = /\r?\n/u }) {
this.ast = ast;
this.text = text;
this.#lines = text.split(lineEndingPattern);
// Remove the global(`g`) and sticky(`y`) flags from the `lineEndingPattern` to avoid issues with lastIndex.
this.#lineEndingPattern = new RegExp(
lineEndingPattern.source,
lineEndingPattern.flags.replace(/[gy]/gu, ""),
);
}

/**
* Finds the next line in the source text and updates `#lines` and `#lineStartIndices`.
* @param {string} text The text to search for the next line.
* @returns {boolean} `true` if a next line was found, `false` otherwise.
*/
#findNextLine(text) {
const match = this.#lineEndingPattern.exec(text);

if (!match) {
return false;
}

this.#lines.push(text.slice(0, match.index));
this.#lineStartIndices.push(
(this.#lineStartIndices.at(-1) ?? 0) +
match.index +
match[0].length,
);

return true;
}

/**
* Ensures `#lines` is lazily calculated from the source text.
* @returns {void}
*/
#ensureLines() {
// If `#lines` has already been calculated, do nothing.
if (this.#lines.length === this.#lineStartIndices.length) {
return;
}

while (
this.#findNextLine(this.text.slice(this.#lineStartIndices.at(-1)))
) {
// Continue parsing until no more matches are found.
}

this.#lines.push(this.text.slice(this.#lineStartIndices.at(-1)));

Object.freeze(this.#lines);
}

/**
* Ensures `#lineStartIndices` is lazily calculated up to the specified index.
* @param {number} index The index of a character in a file.
* @returns {void}
*/
#ensureLineStartIndicesFromIndex(index) {
// If we've already parsed up to or beyond this index, do nothing.
if (index <= (this.#lineStartIndices.at(-1) ?? 0)) {
return;
}

while (
index > (this.#lineStartIndices.at(-1) ?? 0) &&
this.#findNextLine(this.text.slice(this.#lineStartIndices.at(-1)))
) {
// Continue parsing until no more matches are found.
}
}

/**
* Ensures `#lineStartIndices` is lazily calculated up to the specified loc.
* @param {Object} loc A line/column location.
* @param {number} loc.line The line number of the location. (0 or 1-indexed based on language.)
* @param {number} lineStart The line number at which the parser starts counting.
* @returns {void}
*/
#ensureLineStartIndicesFromLoc(loc, lineStart) {
// Calculate line indices up to the potentially next line, as it is needed for the follow‑up calculation.
const nextLocLineIndex = loc.line - lineStart + 1;
const lastCalculatedLineIndex = this.#lineStartIndices.length - 1;
let additionalLinesNeeded = nextLocLineIndex - lastCalculatedLineIndex;

// If we've already parsed up to or beyond this line, do nothing.
if (additionalLinesNeeded <= 0) {
return;
}

while (
additionalLinesNeeded > 0 &&
this.#findNextLine(this.text.slice(this.#lineStartIndices.at(-1)))
) {
// Continue parsing until no more matches are found or we have enough lines.
additionalLinesNeeded -= 1;
}
}

/**
Expand All @@ -271,6 +402,135 @@ export class TextSourceCodeBase {
);
}

/**
* Converts a source text index into a `{ line: number, column: number }` pair.
* @param {number} index The index of a character in a file.
* @throws {TypeError|RangeError} If non-numeric index or index out of range.
* @returns {{line: number, column: number}} A `{ line: number, column: number }` location object with 0 or 1-indexed line and 0 or 1-indexed column based on language.
* @public
*/
getLocFromIndex(index) {
if (typeof index !== "number") {
throw new TypeError("Expected `index` to be a number.");
}

if (index < 0 || index > this.text.length) {
throw new RangeError(
`Index out of range (requested index ${index}, but source text has length ${this.text.length}).`,
);
}

const {
start: { line: lineStart, column: columnStart },
end: { line: lineEnd, column: columnEnd },
} = this.getLoc(this.ast);

// If the index is at the start, return the start location of the root node.
if (index === 0) {
return {
line: lineStart,
column: columnStart,
};
}

// If the index is `this.text.length`, return the location one "spot" past the last character of the file.
if (index === this.text.length) {
return {
line: lineEnd,
column: columnEnd,
};
}

// Ensure `#lineStartIndices` are lazily calculated.
this.#ensureLineStartIndicesFromIndex(index);

/*
* To figure out which line `index` is on, determine the last place at which index could
* be inserted into `#lineStartIndices` to keep the list sorted.
*/
const lineNumber =
(index >= (this.#lineStartIndices.at(-1) ?? 0)
? this.#lineStartIndices.length
: findLineNumberBinarySearch(this.#lineStartIndices, index)) -
1 +
lineStart;

return {
line: lineNumber,
column:
index -
this.#lineStartIndices[lineNumber - lineStart] +
columnStart,
};
}

/**
* Converts a `{ line: number, column: number }` pair into a source text index.
* @param {Object} loc A line/column location.
* @param {number} loc.line The line number of the location. (0 or 1-indexed based on language.)
* @param {number} loc.column The column number of the location. (0 or 1-indexed based on language.)
* @throws {TypeError|RangeError} If `loc` is not an object with a numeric
* `line` and `column`, if the `line` is less than or equal to zero or
* the `line` or `column` is out of the expected range.
* @returns {number} The index of the line/column location in a file.
* @public
*/
getIndexFromLoc(loc) {
if (
loc === null ||
typeof loc !== "object" ||
typeof loc.line !== "number" ||
typeof loc.column !== "number"
) {
throw new TypeError(
"Expected `loc` to be an object with numeric `line` and `column` properties.",
);
}

const {
start: { line: lineStart, column: columnStart },
end: { line: lineEnd, column: columnEnd },
} = this.getLoc(this.ast);

if (loc.line < lineStart || lineEnd < loc.line) {
throw new RangeError(
`Line number out of range (line ${loc.line} requested). Valid range: ${lineStart}-${lineEnd}`,
);
}

// If the loc is at the start, return the start index of the root node.
if (loc.line === lineStart && loc.column === columnStart) {
return 0;
}

// If the loc is at the end, return the index one "spot" past the last character of the file.
if (loc.line === lineEnd && loc.column === columnEnd) {
return this.text.length;
}

// Ensure `#lineStartIndices` are lazily calculated.
this.#ensureLineStartIndicesFromLoc(loc, lineStart);

const isLastLine = loc.line === lineEnd;
const lineStartIndex = this.#lineStartIndices[loc.line - lineStart];
const lineEndIndex = isLastLine
? this.text.length
: this.#lineStartIndices[loc.line - lineStart + 1];
const positionIndex = lineStartIndex + loc.column - columnStart;

if (
loc.column < columnStart ||
(isLastLine && positionIndex > lineEndIndex) ||
(!isLastLine && positionIndex >= lineEndIndex)
) {
throw new RangeError(
`Column number out of range (column ${loc.column} requested). Valid range for line ${loc.line}: ${columnStart}-${lineEndIndex - lineStartIndex + columnStart + (isLastLine ? 0 : -1)}`,
);
}

return positionIndex;
}

/**
* Returns the range information for the given node or token.
* @param {Options['SyntaxElementWithLoc']} nodeOrToken The node or token to get the range information for.
Expand Down Expand Up @@ -356,6 +616,8 @@ export class TextSourceCodeBase {
* @public
*/
get lines() {
this.#ensureLines(); // Ensure `#lines` is lazily calculated.

return this.#lines;
}

Expand Down
Loading
Loading