Skip to content

Commit

Permalink
chore: formatting, add __all__
Browse files Browse the repository at this point in the history
  • Loading branch information
amaanq committed Nov 12, 2023
1 parent 9d19117 commit 882ece6
Show file tree
Hide file tree
Showing 8 changed files with 256 additions and 220 deletions.
99 changes: 57 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,55 @@
py-tree-sitter
==================
# py-tree-sitter

[![Build Status](https://github.com/tree-sitter/py-tree-sitter/actions/workflows/ci.yml/badge.svg)](https://github.com/tree-sitter/py-tree-sitter/actions/workflows/ci.yml)
[![Build status](https://ci.appveyor.com/api/projects/status/mde790v0v9gux85w/branch/master?svg=true)](https://ci.appveyor.com/project/maxbrunsfeld/py-tree-sitter/branch/master)

This module provides Python bindings to the [tree-sitter](https://github.com/tree-sitter/tree-sitter) parsing library.
This module provides Python bindings to the [tree-sitter](https://github.com/tree-sitter/tree-sitter)
parsing library.

## Installation

This package currently only works with Python 3. There are no library dependencies, but you do need to have a C compiler installed.
This package currently only works with Python 3. There are no library dependencies,
but you do need to have a C compiler installed.

```sh
pip3 install tree_sitter
```

## Usage

#### Setup
### Setup

First you'll need a Tree-sitter language implementation for each language that you want to parse. You can clone some of the [existing language repos](https://github.com/tree-sitter) or [create your own](http://tree-sitter.github.io/tree-sitter/creating-parsers):
First you'll need a Tree-sitter language implementation for each language that you
want to parse. You can clone some of the [existing language repos](https://github.com/tree-sitter)
or [create your own](http://tree-sitter.github.io/tree-sitter/creating-parsers):

```sh
git clone https://github.com/tree-sitter/tree-sitter-go
git clone https://github.com/tree-sitter/tree-sitter-javascript
git clone https://github.com/tree-sitter/tree-sitter-python
```

Use the `Language.build_library` method to compile these into a library that's usable from Python. This function will return immediately if the library has already been compiled since the last time its source code was modified:
Use the `Language.build_library` method to compile these into a library that's
usable from Python. This function will return immediately if the library has
already been compiled since the last time its source code was modified:

```python
from tree_sitter import Language, Parser
from tree_sitter import Language

Language.build_library(
# Store the library in the `build` directory
'build/my-languages.so',

# Include one or more languages
[
'vendor/tree-sitter-go',
'vendor/tree-sitter-javascript',
'vendor/tree-sitter-python'
]
# Store the library in the `build` directory
"build/my-languages.so",
# Include one or more languages
["vendor/tree-sitter-go", "vendor/tree-sitter-javascript", "vendor/tree-sitter-python"],
)
```

Load the languages into your app as `Language` objects:

```python
GO_LANGUAGE = Language('build/my-languages.so', 'go')
JS_LANGUAGE = Language('build/my-languages.so', 'javascript')
PY_LANGUAGE = Language('build/my-languages.so', 'python')
GO_LANGUAGE = Language("build/my-languages.so", "go")
JS_LANGUAGE = Language("build/my-languages.so", "javascript")
PY_LANGUAGE = Language("build/my-languages.so", "python")
```

#### Basic Parsing
Expand All @@ -64,11 +64,16 @@ parser.set_language(PY_LANGUAGE)
Parse some source code:

```python
tree = parser.parse(bytes("""
tree = parser.parse(
bytes(
"""
def foo():
if bar:
baz()
""", "utf8"))
""",
"utf8",
)
)
```

If you have your source code in some data structure other than a bytes object,
Expand All @@ -81,14 +86,19 @@ terminates parsing for that line. The bytes must encode the source as UTF-8.
For example, to use the byte offset:

```python
src = bytes("""
src = bytes(
"""
def foo():
if bar:
baz()
""", "utf8")
""",
"utf8",
)


def read_callable(byte_offset, point):
return src[byte_offset:byte_offset+1]
return src[byte_offset : byte_offset + 1]


tree = parser.parse(read_callable)
```
Expand All @@ -98,11 +108,13 @@ And to use the point:
```python
src_lines = ["def foo():\n", " if bar:\n", " baz()"]


def read_callable(byte_offset, point):
row, column = point
if row >= len(src_lines) or column >= len(src_lines[row]):
return None
return src_lines[row][column:].encode('utf8')
return src_lines[row][column:].encode("utf8")


tree = parser.parse(read_callable)
```
Expand Down Expand Up @@ -145,30 +157,31 @@ a `TreeCursor`:
```python
cursor = tree.walk()

assert cursor.node.type == 'module'
assert cursor.node.type == "module"

assert cursor.goto_first_child()
assert cursor.node.type == 'function_definition'
assert cursor.node.type == "function_definition"

assert cursor.goto_first_child()
assert cursor.node.type == 'def'
assert cursor.node.type == "def"

# Returns `False` because the `def` node has no children
assert not cursor.goto_first_child()

assert cursor.goto_next_sibling()
assert cursor.node.type == 'identifier'
assert cursor.node.type == "identifier"

assert cursor.goto_next_sibling()
assert cursor.node.type == 'parameters'
assert cursor.node.type == "parameters"

assert cursor.goto_parent()
assert cursor.node.type == 'function_definition'
assert cursor.node.type == "function_definition"
```

#### Editing

When a source file is edited, you can edit the syntax tree to keep it in sync with the source:
When a source file is edited, you can edit the syntax tree to keep it in sync with
the source:

```python
tree.edit(
Expand All @@ -190,30 +203,32 @@ new_tree = parser.parse(new_source, tree)

This will run much faster than if you were parsing from scratch.

The `Tree.get_changed_ranges` method can be called on the *old* tree to return
The `Tree.get_changed_ranges` method can be called on the _old_ tree to return
the list of ranges whose syntactic structure has been changed:

```python
for changed_range in tree.get_changed_ranges(new_tree):
print('Changed range:')
print(f' Start point {changed_range.start_point}')
print(f' Start byte {changed_range.start_byte}')
print(f' End point {changed_range.end_point}')
print(f' End byte {changed_range.end_byte}')
print("Changed range:")
print(f" Start point {changed_range.start_point}")
print(f" Start byte {changed_range.start_byte}")
print(f" End point {changed_range.end_point}")
print(f" End byte {changed_range.end_byte}")
```

#### Pattern-matching

You can search for patterns in a syntax tree using a *tree query*:
You can search for patterns in a syntax tree using a _tree query_:

```python
query = PY_LANGUAGE.query("""
query = PY_LANGUAGE.query(
"""
(function_definition
name: (identifier) @function.def)
(call
function: (identifier) @function.call)
""")
"""
)

captures = query.captures(tree.root_node)
assert len(captures) == 2
Expand Down
11 changes: 7 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
[build-system]
requires = [
"setuptools>=43.0.0",
"wheel>=0.36.2",
]
requires = ["setuptools>=43.0.0", "wheel>=0.36.2"]
build-backend = "setuptools.build_meta"

[tool.ruff]
line-length = 100

[tool.ruff.pycodestyle]
max-line-length = 102
2 changes: 1 addition & 1 deletion script/lint
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash

sources=$(git ls-files | grep '.py$')
flake8 --max-line-length 100 $sources
flake8 --max-line-length 102 --ignore=E203,W503 $sources
4 changes: 1 addition & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,7 @@
"Topic :: Software Development :: Compilers",
"Topic :: Text Processing :: Linguistic",
],
install_requires=[
"setuptools>=60.0.0; python_version>='3.12'"
],
install_requires=["setuptools>=60.0.0; python_version>='3.12'"],
packages=["tree_sitter"],
package_data={"tree_sitter": ["py.typed", "*.pyi"]},
ext_modules=[
Expand Down
Loading

0 comments on commit 882ece6

Please sign in to comment.