-
Notifications
You must be signed in to change notification settings - Fork 5
Add a simple example without Langium #59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
cc688eb
Add tokenizer and parser with tests
Lotes a945fd5
Add type system file
Lotes 6debdef
Add type system
Lotes 7a27dc6
Add a first test
Lotes bf69612
Make validations work
Lotes 0100977
Add at least one negative test
Lotes 1ba0b49
Remove duplicated entry
Lotes ea82184
Save so far
Lotes 75fba20
Add a custom validation
Lotes 02c8a93
Add better README
Lotes 82d87fd
Update Readme
Lotes 291db6a
renamed files
JohannesMeierSE 9a3305f
adapted example to the current Typir API
JohannesMeierSE 44f717d
simplified validation
JohannesMeierSE d84f7da
updated descriptions, configurations and clean scripts
JohannesMeierSE File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| { | ||
| "typescript.tsdk": "node_modules/typescript/lib" | ||
| "typescript.tsdk": "node_modules/typescript/lib" | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| # A handwritten parser | ||
|
|
||
| This package contains a handwritten parser for a simple expression language. | ||
| The language supports: | ||
|
|
||
| - Variables declarations with the types `number` and `string` | ||
| - Variable Assignments | ||
| - Arithmetic expressions | ||
| - Print statements | ||
| - Expressions, like basic arithmetic operations, string concatenation, variable references, literals and parentheses | ||
|
|
||
| ## How does it work? | ||
JohannesMeierSE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| Parsing is a linear process that takes a string of text and produces a tree-like structure that represents the structure of the text. | ||
|
|
||
| ```mermaid | ||
| flowchart LR | ||
| A[Lexer] --> B[Parser] | ||
| CC@{shape: brace-r, label: "Typir is applied here"} --> C | ||
| B --> C[Type System] | ||
| C --> D[Validator] | ||
|
|
||
| style C fill:#f9f,stroke:#333,stroke-width:4px | ||
| ``` | ||
|
|
||
| The following sections describe each step in the process. | ||
|
|
||
| ### Lexer | ||
|
|
||
| **Input**: A string of text | ||
|
|
||
| **Output**: A list of tokens | ||
|
|
||
| **Task**: Splits the text to tokens and classifies each token. | ||
|
|
||
| ```mermaid | ||
| flowchart LR | ||
| subgraph Text | ||
| A["variable = 123"] | ||
| end | ||
| A --> B[Lexer] | ||
| B --> Tokens | ||
| subgraph Tokens | ||
| T1[variable:ID] | ||
| T2[=:ASSIGN] | ||
| T3[123:NUMBER] | ||
| end | ||
| ``` | ||
|
|
||
| ### Parser | ||
|
|
||
| **Input**: A list of tokens | ||
|
|
||
| **Output**: An Abstract Syntax Tree (AST) | ||
|
|
||
| **Task**: Takes token and arranges them as a tree. | ||
|
|
||
| ```mermaid | ||
| flowchart LR | ||
| subgraph Tokens | ||
| T1[variable:ID] | ||
| T2[=:ASSIGN] | ||
| T3[123:NUMBER] | ||
| end | ||
|
|
||
| Tokens --> D[Parser] | ||
| subgraph AST | ||
| EE1[variable] | ||
| EE2[=] | ||
| EE3[123] | ||
| EE2 --> EE1 | ||
| EE2 --> EE3 | ||
| end | ||
| D --> AST | ||
| ``` | ||
|
|
||
| ### Type system | ||
|
|
||
| **Input**: An AST | ||
|
|
||
| **Output**: A typed AST | ||
|
|
||
| **Task**: Assigns types to the nodes of the AST. | ||
|
|
||
| ```mermaid | ||
| flowchart LR | ||
| subgraph AST | ||
| EE1[variable] | ||
| EE2[=] | ||
| EE3[123] | ||
| EE2 --> EE1 | ||
| EE2 --> EE3 | ||
| end | ||
| FF@{shape: brace-r, label: "described by Typir"} --> F | ||
| AST --> F[Type System] | ||
| F --> AST2 | ||
| subgraph AST2["Typed AST"] | ||
| FF1[variable:STRING] | ||
| FF2[=] | ||
| FF3[123:NUMBER] | ||
| FF2 --> FF1 | ||
| FF2 --> FF3 | ||
| end | ||
|
|
||
| style F fill:#f9f,stroke:#333,stroke-width:4px | ||
| ``` | ||
|
|
||
| ### Validator | ||
|
|
||
| **Input**: A typed AST | ||
|
|
||
| **Output**: a list of errors | ||
|
|
||
| **Task**: Checks if the AST is valid. | ||
|
|
||
| ```mermaid | ||
| flowchart LR | ||
| subgraph AST["Typed AST"] | ||
| FF1[variable:STRING] | ||
| FF2[=] | ||
| FF3[123:NUMBER] | ||
| FF2 --> FF1 | ||
| FF2 --> FF3 | ||
| end | ||
| AST --> H[Validator] | ||
| H --> Errors | ||
| subgraph Errors | ||
| I1["Variable got wrong type assigned"] | ||
| end | ||
| style I1 fill:#fdd,stroke:#333,stroke-width:4px | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| { | ||
| "name": "typir-example-expression", | ||
| "displayName": "expression", | ||
| "version": "0.2.0", | ||
| "private": true, | ||
| "description": "", | ||
| "author": { | ||
| "name": "TypeFox", | ||
| "url": "https://www.typefox.io" | ||
| }, | ||
| "license": "MIT", | ||
| "type": "module", | ||
| "engines": { | ||
| "vscode": "^1.67.0" | ||
| }, | ||
| "volta": { | ||
| "node": "18.20.4", | ||
| "npm": "10.7.0" | ||
| }, | ||
| "scripts": { | ||
| "build": "tsc -b tsconfig.json", | ||
| "clean": "shx rm -rf out *.tsbuildinfo", | ||
| "lint": "eslint src --ext ts", | ||
| "test": "vitest", | ||
| "watch": "concurrently -n tsc -c blue \"tsc -b tsconfig.json --watch\"" | ||
| }, | ||
| "dependencies": { | ||
| "typir": "~0.2.0" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| /****************************************************************************** | ||
| * Copyright 2025 TypeFox GmbH | ||
| * This program and the accompanying materials are made available under the | ||
| * terms of the MIT License, which is available in the project root. | ||
| ******************************************************************************/ | ||
| export interface BinaryExpression { | ||
| type: 'binary'; | ||
| left: Expression; | ||
| right: Expression; | ||
| op: '+'|'-'|'/'|'*'|'%'; | ||
| } | ||
|
|
||
| export function isBinaryExpression(node: unknown): node is BinaryExpression { | ||
| return isAstNode(node) && node.type === 'binary'; | ||
| } | ||
|
|
||
| export interface UnaryExpression { | ||
| type: 'unary'; | ||
| operand: Expression; | ||
| op: '+'|'-'; | ||
| } | ||
|
|
||
| export function isUnaryExpression(node: unknown): node is UnaryExpression { | ||
| return isAstNode(node) && node.type === 'unary'; | ||
| } | ||
|
|
||
| export interface VariableUsage { | ||
| type: 'variable-usage'; | ||
| ref: VariableDeclaration; | ||
| } | ||
|
|
||
|
|
||
| export function isVariableUsage(node: unknown): node is VariableUsage { | ||
| return isAstNode(node) && node.type === 'variable-usage'; | ||
| } | ||
|
|
||
|
|
||
| export interface Numeric { | ||
| type: 'numeric'; | ||
| value: number; | ||
| } | ||
|
|
||
| export function isNumeric(node: unknown): node is Numeric { | ||
| return isAstNode(node) && node.type === 'numeric'; | ||
| } | ||
|
|
||
| export interface CharString { | ||
| type: 'string'; | ||
| value: string; | ||
| } | ||
|
|
||
| export function isCharString(node: unknown): node is CharString { | ||
| return isAstNode(node) && node.type === 'string'; | ||
| } | ||
|
|
||
| export type Expression = UnaryExpression | BinaryExpression | VariableUsage | Numeric | CharString; | ||
|
|
||
| export interface VariableDeclaration { | ||
| type: 'variable-declaration'; | ||
| name: string; | ||
| value: Expression; | ||
| } | ||
|
|
||
| export function isVariableDeclaration(node: unknown): node is VariableDeclaration { | ||
| return isAstNode(node) && node.type === 'variable-declaration'; | ||
| } | ||
|
|
||
| export interface Assignment { | ||
| type: 'assignment'; | ||
| variable: VariableDeclaration; | ||
| value: Expression; | ||
| } | ||
|
|
||
| export function isAssignment(node: unknown): node is Assignment { | ||
| return isAstNode(node) && node.type === 'assignment'; | ||
| } | ||
|
|
||
|
|
||
| export interface Printout { | ||
| type: 'printout'; | ||
| value: Expression; | ||
| } | ||
|
|
||
| export function isPrintout(node: unknown): node is Printout { | ||
| return isAstNode(node) && node.type === 'printout'; | ||
| } | ||
|
|
||
| export type Statement = VariableDeclaration | Printout | Assignment; | ||
|
|
||
| export type Model = Statement[]; | ||
|
|
||
| export type Node = Expression | Printout | VariableDeclaration | Assignment; | ||
|
|
||
| export function isAstNode(node: unknown): node is Node { | ||
| return Object.getOwnPropertyNames(node).includes('type') && ['variable-usage', 'unary', 'binary', 'numeric', 'string', 'printout', 'variable-declaration', 'assignment'].includes((node as Node).type); | ||
| } | ||
|
|
||
| export namespace AST { | ||
| export function variable(name: string, value: Expression): VariableDeclaration { | ||
| return { type: 'variable-declaration', name, value }; | ||
| } | ||
| export function assignment(variable: VariableDeclaration, value: Expression): Assignment { | ||
| return { type: 'assignment', variable, value }; | ||
| } | ||
| export function printout(value: Expression): Printout { | ||
| return { type: 'printout', value }; | ||
| } | ||
| export function num(value: number): Numeric { | ||
| return { | ||
| type: 'numeric', | ||
| value | ||
| }; | ||
| } | ||
| export function string(value: string): CharString { | ||
| return { | ||
| type: 'string', | ||
| value | ||
| }; | ||
| } | ||
| export function binary(left: Expression, op: BinaryExpression['op'], right: Expression): BinaryExpression { | ||
| return { | ||
| type: 'binary', | ||
| left, | ||
| op, | ||
| right | ||
| }; | ||
| } | ||
| export function unary(op: UnaryExpression['op'], operand: Expression): UnaryExpression { | ||
| return { | ||
| type: 'unary', | ||
| op, | ||
| operand | ||
| }; | ||
| } | ||
| export function useVariable(variable: VariableDeclaration): VariableUsage { | ||
| return { | ||
| ref: variable, | ||
| type: 'variable-usage' | ||
| }; | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.