From 1d84929678ff3d034eec3dab159196d543d6e508 Mon Sep 17 00:00:00 2001 From: "Jim.Idle" Date: Wed, 17 May 2023 11:33:19 +0800 Subject: [PATCH 1/6] doc: Update the readme at the root of the Go Runtime - Prevent people from trying to use the module in the non-v4 directory - Redirect people to github.com/antlr4-go/antlr Signed-off-by: Jim.Idle --- runtime/Go/antlr/README.adoc | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/runtime/Go/antlr/README.adoc b/runtime/Go/antlr/README.adoc index e219799cbc..f3503bd320 100644 --- a/runtime/Go/antlr/README.adoc +++ b/runtime/Go/antlr/README.adoc @@ -1,4 +1,17 @@ -= Migration to v4 from non tagged code += Migration to v4 -Please note that the source code that was previously located here is now located in the `/v4` subdirectory to accommodate -go modules. If you are not using modules, then use the code in the `/v4` subdirectory from here on out. +== If you are using `GOPATH` and not modules + +Please note that the source code that was previously located in this directory is now located in the official release repository at: github.com/antlr4-go/antlr please use the code in that repo if you have a reason not to use modules. + +== If you are using modules + +Your driver code etc. should now be importing from the new release only repo for the runtime: + +```go +import ( + github.com/antlr4-go/antlr +) +``` + +Please consult From 1a63942873641dac1a0860cb49009f5588189822 Mon Sep 17 00:00:00 2001 From: "Jim.Idle" Date: Wed, 17 May 2023 13:58:41 +0800 Subject: [PATCH 2/6] doc: Update the main package docs for the upcoming release - Point to the new release only Go repo - Modify and clarify project layout suggestions Signed-off-by: Jim.Idle --- runtime/Go/antlr/v4/antlrdoc.go | 53 +++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 16 deletions(-) diff --git a/runtime/Go/antlr/v4/antlrdoc.go b/runtime/Go/antlr/v4/antlrdoc.go index 61576b722f..e7ddaa8c24 100644 --- a/runtime/Go/antlr/v4/antlrdoc.go +++ b/runtime/Go/antlr/v4/antlrdoc.go @@ -11,15 +11,24 @@ From a grammar, ANTLR generates a parser that can build parse trees and also gen # Go Runtime At version 4.11.x and prior, the Go runtime was not properly versioned for go modules. After this point, the runtime -source code is held in the `runtime/Go/antlr/v4` directory, and the go.mod file is updated to reflect the version of -ANTLR4 that it is compatible with (I.E. uses the /v4 path). The runtime is now available as a go module, and can be -imported as `github.com/antlr/antlr4/runtime/Go/antlr/v4` (the go get command should also be used with this path). See -the main documentation for the ANTLR4 project for more information. +source code to be imported was held in the `runtime/Go/antlr/v4` directory, and the go.mod file was updated to reflect the version of +ANTLR4 that it is compatible with (I.E. uses the /v4 path). -This means that if you are using the source code without modules, you should also use the source code in /v4. Though -we highly recommend that you use go modules, as they are now idiomatic Go. +However, this was found to be problematic, as it meant that with the runtime embedded so far underneath the root +of the repo, the `go get` and related commands could not properly resolve the location of the go runtime source code. +This meant that the reference to the runtime in your `go.mod` file would refer to the correct source code, but would not +list the release tag such as @4.12.0 - this was confusing, to say the least. -I am aware that this change will prove Hyrum's Law, but am prepared to live with it for teh common good. JI +As of 4.12.1, the runtime is now available as a go module in its own repo, and can be imported as `github.com/antlr-go/antlr` +(the go get command should also be used with this path). See the main documentation for the ANTLR4 project for more information, +which is available at [ANTLR docs]. The documentation for using the Go runtime is available at [Go runtime docs]. + +This means that if you are using the source code without modules, you should also use the source code in the [new repo]. +Though we highly recommend that you use go modules, as they are now idiomatic for Go. + +I am aware that this change will prove Hyrum's Law, but am prepared to live with it for the common good. + +Go runtime author: [Jim Idle] jimi@idle.ws # Code Generation @@ -30,26 +39,28 @@ runtime for the Go target. To generate code for the go target, it is generally recommended to place the source grammar files in a package of their own, and use the `.sh` script method of generating code, using the go generate directive. In that same directory it is usual, though not required, to place the antlr tool that should be used to generate the code. That does mean -that the antlr tool JAR file will be checked in to your source code control though, so you are free to use any other +that the antlr tool JAR file will be checked in to your source code control though, so you are, of course, free to use any other way of specifying the version of the ANTLR tool to use, such as aliasing in `.zshrc` or equivalent, or a profile in -your IDE, or configuration in your CI system. +your IDE, or configuration in your CI system. Checking in the jar does mean that it is easy to reproduce the build as +it was at any point in its history. -Here is a general template for an ANTLR based recognizer in Go: +Here is a general/recommended template for an ANTLR based recognizer in Go: . - ├── myproject ├── parser │ ├── mygrammar.g4 │ ├── antlr-4.12.0-complete.jar │ ├── error_listeners.go │ ├── generate.go │ ├── generate.sh + ├── parsing - generated code goes here ├── go.mod ├── go.sum ├── main.go └── main_test.go -Make sure that the package statement in your grammar file(s) reflects the go package they exist in. +Make sure that the package statement in your grammar file(s) reflects the go package the generated code will exist in. + The generate.go file then looks like this: package parser @@ -60,15 +71,21 @@ And the generate.sh file will look similar to this: #!/bin/sh - alias antlr4='java -Xmx500M -cp "./antlr4-4.12.0-complete.jar:$CLASSPATH" org.antlr.v4.Tool' - antlr4 -Dlanguage=Go -no-visitor -package parser *.g4 + alias antlr4='java -Xmx500M -cp "./antlr4-4.12.1-complete.jar:$CLASSPATH" org.antlr.v4.Tool' + antlr4 -Dlanguage=Go -no-visitor -package parsing *.g4 -depending on whether you want visitors or listeners or any other ANTLR options. +depending on whether you want visitors or listeners or any other ANTLR options. Not that another option here +is to generate the code into a -From the command line at the root of your package “myproject” you can then simply issue the command: +From the command line at the root of your source package (location of go.mo)d) you can then simply issue the command: go generate ./... +Which will generate the code for the parser, and place it in the parsing package. You can then use the generated code +by importing the parsing package. + +There are no hard and fast rules on this. It is just a recommendation. You can generate the code in any way and to anywhere you like. + # Copyright Notice Copyright (c) 2012-2023 The ANTLR Project. All rights reserved. @@ -77,5 +94,9 @@ Use of this file is governed by the BSD 3-clause license, which can be found in [target languages]: https://github.com/antlr/antlr4/tree/master/runtime [LICENSE.txt]: https://github.com/antlr/antlr4/blob/master/LICENSE.txt +[ANTLR docs]: https://github.com/antlr/antlr4/blob/master/doc/index.md +[new repo]: https://github.com/antlr-go/antlr +[Jim Idle]: https://github.com/jimidle +[Go runtime docs]: https://github.com/antlr/antlr4/blob/master/doc/go-target.md */ package antlr From a4591d737567babff92b360e4510b27b8e72c123 Mon Sep 17 00:00:00 2001 From: "Jim.Idle" Date: Wed, 17 May 2023 14:53:48 +0800 Subject: [PATCH 3/6] docs: Small tweak to represent generated code location in a neater fashion Signed-off-by: Jim.Idle --- runtime/Go/antlr/v4/antlrdoc.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/runtime/Go/antlr/v4/antlrdoc.go b/runtime/Go/antlr/v4/antlrdoc.go index e7ddaa8c24..c04eeea0d2 100644 --- a/runtime/Go/antlr/v4/antlrdoc.go +++ b/runtime/Go/antlr/v4/antlrdoc.go @@ -50,10 +50,10 @@ Here is a general/recommended template for an ANTLR based recognizer in Go: ├── parser │ ├── mygrammar.g4 │ ├── antlr-4.12.0-complete.jar - │ ├── error_listeners.go │ ├── generate.go - │ ├── generate.sh + │ └── generate.sh ├── parsing - generated code goes here + │ └── error_listeners.go ├── go.mod ├── go.sum ├── main.go From 46b3f25f96f184d64ddb9b28c3a144a74ad8dfc7 Mon Sep 17 00:00:00 2001 From: "Jim.Idle" Date: Wed, 17 May 2023 15:05:02 +0800 Subject: [PATCH 4/6] doc: Bump version of the ANTLR tool specified in the docs Signed-off-by: Jim.Idle --- runtime/Go/antlr/v4/antlrdoc.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/runtime/Go/antlr/v4/antlrdoc.go b/runtime/Go/antlr/v4/antlrdoc.go index c04eeea0d2..f038485b8c 100644 --- a/runtime/Go/antlr/v4/antlrdoc.go +++ b/runtime/Go/antlr/v4/antlrdoc.go @@ -49,7 +49,7 @@ Here is a general/recommended template for an ANTLR based recognizer in Go: . ├── parser │ ├── mygrammar.g4 - │ ├── antlr-4.12.0-complete.jar + │ ├── antlr-4.12.1-complete.jar │ ├── generate.go │ └── generate.sh ├── parsing - generated code goes here From 6e97bafc6510234ff0752be11bce4ad505a47515 Mon Sep 17 00:00:00 2001 From: "Jim.Idle" Date: Wed, 17 May 2023 16:46:17 +0800 Subject: [PATCH 5/6] docs: Write release notes and new instrcutions for using the Go target. Signed-off-by: Jim.Idle --- doc/go-changes.md | 179 ++++++++++++++++++++++++++++++++++++++++++++++ doc/go-target.md | 134 ++++++++++++++++++++++------------ 2 files changed, 268 insertions(+), 45 deletions(-) create mode 100644 doc/go-changes.md diff --git a/doc/go-changes.md b/doc/go-changes.md new file mode 100644 index 0000000000..a0aa936bd9 --- /dev/null +++ b/doc/go-changes.md @@ -0,0 +1,179 @@ +# Changes to the Go Runtime over time + +## v4.12.0 to v4.12.1 + +Strictly speaking, if ANTLR was a go only project following [SemVer](https://semver.org/) release v4.12.1 would be +at least a minor version change and arguably a bump to v5. However, we must follow the ANTLR conventions here or the +release numbers would quickly become confusing. I apologize for being unable to follow the Go release rules absolutely +to the letter. + +There are a lot of changes and improvements in this release, but only the change of repo holding the runtime code, +and possibly the removal of interfaces will cause any code changes. There are no breaking changes to the runtime +interfaces. + +ANTLR Go Maintainer: [Jim Idle](https://github.com/jimidle) - Email: [jimi@idle.ws](mailto:jimi@idle.ws) + +### Code Relocation + +For complicated reasons, including not breaking the builds of some users who use a monorepo and eschew modules, as well +as not making substantial changes to the internal test suite, the Go runtime code will continue to be maintained in +the main ANTLR4 repo `antlr/antlr4`. If you wish to contribute changes to the Go runtime code, please continue to submit +PRs to this main repo, against the `dev` branch. + +The code located in the main repo at about the depth of the Mariana Trench, means that the go tools cannot reconcile +the module correctly. After some debate, it was decided that we would create a dedicated release repo for the Go runtime +so that it will behave exactly as the Go tooling expects. This repo is auto-maintained and keeps both the dev and master +branches up to date. + +Henceforth, all future projects using the ANTLR Go runtime, should import as follows: + +```go +import ( + "github.com/antlr4-go/antlr/v4" + ) +``` + +And use the command: + +```shell +go get github.com/antlr4-go/antlr +``` + +To get the module - `go mod tidy` is probably the best way once imports have been changed. + +Please note that there is no longer any source code kept in the ANTLR repo under `github.com/antlr/antlr4/runtime/Go/antlr`. +If you are using the code without modules, then sync the code from the new release repo. + +### Documentation + +Prior to this release, the godocs were essentially unusable as the go doc code was essentially copied without +change, from teh Java runtime. The godocs are now properly formatted for Go and pkg.dev. + +Please feel free to raise an issue if you find any remaining mistakes. Or submit a PR (remember - not to the new repo). +It is expected that it might take a few iterations to get the docs 100% squeaky clean. + +### Removal of Unnecessary Interfaces + +The Go runtime was originally produced as almost a copy of the Java runtime but with go syntax. This meant that everything +had an interface. There is no need to use interfaces in Go if there is only ever going to be one implementation of +some struct and its methods. Interfaces cause an extra deference at runtime and are detrimental to performance if you +are trying to squeeze out every last nanosecond, which some users will be trying to do. + +This is 99% an internal refactoring of the runtime with no outside effects to the user. + +### Generated Recognizers Return *struct and not Interfaces + +The generated recognizer code generated an interface for the parsers and lexers. As they can only be implemented by the +generated code, the interfaces were removed. This is possibly the only place you may need to make a code change to +your driver code. + +If your code looked like this: + +```go +var lexer = parser.NewMySqlLexer(nil) +var p = parser.NewMySqlParser(nil) +``` + +Or this: + +```go +lexer := parser.NewMySqlLexer(nil) +p := parser.NewMySqlParser(nil) +``` + +Then no changes need to be made. However, fi you predeclared the parser and lexer variables with there type, such as like +this: + +```go +var lexer parser.MySqlLexer +var p parser.MySqlParser +// ... +lexer = parser.NewMySqlLexer(nil) +p = parser.NewMySqlParser(nil) +``` + +You will need to change your variable declarations to pointers (note the introduction of the `*` below. + +```go +var lexer *parser.MySqlLexer +var p *parser.MySqlParser +// ... +lexer = parser.NewMySqlLexer(nil) +p = parser.NewMySqlParser(nil) +``` + +This is the only user facing change that I can see. This change though has a very beneficial side effect in that you +no longer need to cast the interface into a struct so that you can access methods and data within it. Any code you +had that needed to do that, will be cleaner and faster. + +The performance improvement is worth the change and there was no tidy way for me to avoid it. + +### Parser Error Recovery Does Not Use Panic + +THe generated parser code was again essentially trying to be Java code in disguise. This meant that every parser rule +executed a `defer {}` and a `recover()`, even if there wer no outstanding parser errors. Parser errors were issued by +issuing a `panic()`! + +While some major work has been performed in the go compiler and runtime to make `defer {}` as fast as possible, +`recover()` is (relatively) slow as it is not meant to be used as a general error mechanism, but to recover from say +an internal library problem if that problem can be recovered to a known state. + +The generated code now stores a recognition error and a flag in the main parser struct and use `goto` to exit the +rule instead of a `panic()`. As might be imagined, this is significantly faster through the happy path. It is also +faster at generating errors. + +The ANTLR runtime tests do check error raising and recovery, but if you find any differences in the error handling +behavior of your parsers, please raise an issue. + +### Reduction in use of Pointers + +Certain internal structs, such as interval sets are small and immutable, but were being passed around as pointers +anyway. These have been change to use copies, and resulted in significant performance increases in some cases. +There is more work to come in this regard. + +### ATN Deserialization + +When the ATN and associated structures are deserialized for the first time, there was a bug that caused a needed +optimization to fail to be executed. This could have a significant performance effect on recognizers that were written +in a suboptimal way (as in poorly formed grammars). This is now fixed. + +### Prediction Context Caching was not Working + +This has a massive effect when reusing a parser for a second and subsequent run. The PredictionContextCache merely +used memory but did not speed up subsequent executions. This is now fixed, and you should see a big difference in +performance when reusing a parser. This single paragraph does not do this fix justice ;) + +### Cumulative Performance Improvements + +Though too numerous to mention, there are a lot of small performance improvements, that add up in accumulation. Everything +from improvements in collection performance to slightly better algorithms or specific non-generic algorithms. + +### Cumulative Memory Improvements + +The real improvements in memory usage, allocation and garbage collection are saved for the next major release. However, +if your grammar is well-formed and does not require almost infinite passes using ALL(*), then both memory and performance +will be improved with this release. + +### Bug Fixes + +Other small bug fixes have been addressed, such as potential panics in funcs that did not check input parameters. There +are a lot of bug fixes in this release that most people were probably not aware of. All known bugs are fixed at the +time of release preparation. + +### A Note on Poorly Constructed Grammars + +Though I have made some significant strides on improving the performance of poorly formed grammars, those that are +particularly bad will see much less of an incremental improvement compared to those that are fairly well-formed. + +This is deliberately so in this release as I felt that those people who have put in effort to optimize the form of their +grammar are looking for performance, where those that have grammars that parser in seconds, tens of seconds or even +minutes, are presumed to not care about performance. + +A particularly good (or bad) example is the MySQL grammar in the ANTLR grammar repository (apologies to the Author +if you read this note - this isn't an attack). Although I have improved its runtime performance +drastically in the Go runtime, it still takes about a minute to parse complex select statements. As it is constructed, +there are no magic answers. I will look in more detail at improvements for such parsers, such as not freeing any +memory until the parse is finished (improved 100x in experiments). + +The best advice I can give is to put some effort in to the actual grammar itself. well-formed grammars will potentially +see some huge improvements with this release. Badly formed grammars, not so much. \ No newline at end of file diff --git a/doc/go-target.md b/doc/go-target.md index 03cfe58adb..d229a1cd55 100644 --- a/doc/go-target.md +++ b/doc/go-target.md @@ -1,32 +1,54 @@ # ANTLR4 Language Target, Runtime for Go -### Removal of non v4 module +### Changes from ANTLR 4.12.0 -Prior to the release of the v4 tagged runtime, the source code for the Go runtime module existed at the root of -`runtime/Go/antlr`, which is the pre-v4 version of the code, and under `runtime/Go/antlr/v4`. If your project -was not using modules, you could merely sync to the latest hash in the master branch and use the code, +Please see [Changes in ANTLR Go runtimes](go-changes.md), but in summary: + - The Go runtime is now stored in the repo `antlr4-go/antlr` - change your import, remove the old location from + `go.mod` and use `go get github.com/antlr4-go/antlr` + - There are some new `@actions` for adding to the generated import statements and recognizer structure + - The recognizer rules are no longer called via an interface, for performance reasons + - Memory usage improvements + - Performance improvements + - Documentation in true Go format + - Git tags now work correctly with go tools -As of now, you can still use the code without modules, but you must use the code under the `/v4` directory and -not the code at the runtime root. This is for historic reasons as the code was originally written before modules were a -thing and the go runtime source was (and still is) part of the monorepo that is `antlr/antlr4`. +### Removal of non v4 code -We strongly advise you to use modules, and to use the /v4 version of the source code, though it is not required. See -below for more information. +Prior to the release of the v4 tagged runtime, the source code for the Go runtime module existed at +`runtime/Go/antlr`, which is the pre-v4 version of the code, and also under `runtime/Go/antlr/v4`. If your project +was not using modules, you could merely sync to the latest hash in the master branch and use the code. This has changed. + +As of the current release, the source code for the Go runtime module has been moved to its own repo in its own +GitHub organization. As of now, you can still use the code without modules, but you must use the code +in the repo at https://github.com/antlr4-go/antlr instead of the code in the main ANTLR repo. + +This is for historic reasons as the code was originally written before modules were a +thing, and the go runtime source was - and the maintainer's version still is - a part of the monorepo +that is `antlr/antlr4/...`. + +Note that I am unable to properly deprecate the go.mod in the non-V4 directory, for hte same reason that I +cannot use tag the v4 module at this depth in the source tree. + +We strongly advise you to use modules, though it is not required. See below for more information. + +ANTLR Go Maintainer: [Jim Idle](https://github.com/jimidle) - Email: [jimi@idle.ws](mailto:jimi@idle.ws) ### First steps #### 1. Install ANTLR4 -[The getting started guide](getting-started.md) should get you started. +See: [The getting started guide](getting-started.md). #### 2. Get the Go ANTLR runtime -Each target language for ANTLR has a runtime package for running parser generated by ANTLR4. -The runtime provides a common set of tools for using your parser. Note that if you have existing projects and have +Each target language for ANTLR has a runtime package for running a recognizer generated by ANTLR4. +The runtime provides a common set of tools for using your parser/lexer. Note that if you have existing projects and have yet to replace the `v1.x.x` modules with the `v4` modules, then you can skip ahead to the section *Upgrading to v4 from earlier versions* -The Go runtime uses modules and has a version path of `/v4` to stay in sync with the runtime versions of all the other runtimes. +The Go runtime uses modules and has a version path of `/v4` to stay in sync with the runtime versions of all the other +runtimes and the tool itself. + Setup is the same as any other module based project: ```bash @@ -37,26 +59,40 @@ $ go mod init mymodproject After which, you can use go get, to get the latest release version of the ANTLR v4 runtime using: ```bash -go get github.com/antlr/antlr4/runtime/Go/antlr/v4 +go get github.com/antlr4-go/antlr ``` -If your project is already using the v4 runtime, then you can upgrade to the latest release using the usual: +If your project was already using the v4 runtime from the main ANTLR repo, then you can upgrade to the latest release +by removing the `github.com/antlr/antlr4/runtime/Go/antlr/v4` reference in your module, and changing the associated +import in your project code. The following script may be useful in changing your imports: -```bash -go get -u github.com/antlr/antlr4/runtime/Go/antlr/v4 +```shell +find . -type f \ + -name '*.go' \ + -exec sed -i -e 's,github.com/antlr/antlr4/runtime/Go/antlr/v4,github.com/antlr4-go/antlr/v4,g' {} \; +``` +Note that the import package still imports with the final path as `antlr`, so only the import statement itself needs to +change. + +If you are already using the repo and import `github.com/antlr4-go/antlr/v4` then you can upgrade to the latest version +using the standard. + +```shell +go get -u get github.com/antlr4-go/antlr ``` + If you have not yet upgraded existing projects to the `/v4` module path, consult the section *Upgrading to v4 from earlier versions* -The ANTLR runtime has only one external dependency, and that is part of the go system itself: +The ANTLR runtime has only one external transient dependency, and that is part of the go system itself: ``` golang.org/x/exp ``` A complete list of releases can be found on [the release page](https://github.com/antlr/antlr4/releases). The Go -runtime will be tagged using standard Go tags, so release 4.12.0 will be tagged with `v4.12.0` and go get will pick -that up from the ANTLR repo. +runtime will be tagged using standard Go tags, so release 4.12.1 in the `antlr4-go/antlr` repo, will be tagged with +`v4.12.1` and go get will pick that up from the ANTLR repo. #### 3. Configuring `go generate` in your project @@ -72,17 +108,18 @@ place the ANTLR grammar files in their own package in your project structure. He ├── myproject ├── parser │ ├── mygrammar.g4 - │ ├── antlr-4.12.0-complete.jar - │ ├── error_listeners.go + │ ├── antlr-4.12.1-complete.jar │ ├── generate.go - │ ├── generate.sh + │ └── generate.sh + ├── parsing # Generated code goes here + │ └── error_listeners.go ├── go.mod ├── go.sum ├── main.go └── main_test.go ``` -Make sure that the package statement in your grammar file(s) reflects the go package they exist in. +Make sure that the package statement in your grammar file(s) reflects the go package the go code will be generated in. The `generate.go` file then looks like this: ```golang @@ -94,14 +131,13 @@ The `generate.go` file then looks like this: And the `generate.sh` file will look similar to this: ```shell - #!/bin/sh - alias antlr4='java -Xmx500M -cp "./antlr-4.12.0-complete.jar:$CLASSPATH" org.antlr.v4.Tool' - antlr4 -Dlanguage=Go -no-visitor -package parser *.g4 + alias antlr4='java -Xmx500M -cp "./antlr-4.12.1-complete.jar:$CLASSPATH" org.antlr.v4.Tool' + antlr4 -Dlanguage=Go -no-visitor -package parsing *.g4 ``` -From the command line at the root of your package “myproject” you can then simply issue the command: +From the command line at the root of your package - the location of the `go.mod` file - you can then simply issue the command: ```shell go generate ./... @@ -118,17 +154,17 @@ Suppose you're using a UNIX system and have set up an alias for the ANTLR4 tool To generate your go parser, you'll need to invoke: -```bash -antlr4 -Dlanguage=Go MyGrammar.g4 +```shell + antlr4 -Dlanguage=Go MyGrammar.g4 ``` For a full list of antlr4 tool options, please visit the [tool documentation page](tool-options.md). ### Upgrading to `/v4` from the default path -*NB: While switch to new module path would normally imply that the public interface for the runtime has changed, this is -not actually the case - you will not need to change your existing code to upgrade. The main point of the path change is so -that git tagging works with the ANTLR Go runtime.* +*NB: While switching to new module path would normally imply that the public interface for the runtime has changed, this is +not actually the case - you will not need to change your existing code to upgrade. The main point of the repo change is so +that git tagging works with the ANTLR Go runtime and the go tools.* Prior to release v4.11.0 the Go runtime shipped with a module but the module had no version path. This meant that the tags in the ANTLR repo did not work, as any tag above `v1` must refer to a matching module path. @@ -136,22 +172,24 @@ So the command `go get github.com/antlr/antlr4/runtime/Go/antlr` would just brin whatever was the `HEAD` of the master branch. While this *kind of* worked, it is obviously subject to problems and does not fit properly with the idiomatic ways of Go. -As of v4.11.0 the module path for the Go runtime is properly in sync with the repo tags. However, this means you need to -perform a few simple actions in order to upgrade to the `/v4` path. +As of v4.12.1 the runtime code exists in its own repo, `github.com/antlr4-go/antlr`, and is correctly tagged. +However, this means you need to perform a few simple actions in order to upgrade to the `/v4` path. - - Firstly, make sure that you are using an ANTLR tool jar with a version number of 4.11.0 or greater. - - Next you replace any mention of the old (default) path to ANTLR in your go source files. Don't worry that this will -modify your generated files as... + - Firstly, make sure that you are using an ANTLR tool jar with a version number of 4.12.1 or greater. + - Next you replace any mention of the old (default) path to ANTLR in your go source files. + - If using modules, remove any existing reference to the ANTLR Go runtime - Now regenerate your grammar files either manually or using `go generate ./...` (see above) + - Consider whether you can move to using modules in your project -A quick way to replace original module path references is to use this script from your module's base directory: +A quick way to replace the original module path references is to use this script from your module's base directory: ```shell find . -type f \ -name '*.go' \ - -exec sed -i -e 's,github.com/antlr/antlr4/runtime/Go/antlr,github.com/antlr/antlr4/runtime/Go/antlr/v4,g' {} \; + -exec sed -i -e 's,github.com/antlr/antlr4/runtime/Go/antlr,github.com/antlr4-go/antlr/v4,g' {} \; ``` -After performing the steps above, issuing: + +After performing the steps above, and you are using modules issuing: ```shell go mod tidy @@ -159,7 +197,7 @@ go mod tidy Should fix up your `go.mod` file to reference only the `v4` version of the ANTLR Go runtime: ```shell -require github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.11.0-xxxxxx-xxxxxxxxx +require github.com/antlr/antlr4/runtime/Go/antlr/v4 v4.12.1 ``` From this point on, your go mod commands will work correctly with the ANTLR repo and upgrades and downgrades will work @@ -170,7 +208,7 @@ as you expect. As will branch version such as @dev You can reference the go ANTLR runtime package like this: ```golang -import "github.com/antlr/antlr4/runtime/Go/antlr/v4" +import "github.com/antlr4-go/antlr/v4" ``` ### Complete example @@ -196,8 +234,8 @@ encountered `ParseTreeContext`'s. Assuming the generated parser code is in the ` package main import ( - "github.com/antlr/antlr4/runtime/Go/antlr/v4" - "./parser" + "github.com/antlr4-go/antlr/v4" + "./parser" // Note that with modules you may not be able to use a relative immport path "os" "fmt" ) @@ -226,6 +264,12 @@ func main() { } ``` +Fix up your `go.mod` file: + +```shell +go mod tidy +``` + This one expects the input to be passed on the command line: ``` From 166d4b530907193ff1aca2adbbc25c722f241803 Mon Sep 17 00:00:00 2001 From: "Jim.Idle" Date: Thu, 18 May 2023 17:35:37 +0800 Subject: [PATCH 6/6] doc: Correct the name of the new ANTLR go repo Signed-off-by: Jim.Idle --- runtime/Go/antlr/v4/antlrdoc.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/runtime/Go/antlr/v4/antlrdoc.go b/runtime/Go/antlr/v4/antlrdoc.go index f038485b8c..3bb4fd7c4e 100644 --- a/runtime/Go/antlr/v4/antlrdoc.go +++ b/runtime/Go/antlr/v4/antlrdoc.go @@ -19,7 +19,7 @@ of the repo, the `go get` and related commands could not properly resolve the lo This meant that the reference to the runtime in your `go.mod` file would refer to the correct source code, but would not list the release tag such as @4.12.0 - this was confusing, to say the least. -As of 4.12.1, the runtime is now available as a go module in its own repo, and can be imported as `github.com/antlr-go/antlr` +As of 4.12.1, the runtime is now available as a go module in its own repo, and can be imported as `github.com/antlr4-go/antlr` (the go get command should also be used with this path). See the main documentation for the ANTLR4 project for more information, which is available at [ANTLR docs]. The documentation for using the Go runtime is available at [Go runtime docs]. @@ -95,7 +95,7 @@ Use of this file is governed by the BSD 3-clause license, which can be found in [target languages]: https://github.com/antlr/antlr4/tree/master/runtime [LICENSE.txt]: https://github.com/antlr/antlr4/blob/master/LICENSE.txt [ANTLR docs]: https://github.com/antlr/antlr4/blob/master/doc/index.md -[new repo]: https://github.com/antlr-go/antlr +[new repo]: https://github.com/antlr4-go/antlr [Jim Idle]: https://github.com/jimidle [Go runtime docs]: https://github.com/antlr/antlr4/blob/master/doc/go-target.md */