Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python parsing Bug: Issues with codebases like dspy, textgrad , litellm etc #28

Closed
JayGhiya opened this issue Jul 9, 2024 · 11 comments
Closed

Python parsing Bug: Issues with codebases like dspy, textgrad , litellm etc #28

JayGhiya opened this issue Jul 9, 2024 · 11 comments

Comments

@JayGhiya
Copy link

JayGhiya commented Jul 9, 2024

Example exception that happens

[SCANNER] o.a.scanner.analyser.PythonAnalyser analysis file: /Users/jayghiya/Documents/unoplat/textgrad/textgrad/./tasks/gpqa.py
enterSimple_stmt ->Import_stmtContext
enterSimple_stmt ->Import_stmtContext
enterSimple_stmt ->Import_stmtContext
enterSimple_stmt ->From_stmtContext
enterSimple_stmt ->From_stmtContext
enterSimple_stmt ->From_stmtContext
enterSimple_stmt ->From_stmtContext
enterSimple_stmt ->Return_stmtContext
enterSimple_stmt ->From_stmtContext
enterSimple_stmt ->Assert_stmtContext
enterSimple_stmt ->Return_stmtContext
enterSimple_stmt ->Return_stmtContext
enterSimple_stmt ->Return_stmtContext
enterSimple_stmt ->Return_stmtContext
enterSimple_stmt ->Return_stmtContext
enterSimple_stmt ->Return_stmtContext
enterSimple_stmt ->Return_stmtContext
Exception in thread "main" java.lang.NullPointerException
        at chapi.ast.pythonast.PythonAstBaseListener.buildTestContext(PythonAstBaseListener.kt:126)
        at chapi.ast.pythonast.PythonAstBaseListener.buildAssignPart(PythonAstBaseListener.kt:115)
        at chapi.ast.pythonast.PythonAstBaseListener.buildExprStmt(PythonAstBaseListener.kt:105)
        at chapi.ast.pythonast.PythonFullIdentListener.enterSimple_stmt(PythonFullIdentListener.kt:116)
        at chapi.ast.antlr.PythonParser$Simple_stmtContext.enterRule(PythonParser.java:2133)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.enterRule(ParseTreeWalker.java:50)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:33)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at org.antlr.v4.runtime.tree.ParseTreeWalker.walk(ParseTreeWalker.java:36)
        at chapi.ast.pythonast.PythonAnalyser.analysis(PythonAnalyser.kt:15)
        at org.archguard.scanner.analyser.PythonAnalyser.analysisByFile(PythonAnalyser.kt:30)
        at org.archguard.scanner.analyser.PythonAnalyser.access$analysisByFile(PythonAnalyser.kt:11)
        at org.archguard.scanner.analyser.PythonAnalyser$analyse$1$2$1.invokeSuspend(PythonAnalyser.kt:21)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at org.archguard.scanner.analyser.PythonAnalyser.analyse(PythonAnalyser.kt:17)
        at org.archguard.scanner.core.sourcecode.LanguageSourceCodeAnalyser$DefaultImpls.analyse(LanguageSourceCodeAnalyser.kt:29)
        at org.archguard.scanner.analyser.PythonAnalyser.analyse(PythonAnalyser.kt:11)
        at org.archguard.scanner.ctl.loader.SourceCodeWorker$run$1.invokeSuspend(AnalyserDispatcher.kt:79)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at org.archguard.scanner.ctl.loader.SourceCodeWorker.run(AnalyserDispatcher.kt:76)
        at org.archguard.scanner.ctl.loader.AnalyserDispatcher.dispatch(AnalyserDispatcher.kt:39)
        at org.archguard.scanner.ctl.Runner.run(Runner.kt:107)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:198)
        at com.github.ajalt.clikt.parsers.Parser.parse(Parser.kt:18)
        at com.github.ajalt.clikt.core.CliktCommand.parse(CliktCommand.kt:400)
        at com.github.ajalt.clikt.core.CliktCommand.parse$default(CliktCommand.kt:397)
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:415)
        at com.github.ajalt.clikt.core.CliktCommand.main(CliktCommand.kt:440)
        at org.archguard.scanner.ctl.RunnerKt.main(Runner.kt:111)
@JayGhiya JayGhiya mentioned this issue Jul 9, 2024
Closed
@JayGhiya
Copy link
Author

JayGhiya commented Jul 9, 2024

@phodal - we are doing this effort inspired from you and your work here - https://github.com/unoplat/unoplat-code-confluence .

@phodal
Copy link
Owner

phodal commented Jul 9, 2024

Can you paste the failed code ?

@phodal
Copy link
Owner

phodal commented Jul 9, 2024

But in other way, In you use Python for your project, I suggest you can use TreeSitter to parse code.

phodal added a commit that referenced this issue Jul 9, 2024
@phodal
fix(parser): handle null contexts in PythonAstBaseListener.kt #28
7c3f1db 
Avoid crashes due to null contexts by using nullable types and safe calls in the PythonAstBaseListener. This ensures the parser can handle cases where child contexts may not be present.
@JayGhiya
Copy link
Author

JayGhiya commented Jul 9, 2024 •
edited
Loading

Thanks sure will do first thing in morning it's late here in India. Also the codebases I tested with were dspy litellm textgrad etc using arcguard cli as we use that as a base for the project - unoplat code confluence

@JayGhiya
Copy link
Author

JayGhiya commented Jul 9, 2024 •
edited
Loading

this was the file where it failed and for which the exception is shared- https://github.com/zou-group/textgrad/blob/main/textgrad/tasks/gpqa.py . @phodal

@JayGhiya
Copy link
Author

@phodal -
Also two more files where it failed - https://github.com/BerriAI/litellm/blob/main/litellm/tests/test_alerting.py and https://github.com/stanfordnlp/dspy/blob/main/dspy/predict/aggregation.py. Do not know if it is the same issue. trying to give you all the data points

@JayGhiya
Copy link
Author

yea you are right we could use tree sitter for python but we would want to be based on chapi/arcguard to keep things simple and work closely with arcguard.

phodal added a commit that referenced this issue Jul 10, 2024
@phodal
chore: add test for #28
a79e726
@phodal phodal closed this as completed in 06d48f0 Jul 10, 2024
@phodal
Copy link
Owner

phodal commented Jul 10, 2024

already publish new version 2.3.6 to maven central, it's still need 15~30mins maybe for maven to publish.

@JayGhiya
Copy link
Author

JayGhiya commented Jul 10, 2024 •
edited
Loading

So is the new version for arcguard ? @phodal . As we also like most people just use arcguard cli to parse codebases into chapi - Thankyou @phodal ! Also if you get time we would to plug https://github.com/unoplat/unoplat-code-confluence it in to your agents if the current work/roadmap excites/aligns with you.

@phodal
Copy link
Owner

phodal commented Jul 10, 2024

OK, it will do it later.

phodal added a commit to archguard/archguard that referenced this issue Jul 10, 2024
@phodal
chore(dependencies): bump chapi version to 2.3.6 for phodal/chapi#28
c9c155e 
Update the `chapi` library version from 2.3.5 to 2.3.6 in the `gradle/libs.versions.toml` file to incorporate the latest changes and potentially bug fixes.
@phodal
Copy link
Owner

phodal commented Jul 10, 2024

@JayGhiya Already publish new version to GitHub, and for Maven Central still need 15~30mins to waiting for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@phodal @JayGhiya