JSON generation failures

[benbot]

I believe i'm on the development brach as plandex version says "development", but I got plandex through nixpkgs.

I'm trying to test out plandex by having it generate some tests for an elixir project i'm working on.

First I tried to plandex tell it to generate some missing tests, it froze during this process 2/3 times.
When it froze, the only way I knew that it erroed was the status message in plandex ps. It gave me no other indication that an error occurred, the generation just froze and appeared to keep spinning. I still don't know what the exact error was.

Afterwards, it seemed to complete its changes, but plandex apply always results in

🚨 Server error
  → Error unmarshalling build response
    → Json: cannot unmarshal string into Go struct field ChangesWithLineNums.changes of type []*shared.StreamedChangeWithLineNums

So after an hour and 30 cents, I'm giving up on plandex for now.

Maybe it doesn't like elixir very much, but it's hard for me to tell as these error messages give me nothing to work with and it's not clear how to view the intermediate prompts.

[danenania]

Hey @benbot, sorry that you ran into issues!

Were you using the default models? That error message (will be improved in the next release) means that the LLM couldn't generate valid json. This shouldn't ever be happening these days with OpenAI models so it's definitely a bit strange to see if you're using the defaults.

I'm not familiar with nix (the package was added by a contributor) so I'm not sure how to check whether you're getting the latest version there. Perhaps that could be a factor.

[notatestuser]

I've seen this one too here. I'm not using default models - it's on Claude 3.5 Sonnet for everything at the minute.

Edit: Using Version 1.1.1

[benbot]

@danenania I tried with both the default models and claude sonnet.

I took a peek into the nix package definition and it seems to be running version 1.1.1

https://github.com/NixOS/nixpkgs/blob/8d5bdaf3a45a6e42a23ff476ba478731752c7f95/pkgs/by-name/pl/plandex/package.nix#L8

[danenania]

@notatestuser @benbot Sonnet tends to have a lot of JSON generation issues unfortunately. I'd recommend trying the anthropic-claude-3.5-sonnet-gpt-4o model pack that uses sonnet for planning and gpt-4o for builds.

@benbot Ok, sounds like the nix version is fine then. If you start fresh with either the gpt-4o or anthropic-claude-3.5-sonnet-gpt-4o are you still seeing the issue?

[notatestuser]

it doesn't happen for me with anthropic-claude-3.5-sonnet-gpt-4o.