support scrolling

[seanmcguire12]

why

• needed for CU/vision

what changed

• handle scrolling inside performPlaywrightMethod

example usage

await stagehand.page.goto("https://aigrant.com/");
await stagehand.page.act({
  action: "scroll 50% down the page.",
  slowDomBasedAct: false,
});

[changeset-bot]

🦋 Changeset detected

Latest commit: c63bb73

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@browserbasehq/stagehand	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[sameelarif]

tested and confirmed scrolling with act works on my end! did run into some issues though:

1. when slowDomBasedAct is true, the scrolling works but is then ignored by verifyActCompletion, which scrolls down and back to the top of the page.
2. when slowDomBasedAct is false, the underlying observe does not always return a valid _performPlaywrightMethod method. For example, it returned .evaluate instead of scroll/scrollTo/mouse.wheel. This may be an underlying issue with our schema for the observe LLM call response, but it's causing scrolling to fail here.

[seanmcguire12]

tested and confirmed scrolling with act works on my end! did run into some issues though:
1. when slowDomBasedAct is true, the scrolling works but is then ignored by verifyActCompletion, which scrolls down and back to the top of the page.
2. when slowDomBasedAct is false, the underlying observe does not always return a valid _performPlaywrightMethod method. For example, it returned .evaluate instead of scroll/scrollTo/mouse.wheel. This may be an underlying issue with our schema for the observe LLM call response, but it's causing scrolling to fail here.

yeah its not supposed to work if slowDomBasedAct is true. should prob add docs on this. nice find on the _performPlaywrightMethod issue. if only #569 was in, then i could easily debug 👀 @kamath

[miguelg719]

I don't see the llm calling scroll by pixel. If anything by % or maybe like we had in chunking something like 'scroll down'. It would be interesting to add some sort of feedback loop like 'successfully scrolled all the way to the bottom' so the agent doesn't get stuck. It seems performPlaywrightMethod doesn't return anything?