#Web / Puppeteer SDK

@midscene/web/puppeteer provides Agents bound to a Puppeteer page or Browser. Both inherit the base Agent’s AI methods and accept Web configuration options.

#Imports and API overview

import {
  PuppeteerPageAgent,
  PuppeteerAgent,
  PuppeteerBrowserAgent,
} from '@midscene/web/puppeteer';

In the signatures below, Page and Browser come from puppeteer. Option types come from @midscene/web/puppeteer.

#PuppeteerPageAgent

new PuppeteerPageAgent(page: Page, options?: PuppeteerPageAgentOpt)

• page: required, an existing Puppeteer page. Throws when omitted.
• options: optional, accepts shared Agent options and Web options.

const agent = new PuppeteerPageAgent(page, {
  forceSameTabNavigation: false,
});
await agent.aiAct('Click the login button');

The Agent stays bound to the supplied page. By default, forceSameTabNavigation: true intercepts window.open and links with target="_blank" to open them in the current page. Set it to false to preserve new tabs; the Agent will not automatically switch to them.

#PuppeteerBrowserAgent

new PuppeteerBrowserAgent(
  browser: Browser,
  initialPage: Page,
  options?: PuppeteerBrowserAgentOpt,
)

• browser: required, the Browser used to listen for, list, and create pages.
• initialPage: required, the initial active page. Use a page from this browser.
• options: optional, accepts shared Agent options, Web options, and the dedicated options below.

Throws when browser or initialPage is omitted. Browser Agent does not accept forceSameTabNavigation; even passing false through JavaScript throws an error.

#PuppeteerBrowserAgent.create()

PuppeteerBrowserAgent.create(
  browser: Browser,
  options?: PuppeteerBrowserAgentCreateOpt,
): Promise<PuppeteerBrowserAgent>

Creates an Agent asynchronously. The initial page is selected in this order: options.initialPage → (await browser.pages())[0] → await browser.newPage().

const agent = await PuppeteerBrowserAgent.create(browser, {
  autoFollowNewPage: true,
});

#Dedicated options

#Page management APIs

These properties and methods belong only to PuppeteerBrowserAgent.

#activePage

agent.activePage: Page

Read-only property returning the Puppeteer page used by current AI operations. Use setActivePage() to switch pages.

#pages()

agent.pages(): Promise<Page[]>

Returns the pages in the bound browser without changing the active page.

#newPage()

agent.newPage(): Promise<Page>

Creates a page in the bound browser’s default BrowserContext, makes it active, and returns it.

#setActivePage()

agent.setActivePage(page: Page): Promise<void>

Makes the supplied page active and attempts to bring it to the foreground. Throws for an invalid or closed page. Supply a page from the bound browser.

#waitForNewPage()

agent.waitForNewPage(
  action?: () => Promise<unknown> | unknown,
  options?: { timeout?: number },
): Promise<Page>

Registers a browser targetcreated listener (for page targets only) before running the optional action, then returns the new page it observes. options.timeout overrides newPageTimeout for this call, in milliseconds. Throws on timeout or if action fails.

This method does not itself switch the active page. With autoFollowNewPage: true, the independent page listener still switches automatically. Keep that option false to control switching explicitly:

const agent = await PuppeteerBrowserAgent.create(browser, {
  autoFollowNewPage: false,
});
const newPage = await agent.waitForNewPage(
  () => agent.aiTap('Open details'),
  { timeout: 10000 },
);
await agent.setActivePage(newPage);
await agent.aiAssert('The details page is open');

#Configuration options

Both Agents also accept shared Agent options, including model, report, and cache settings.

inputStrategy: 'bulk' requires keyboardTypeDelay to be omitted or set to 0. Use 'sequential' for delayed input. To wait for application readiness, use aiWaitFor() or Puppeteer’s own waiting APIs.

#Lifecycle

agent.destroy(): Promise<void>

Both Agents support destroy(): it waits for associated operations, cleans up Agent-owned resources, calls the internal Device’s destroy(), and finishes report processing. Browser Agent also removes its targetcreated listener for automatic page following.

The Puppeteer Device’s destroy() does not close the supplied Page, BrowserContext, or Browser, or explicitly disconnect a remote browser. The caller manages these Puppeteer resources. Pages and browsers remain usable after the Agent is destroyed.

try {
  await agent.aiAct('Submit the form');
} finally {
  await agent.destroy();
  // The caller closes page, context, or browser according to resource ownership.
}

#Browser support and limitations

Chrome / Chromium is recommended. Midscene’s Puppeteer integration depends on CDP for touch gestures and some interactions. Support for other browsers or protocols in Puppeteer itself does not imply support in Midscene.

#Remote browsers

Agents accept existing Puppeteer pages, with the same construction API as local browsers. For example, after connecting over CDP:

import puppeteer from 'puppeteer';
import { PuppeteerPageAgent } from '@midscene/web/puppeteer';

const browser = await puppeteer.connect({ browserWSEndpoint: cdpWsUrl });
const page = (await browser.pages())[0] ?? await browser.newPage();
const agent = new PuppeteerPageAgent(page);

The caller manages the connection and authentication. Use browser.disconnect() to disconnect Puppeteer from a remote browser while leaving the browser process running; browser.close() closes the browser.

#Getting a CDP WebSocket URL

You can get a CDP WebSocket URL from various sources, for example:

• BrowserBase: Sign up at https://browserbase.com and get your CDP URL
• Browserless: Use https://browserless.io or run your own instance
• Local Chrome: Run Chrome with --remote-debugging-port=9222 and use ws://localhost:9222/devtools/browser/...
• Docker: Run Chrome in a Docker container with debugging port exposed

#FAQ

#Puppeteer installation is slow or gets stuck

Puppeteer downloads a browser binary during postinstall. If that download is slow or blocked, installation can appear to hang.

# Skip browser download first so dependency installation can finish
PUPPETEER_SKIP_DOWNLOAD=true pnpm install

# Then download Chrome from a mirror
# This example uses https://registry.npmmirror.com
# Different mirror providers may use different base-url rules
pnpm exec puppeteer browsers install chrome --base-url="https://registry.npmmirror.com/-/binary/chrome-for-testing"

#Cannot click the dropdown

If the dropdown options do not appear in the report screenshot after you click the field, the page is usually using a native select element. In that case, the expanded dropdown is rendered by the operating system instead of inside the webpage.

Midscene enables forceChromeSelectRendering by default to force Chrome to render the dropdown; set it to false to opt out. For the detailed explanation and how to confirm the issue, see Playwright FAQ — Cannot click the dropdown.

#The webpage continues to flash when running in headed mode

This is caused by a mismatch between the viewport's deviceScaleFactor and the system pixel ratio. Set deviceScaleFactor to 0 to automatically use the device pixel ratio:

await page.setViewport({
  width: 1280,
  height: 720,
  deviceScaleFactor: 0,
});

For more details, see Playwright FAQ — The webpage continues to flash.

#Customize the network timeout

Use waitForNavigationTimeout and waitForNetworkIdleTimeout to adjust or disable the built-in checks. See Configuration options.