• English

      • See interfaces like a human
      E2E testing in natural language

      Midscene.js provides a vision-based GUI Agent and a natural-language E2E testing framework for web, mobile, and desktop apps.

      🔥 Latest news Starting with v1.16, Midscene supports using Jev 1.14’s visual capabilities to drive E2E testing. See what’s new

      15k+Github Stars
      MITFully open source
      #2No.2 in Github trending

      Describe steps and expectations in natural language

      Describe what a user wants to do and what they should see. Express the same intent in YAML or in your existing JavaScript tests.

      YAML · @midscene/test
      Vitest + JavaScript
      beforeEach:
  - gotoUrl: https://your-shop.example

cases:
  - name: "Find headphones under $100"
    steps:
      - aiAct: "Search for headphones and filter the results to prices under $100"
      - aiWaitFor: "The filtered search results are visible"
      - aiAssert: "There are products in the search results, and each costs less than $100"

      A GUI Agent built for testing

      Midscene’s GUI Agent brings visual actions and assertions to your existing JavaScript tests. We recommend Playwright Test for web apps and Vitest for mobile and desktop apps.

      See, act, and verify like a human

      E2E testing verifies complete workflows from the user’s perspective, so Midscene tests the way a user does.

      1. ObserveRead screenshots, not the DOM
      2. ActFind the target, then tap, type, or scroll
      3. VerifyAssert on what users actually see
      See real-world examples

      One API, across platforms

      Visual localization does not depend on the UI rendering stack, so the same API works on web, mobile, desktop, and canvas-based interfaces.

      Web@midscene/webAndroid@midscene/androidiOS@midscene/ios
      Same APIawait agent.aiAct('Search for headphones')
      HarmonyOS@midscene/harmonyDesktop@midscene/computerCustom interfaces@midscene/core

      Model options and cost

      Multiple model options

      Choose from Doubao, DeepSeek, Qwen, GPT, Gemini, Kimi, and more to fit your tasks and budget.

      View model configuration

      Low model API cost

      In this AppControlBench run, Doubao Seed 2.1 Turbo passed 58 of 60 tasks at a total model API cost of $0.59.

      View benchmark report

      Benchmarks

      A new E2E testing framework designed for natural language

      Midscene Test (@midscene/test) is a natural-language E2E testing framework. Describe actions and expectations in YAML, and keep logic that needs precise control in TypeScript. Get started with Midscene Test

      AI and business operations in one YAML file

      Each YAML step calls a Node: an AI, platform, or custom business operation. Compose them into a test case.

      cases/refund.yaml
      beforeEach:  - order.prepare: { status: paid }Business  - gotoUrl: ${shopUrl}/orders/latestBusinesscases:  - name: A paid order supports a full refund    steps:      - aiAct: Apply for a full refund and submitAI      - aiAssert: The page shows that the refund was submittedAI      - order.checkRefund: { status: refunding }BusinessafterEach:  - clearCookies: {}Business

      Test cases maintained by people and AI

      Wrap reusable business logic in custom Nodes. Midscene Test generates a Markdown Node Spec listing every Node and its inputs, so QA engineers and AI agents can write cases without reading TypeScript. Learn how to develop custom Nodes

      1. nodes/order.ts
        defineNode({
  name: 'order.prepare',
  description: "Create a test order",
  inputSchema: z.strictObject({
    status: z.literal('paid'),
  }),
  async execute({ input }) {
    await orders.create(input);
  },
});
      2. midscene-node-spec.md
        ### order.prepare
Create a test order
- status: "paid"
      3. case.yaml
        steps:
  - order.prepare:
      status: paid

      Test framework capabilities

      Midscene Test also includes the capabilities expected of a test framework: lifecycle hooks and cleanup, retries and timeouts, concurrent execution with isolation, and step-level reports.

      Lifecycle hooks and cleanup

      Prepare and clean up resources at the project, file, and case level with setup, fileSetup, beforeAll, beforeEach, and their counterparts. Cleanup still runs after failures, timeouts, or Ctrl+C.

      Project lifecycle

      Retries and timeouts

      Retry failed cases; each attempt reruns beforeEach, steps, and afterEach. Set step timeouts, keep going after a failed step with continue-on-error, and stop early with bail.

      Handling failures

      Concurrency and isolation

      Run YAML files concurrently with maxConcurrentFiles. fileSetup gives each file its own page, Agent, or device, so concurrent files do not share state.

      Concurrency and isolation

      Step-level reports

      The HTML report records screenshots, inputs, outputs, errors, and Agent replay for every step. A #runner-step=last-error link opens the latest failed step from CI notifications or PR comments, and summary.json and result.md are written for scripts and AI agents.

      Output and reports

      Who is using Midscene

      ByteDanceAlibabaCtripXiaomiRome OSVolcengineDouyinTikTokLarkSoda MusicDoubao

      Next steps

      To let AI coding agents operate your UI, explore Midscene Skills.