#iOS environment setup

Midscene connects to and controls iOS devices through WebDriverAgent (WDA), supporting UI automation across native apps, hybrid apps, and system interfaces.

This guide walks you through setting up the iOS environment required by Midscene Test, the JavaScript SDK, and Playground. You will first connect your device through WDA, and then use Playground to verify screen capture, touch input, and model calls.

#Connect the device through WDA

Midscene communicates with iOS physical devices and simulators through a running WebDriverAgent (WDA) service. If WDA is already running and reachable from your computer, skip directly to Verify the WDA service status.

#1. Build and start WebDriverAgent

Before you begin, prepare the following iOS development environment:

• macOS (required for iOS development and device debugging)
• Xcode and Xcode Command Line Tools
• An iOS Simulator or physical iOS device

Follow the official Appium documentation to build and start WebDriverAgent (WDA):

• Simulator setup: Run Prebuilt WDA
• Physical device setup: Real Device Configuration

#2. Configure a physical device and forward the port

iOS Simulators expose WDA directly on localhost and do not require port forwarding—if you are using a simulator or a remote WDA service, skip to the next step.

When using a physical iOS device:

1. Connect the device to your Mac via USB and tap Trust This Computer when prompted on the device.
2. Enable Developer Mode (Settings > Privacy & Security > Developer Mode) and UI Automation (Settings > Developer > Enable UI Automation) on the device.
3. After signing and starting WDA on the device, use iproxy (installable via brew install libimobiledevice) to forward the WDA port (8100 by default) from the device to your Mac:

iproxy 8100 8100 YOUR_DEVICE_ID

Replace YOUR_DEVICE_ID with your device's UDID, and keep both WDA and iproxy running. When connecting multiple devices or using custom ports, forward each device to a distinct local port and use that port in Playground or your project configuration.

#3. Verify the WDA service status

Open the WDA /status endpoint in your browser or terminal from the computer running Midscene:

• Default local URL: http://localhost:8100/status (if using a custom port, remote host, or gateway, use the actual service address)
• Expected JSON response:

{
  "value": {
    "build": {
      "version": "10.1.1",
      "time": "Sep 24 2025 18:56:41",
      "productBundleIdentifier": "com.facebook.WebDriverAgentRunner"
    },
    "os": {
      "testmanagerdVersion": 65535,
      "name": "iOS",
      "sdkVersion": "26.0",
      "version": "26.0"
    },
    "device": "iphone",
    "ios": {
      "ip": "10.91.115.63"
    },
    "message": "WebDriverAgent is ready to accept commands",
    "state": "success",
    "ready": true
  },
  "sessionId": "BCAD9603-F714-447C-A9E6-07D58267966B"
}

A JSON response containing "ready": true confirms that WDA is ready to accept commands. Note the WDA host and port (or gateway base URL) so you can use the same connection settings in Playground and your project.

#Verify in Playground

With Playground, you can verify screen capture, touch input, and model calls without writing code.

Before starting, prepare your model configuration using a model API key or a ChatGPT subscription.

#Launch Playground

Make sure Node.js (^20.19.0 || ^22.12.0 || >=24.0.0) is installed and both node -v and npx --version work in your terminal.

1. Start the iOS Playground CLI:

npx --yes @midscene/ios-playground

2. In the Playground window that opens, click the gear icon in the top-right corner, paste your model configuration, and save.
3. If your browser does not open automatically, visit the local Playground URL printed in the terminal, and keep the terminal process running.
4. If the connection panel appears, enter your WDA host and port (localhost and 8100 by default) and create the Agent (if WDA is already running on the default local port, Playground may connect automatically; when connecting through a gateway, enter the full WebDriverAgent base URL instead of host / port).
5. Confirm that the preview area displays your device or simulator screen. If no screen appears, check your WDA service and port forwarding first.

#Run an action and verify the result

On your iOS device or simulator, open Safari, complete any first-run prompts, and navigate to TodoMVC. Keep the device unlocked with Safari in the foreground.

1. In Playground, select aiAct, enter the following instruction, and run it:

Add a todo named "Try Midscene", then mark it as completed.

2. Once the completed todo item appears on the page, switch to aiAssert, enter the following assertion, and run it:

The list contains a todo named "Try Midscene", and it is marked as completed.

3. Confirm that the assertion passes and inspect the step-by-step screenshots in the results panel. At this point, WDA connectivity, screen capture, touch input, and model calls have all been verified together.

If execution fails, check the failed step and error message: for missing screenshots or unresponsive touch input, see the FAQ below; for model request errors, see Verify model connectivity.

#Clean up the example data

After verification, delete the "Try Midscene" item in TodoMVC. If you plan to run the Midscene Test iOS example next, switch to the All filter and make sure the list is completely empty (including completed items), as the example template checks for an empty list before running and does not overwrite existing items.

#See it in action

Prompt: Open Twitter and like the first post from @midscene_ai. (Full report)

#Use the device in tests or scripts

Once device connectivity and interactions are verified in Playground, you can integrate the device into your test or script project:

Choose your next step based on how you use Midscene:

• Midscene Test: Continue to Configure the model and runtime environment and run the example (if you have not initialized a project yet, start from Create a project and select iOS).
• JavaScript test runner: Continue with Integrate with Vitest.
• Standalone scripts: Follow Write standalone automation scripts and select the iOS example.
• Platform SDK reference: See the iOS JavaScript SDK for device selection, constructor options, platform-specific methods, and lifecycle management.

#FAQ

#Why can't I control my physical device even though WebDriverAgent is connected?

Check the following settings on your iOS device:

1. Developer Mode: Make sure Settings > Privacy & Security > Developer Mode is enabled and the device has been restarted.
2. UI Automation: Make sure Settings > Developer > Enable UI Automation is turned on.
3. Computer Trust: Reconnect the USB cable and make sure you have tapped Trust This Computer on the device prompt.

#What are the differences between iOS Simulators and physical devices?

#Model calls fail

Check the error message of the failed step and verify the model configuration in Playground settings or your project's .env file. Because device connectivity and model requests are independent, follow Verify model connectivity to troubleshoot API key, endpoint, or network issues. Note that CLI and project checks read your project's environment variables rather than browser-stored Playground settings.

#Advanced connections and debugging

#How do I customize the WebDriverAgent port and host?

See the iOS JavaScript SDK guide.

#How do I get a smoother live screen preview in Playground?

iOS Playground supports two screen preview modes:

• Polling mode (default): Captures screenshots sequentially via the WDA screenshot endpoint at roughly 5–10 fps.
• Native MJPEG stream (recommended): Connects directly to WDA's built-in MJPEG server for higher frame rates and lower latency.

To enable the native MJPEG stream on a physical device, forward WDA's MJPEG port (9100 by default) to your Mac (not required for Simulators):

iproxy 9100 9100 YOUR_DEVICE_ID

Playground automatically probes local port 9100 on startup: if available, the terminal logs MJPEG: streaming via native WDA MJPEG server; otherwise, it falls back to polling mode automatically.