How to Write iOS Automation Scripts for Short Video Publishing: Full Working Code

1. Six accounts, four hours a day

A local-business operator I know runs six short video accounts, posting one restaurant video each per day. At that size, the bottleneck is never editing. It is publishing.

The routine: footage is edited the night before, then each morning he works through the accounts one at a time — get the video onto the phone, open the app, find the publish entry, select the clip, write a title, add hashtags, tap publish, wait for the upload, then check it actually went out. Six minutes per video, roughly forty minutes for six accounts. With interruptions and the occasional stuck upload, the real figure is closer to four hours.

He wanted that to become one tap in the morning.

This article is the process of turning that route into a script. Every step comes with code, annotated with the official function signatures and return conventions — because the slow part of writing these scripts is not the logic, it is working out what each function actually returns.

One thing to state up front: every function name here comes from the official EC iOS documentation and was verified line by line. Writing iPhone automation scripts, the time really goes into return values, not logic.

The approach below works for any short video app — the differences are the bundle ID and the UI elements, and those two are what you change.

---

2. See the route before writing code

Break the flow down first and map each step to the capability it needs:

Step	What happens	What to use
1	Video gets into the album	The upload button in the toolbar, or the utils album calls
2	Open the target app	Right-click run in the central control, or the openApp HTTP endpoint
3	Wait for the UI to be genuinely ready	Node fetch plus a text query
4	Enter the publish screen	Node click, or coordinate click
5	Select the video from the album	Node click
6	Fill title and hashtags	The IME interface, imeApi
7	Tap publish and verify	Node click, then a second query or screenshot

Of the seven, steps 1, 3 and 7 are where things go wrong, and each is covered below.

Once this route runs, writing ios automation scripts does not make a single post dramatically faster. What changes with short video automation is that “all six went out” stops needing your attention.

---

3. Setup: decide which route you are on first

The same business logic calls a different event module depending on the route:

Route	Event module	System requirement	What you need
Proxy mode	agentEvent	iOS 13+	A signed IPA
USB HID	usbHidEvent	iOS 17+	One data cable
Bluetooth BLE	bleEvent	iOS 17+	An ESP32 board
OTG HID	otgEvent	iOS 17+	A board plus an adapter

That iOS 17 threshold matters. The documentation is direct about it: USB HID and Bluetooth BLE both require iOS 17 or above, and anything lower has to use proxy mode. The upside of all this is that no jailbreak is needed on any of the four routes — proxy mode needs a signature, HID needs a cable, and neither touches the operating system.

The good news is that all four modules use identical method names — clickPoint, swipeToPoint, doubleClickPoint and the rest. Write the business logic once and switch routes by changing the module name. Worth exploiting when designing a script: wrap your click operations in one place, and a route change touches that one place.

The script skeleton

Whichever route you take, the opening looks like this:

function main() {
    logd("Checking the automation environment...");
    if (!autoServiceStart(3)) {
        logw("Automation service failed to start, cannot run");
        exit();
        return;
    }

    // Set node fetch parameters once, at the top
    setFetchNodeParam({
        "labelFilter": "2",       // only nodes that have a label
        "visibleFilter": "2",     // only visible=true nodes
        "maxDepth": "20",         // fewer levels is faster, 1-500 advised
        "excludedAttributes": "visible,selected,enable,accessible"
    });

    // ... your logic

    logd("Script finished");
}

// Environment check: the helper pattern used in the official examples
function autoServiceStart(time) {
    for (let i = 0; i < time; i++) {
        if (isServiceOk()) {
            return true;
        }
        let started = startEnv();
        logd("Service start attempt " + (i + 1) + ": " + started);
        if (isServiceOk()) {
            return true;
        }
    }
    return isServiceOk();
}

main();

Two details are worth spelling out.

autoServiceStart is not a built-in — it is the helper pattern shown in the official examples: call startEnv() in a loop and check isServiceOk(), up to time attempts. That is far more reliable than a sleep(5000) and hoping.

Note this skeleton is the proxy mode version. On USB HID, the opening is the HID session instead:

let r = usbHidEvent.sessionStart(true);   // parameter: try enhanced compatibility, default true
if (!(r == null || r === "")) {
    logw("Failed to open HID session: " + r);
    return;
}
r = usbHidEvent.setScreenSize(1170, 2532);  // coordinate conversion depends on this

setScreenSize is mandatory on the HID route, and its arguments must match your mirroring or screenshot resolution. Change device, or rotate the screen, and you set it again — otherwise every coordinate is wrong. Bluetooth and OTG follow the same pattern with their own session calls.

The excludedAttributes field in setFetchNodeParam is the performance lever. The documentation says it “increases fetch speed” — with many devices, skipping a few unused attributes adds up quickly.

---

4. Step one: getting the video into the album

This is the first trap, because the two routes do completely different things.

Proxy mode / offline main program (iOS 15+)

Use the upload video button in the right-hand toolbar of the mirroring window, or right-click the small screen and choose images and videos. You can select a folder to upload in bulk, pushing the day’s clips across in one go.

HID routes (no proxy IPA installed)

The documentation is specific here, and it has to be done on the phone first:

1. In the Shortcuts app, create a shortcut named iOS USB插入视频或图片到相册 — that exact Chinese name, because it is the label the HID hotkey looks for
2. Go to Settings → Accessibility → Keyboards & Typing → Full Keyboard Access → Commands, and find that shortcut under Shortcuts
3. From then on, send the video using the HID hotkey

Skip this and nothing reaches the album on a HID route — plenty of people conclude the script is broken when the setup is what is missing.

The album interfaces

// Request album permission first (a dialog appears, tap allow once)
utils.requestPhotoAuthorization();

// To clear the album (note: a confirmation dialog appears, you must tap delete)
utils.deleteAllVideos();
utils.deleteAllPhotos();

Two behaviours you have to know:

These calls are asynchronous — do not rely on the return value. The documentation says plainly to ignore it, so that the click simulation does not get stuck.

Clearing cannot be undone. And a confirmation dialog appears when you call it, so you have to simulate the delete tap — which is exactly why it cannot be used unconditionally. Confirm the device and the path before any trial run.

---

5. Step two: open the app, and wait for the UI to be ready

“App opened” and “UI usable” are two different things. Tap the icon and the splash screen is still spinning; look for a publish button now and you will not find it.

Opening the app itself has two approaches:

Approach one: select the device in the central control and right-click to run the script. Good for manual triggering.

Approach two: use the HTTP API , for integration with other systems:

POST http://127.0.0.1:8019/openapi/openApp
{"deviceId": "device id", "bundleId": "target app bundle id"}

Port 8019 is the central control address, and the documentation provides Python, Node.js, cURL and C# examples.

After opening, do not use a fixed wait — query for the node text instead:

const BUNDLE_ID = "com.example.shortvideo";   // replace with your target app

releaseNode();   // release the previous node data
lockNode();      // lock the current UI nodes

let entry = labelMatch(".*发布.*").getOneNodeInfo(15000);  // wait up to 15s
if (!entry) {
    logw("Publish entry not found in 15s - still loading, or the UI changed");
    image.captureFullScreen();   // screenshot as evidence
    exit();
    return;
}

Three points from the documentation matter here.

Node queries use a chained call. labelMatch("regex").getOneNodeInfo(timeoutMs) — you do not pass a config object. The available filters are id, idMatch, label, labelMatch, name, nameMatch, type, typeMatch, value, valueMatch and xpath, plus attribute matches such as visible and enable.

The timeout inside getOneNodeInfo(timeout) is your element wait. Passing 15000 means “up to 15 seconds”, which beats sleep(15000) because it returns the moment the element appears.

Always releaseNode() then lockNode() before querying. The official example does this consistently: release the old data, then lock the new. Without releasing, you may read nodes from the previous screen.

---

6. Step three: finding the publish button — choosing between three methods

This is the most practically useful part of the article.

Method	What to use	When it fits	When it does not
Node lookup	labelMatch(...), id(...)	Elements have readable text or attributes	HID, Bluetooth and OTG routes have no node selector
Image matching	image.findImage, template matching	The entry point is an icon with no text	Redesign the UI and the template has to be redone
Visual location	The VLM step in a workflow	The node tree cannot reach it and something needs to “see” it	Requires a configured model

Prefer node lookup , because matching on text survives the interface moving around:

let btn = labelMatch("发布").getOneNodeInfo(5000);
if (btn) {
    btn.clickCenter();     // click the node centre
} else {
    logw("Publish button not found");
}

What a node object offers, per the documentation: id, xpath, label, name, type, value, bounds, index, depth, visible, enable, and the methods clickCenter, clickRandom, parent, child, allChildren, siblings, previousSiblings, nextSiblings.

When the node is found but the tap does not land, fall back to coordinates:

let nd = labelMatch("发布").getOneNodeInfo(5000);
if (nd) {
    // Use the node's own coordinates rather than measuring your own
    clickPoint(nd.bounds.centerX(), nd.bounds.centerY());
}

bounds.centerX() and centerY() are computed from the node itself, which makes them more reliable than hand-measured coordinates — no changes needed when the model or resolution changes.

If you genuinely need hard coordinates, remember they share the pixel coordinate system of the mirroring view and screenshots. After a resolution change or a rotation you must set the screen size again, or everything is offset.

---

7. Step four: filling title and hashtags

The action is simple, but there is a trap that catches almost everyone.

// Enter the title
if (imeApi.isOk()) {
    let r = imeApi.input("Today's pick: the noodle shop on the corner");
    logd("Input result: " + r);
}

// Hashtags are steadier through the clipboard
imeApi.setClipboard("#food #localfood #today");
imeApi.paste();

// Do not skip this: dismiss the keyboard
imeApi.dismiss();

Why is imeApi.dismiss() mandatory?

Because after typing, the keyboard occupies the lower half of the screen, and the publish button usually sits underneath it. Without dismissing, your tap lands on the keyboard and the button is never pressed. This failure raises no error — it just never publishes, which makes it hard to diagnose.

Watch the return value of imeApi.input(content): an empty string means the input failed, a non-empty value is the text that was entered. So testing != null is not enough.

On a HID route, input is a different module:

usbHidEvent.inputText("today's pick");     // goes through clipboard paste
usbHidEvent.typeText("Today's store");     // for keyboard typing; falls back to paste for non-English

One documentation note: if typing English into the phone keeps adding stray spaces, turn off Smart Punctuation in Settings → General → Keyboard.

---

8. Step five: tap publish, then confirm it really went out

Tapping publish is one line. Confirming it is the hard part.

let pub = labelMatch("发布").getOneNodeInfo(3000);
if (pub) {
    pub.clickCenter();
} else {
    logw("Publish button gone - already tapped, or the UI changed");
    exit();
    return;
}

// Wait for the result, not with a fixed sleep
releaseNode();
lockNode();
let ok = labelMatch(".*发布成功.*").getOneNodeInfo(20000);

if (!ok) {
    logw("No success message within 20s - needs a human look");
    image.captureFullScreen();   // capture evidence for later
} else {
    logd("Published successfully");
}

Why the check is not optional: a failed short video publish is often silent. No error dialog, just a draft that never went out, or an “uploading” state that quietly stops. Without the check you may discover days later that one account missed several posts.

For OCR, workflows offer ocr_screen_auto and ocr_screen_no_auto, with engines paddleOcrNcnnV5 (default), v5, v4 and ocrLite. In a script, call image.captureFullScreen() and run it through the OCR interface.

---

9. Putting it together: the full script

Combining everything above, ready to paste into the IDE and run after changing the bundle ID:

const BUNDLE_ID = "com.example.shortvideo";   // <- change to your target app
const TITLE = "Today's pick: the noodle shop on the corner";
const TOPICS = "#food #localfood";

function main() {
    logd("=== Short video publish started ===");

    if (!autoServiceStart(3)) {
        logw("Automation service failed to start");
        exit();
        return;
    }

    setFetchNodeParam({
        "labelFilter": "2",
        "visibleFilter": "2",
        "maxDepth": "30",
        "excludedAttributes": "visible,selected,enable,accessible"
    });

    // 1. Wait for the publish entry (doubles as the loading wait)
    if (!waitAndClick(".*发布.*", 15000)) {
        logw("Publish entry not found, aborting");
        shotForDebug();
        exit();
        return;
    }

    // 2. Select the newest video in the album
    if (!waitAndClick("相册", 8000)) {
        logw("Cannot open the album");
        exit();
        return;
    }
    clickPoint(195, 640);          // <- position of the first video, measure your own
    sleep(800);
    if (!waitAndClick("下一步|确定", 8000)) {
        logw("Video not selected");
        exit();
        return;
    }

    // 3. Fill in the title
    if (imeApi.isOk()) {
        imeApi.input(TITLE);
    }
    imeApi.setClipboard(TOPICS);
    imeApi.paste();
    imeApi.dismiss();              // dismiss the keyboard, mandatory

    // 4. Publish
    if (!waitAndClick("发布", 8000)) {
        logw("Publish button not reachable");
        exit();
        return;
    }

    // 5. Verify
    releaseNode();
    lockNode();
    if (labelMatch(".*发布成功.*").getOneNodeInfo(20000)) {
        logd("=== Published ===");
    } else {
        logw("No success message - needs a human look");
        image.captureFullScreen();
    }

    releaseNode();
}

// Wait for a node and click it; returns whether it succeeded
function waitAndClick(labelRegex, timeout) {
    releaseNode();
    lockNode();
    let nd = labelMatch(labelRegex).getOneNodeInfo(timeout);
    if (!nd) {
        return false;
    }
    clickPoint(nd.bounds.centerX(), nd.bounds.centerY());
    return true;
}

function autoServiceStart(time) {
    for (let i = 0; i < time; i++) {
        if (isServiceOk()) return true;
        startEnv();
        if (isServiceOk()) return true;
    }
    return isServiceOk();
}

function shotForDebug() {
    image.captureFullScreen();
}

main();

A few things worth noting in the code.

The waitAndClick wrapper. “Wait for an element, then click it” appears five times in this flow. Wrapping it leaves five lines in the main sequence, and changing timeouts or the click method touches one place.

Click through node coordinates (nd.bounds.centerX()) wherever possible. Only the video picker uses hard coordinates, because list items usually have no readable text. Measure that one for your own screen.

Every failure exits with a log. Publishing is a long chain, and any silent failure in the middle shows up later as something strange. Failing early is better.

---

10. Running it across many devices

Once the script works on one device, the next step is six accounts at once. Multi-account operations have always been the painful part: clicking through one account at a time, and by the fourth or fifth you start cutting corners. Three approaches, by effort:

Approach one: bulk run from the central control. Put the compiled iec file into the script directory at the bottom left, right-click, refresh, then right-click the script and run it — it executes across the selected devices. With many devices, group them in the group bar on the left first, then run per group. This is where ios cluster control does most of its work: the script is written once, and the central control handles the rollout.

One thing that trips people up: running scripts requires a USB device licence, while mirroring requires a USB mirroring licence, and they are not the same. Buying the wrong one is the most common mistake at this stage.

Approach two: dispatch over HTTP. The POST /openapi/openApp endpoint above, combined with the other action endpoints, lets an external system drive many devices. Useful for wiring publishing into your own back office or project tool.

Approach three: workflow plus scheduling. Turn the logic into a workflow and trigger it on a schedule. One hard constraint applies: a device runs only one task at a time, so parallelism comes from having more devices, not from concurrency on one.

The concurrency ceiling can be raised in settings (maximum devices per task), and takes effect after a restart.

On timing: do not schedule all six for the same minute. Publishing from phones with suspiciously uniform timing is itself a pattern. Spread it across minutes, or stagger by group.

---

11. Troubleshooting

Organised as symptom, then what to check first. All of these come from the documentation.

Symptom	Check first
Mouse clicks on the phone do nothing	Right-click the device screen or list → USB HID → rebuild USBHID session
Mirroring misbehaves after switching scene mode	Restart the phone and retry
English typed from the computer gains stray spaces	Turn off Smart Punctuation in Settings → General → Keyboard
The UI changed and nodes are not found	Screenshot to see which screen it stopped on, then match the new text
Several devices together slow down	Raise maximum devices per task in settings, then restart
Confirm how many devices are actually connected	Open 127.0.0.1:8020/devapi/devicenum in a browser
Memory keeps climbing	Adjust the central control’s ioscenter.vmoptions, or hit 127.0.0.1:8020/devapi/gc
Work out which step failed	Execution history plus real-time log, device by device

---

12. What not to hand to a script

Finally, the boundary.

A script handles publishing content that is already made, on time: moving assets, opening the app, selecting the video, filling fields, tapping publish, verifying the result. It does all six more consistently than a person, because it does not start cutting corners at the fourth account.

These stay with people: topic selection, copy, thumbnails, and comment replies. Their value lies precisely in being different every time; automating them kills the account.

And one more practical note: do not use scripts to inflate numbers. Automating the publish route solves efficiency, not volume. Using multiple accounts to manufacture engagement on the same content is a different thing entirely, and the accounts will suffer first.

Once the route runs, that operator’s morning becomes: read yesterday’s results, confirm all six went out, then spend the time on topics and copy — which is what actually affects view counts.

Running scripts requires a USB device licence and mirroring requires a USB mirroring licence; you need both. To see how many devices are scheduled together, read bulk short video publishing with Apple cluster control; for device grouping and scheduling detail, see running thirty iPhones.

About EasyClick: A phone automation AI-agent platform covering Android no-root, iOS no-jailbreak (proxy / Bluetooth HID / OTG HID) and HarmonyOS Next, offering script development, Apple cluster control, local central control & mirroring, and cloud control systems. → Explore all products