Why not a cloud device farm
Cloud device farms are genuinely good at what they are for: layout sweeps, OS-version compatibility, a thousand devices you will never own. We use simulators heavily for that class of work too. The problem is that everything interesting about our app happens at the layer cloud farms abstract away.
boopr keeps its keys in hardware (Secure Enclave on iPhone, StrongBox where Android offers it), gates them behind the system biometric prompt, blanks sensitive screens in captures, and detects OS screenshot events. None of that can be faked on a simulator, and most of it cannot be exercised on a rented cloud device either: you cannot un-enroll Face ID on someone else's phone, you cannot power-cycle their USB port, and a secure screen that renders black in a screencap looks identical to a crash unless you know the hardware. Add that our test fleet talks to a staging environment we wipe at will, and the case for owning the phones made itself. The fleet today spans the app's supported floor to current flagships: old and new Pixels, a couple of budget Samsung Galaxies for the One UI divergences, and a bench of iPhones.
The backbone is a smart USB hub
The unglamorous hero of the lab is a Cambrionix sixteen-port managed hub. It looks like a charging brick; the difference is that it has an API. Every port reports live current, voltage, and charging state, and, critically, every port can be switched off and on with an HTTP call. That one feature restructures the whole lab, because power-cycling the port is the recovery primitive. A phone that has stopped responding to every software escalation gets its USB cut and restored, which is the desk-scale equivalent of turning it off and on again, and it works without anyone standing up.
The hub also reads battery health and temperature per device, which matters more than it sounds: the number one hardware risk of running phones plugged in around the clock is battery swelling. The fleet runs with charge limits (the modern devices cap at 80 percent through their own OS settings) and the health numbers get watched, because a swollen battery in a rack of sixteen is not a theoretical failure mode.
Two lessons the hub taught us. First, read the manual about which port the host cable goes in; on ours only one specific rear port carries host data, and the wrong choice enumerates nothing while looking perfectly plugged in. Second, and more transferable: part of the hub's API sits behind a subscription license, and when we automated recovery we made a rule that automation may only ever depend on the license-free surface. Anything you need at 2am to un-wedge a fleet should not have an expiry date.
Driving real iPhones
The iOS stack is Appium with the XCUITest driver, which really means WebDriverAgent: an Apple-signed test runner that lives on each phone and exposes the UI over HTTP. Getting it onto a physical device is where the friction lives, and it is all identity friction. Each phone needs two separate pairings (Xcode's trust and the classic libimobiledevice pairing, each with its own on-screen prompt), a WebDriverAgent build signed for that specific device, and then, the one everyone forgets, a manual trust tap for the developer certificate in Settings. Miss that last tap and everything builds cleanly and then dies at session start with an unhelpful socket error, because iOS silently refuses to launch the untrusted runner.
Two more field notes. Every device needs its own automation server instance and its own WebDriverAgent port, or concurrent sessions collide. And coordinates are in logical points, not pixels, so on a 3x phone everything you measure in a screenshot gets divided by three before you tap it.
Version support has a hole worth knowing about: recent Xcode releases bundle the legacy developer disk images only up to a point, and the newer mechanism starts at iOS 17, which leaves a band of late iOS 16 versions where a perfectly healthy phone simply cannot be automated. We lost our intended floor device, an iPhone X, to exactly that gap and replaced the plan with a floor device inside the supported range. If you are picking cheap old phones for a farm, check the automation support window before you buy, not after.
Face ID is a wall, the passcode is a door
On a simulator you can fake a successful Face ID match with a menu click. On real hardware, a Face ID prompt aimed at a phone in a rack is an absolute wall: the sheet belongs to the system, not the app, the automation agent cannot satisfy it, and the failure wedges the session. Our fixture rule is therefore simple: no biometric enrollment on farm devices, ever, but always a device passcode. With biometrics unenrolled, the system falls back to a typeable passcode sheet, and a typeable sheet is a drivable sheet, even when it is system UI that only accepts coordinate taps on its keypad.
The part we were happiest about: this works with the app's hardware-backed storage, not around it. We never ship a build with the security gates removed for testing; the fixtures satisfy the real gate with a real credential, the same way a user without Face ID would.
The screenshot problem
Here is a puzzle specific to us. boopr tells you when a friend screenshots what you shared, which means QA has to take real screenshots and assert the detection fires. On iOS, a real screenshot means the physical volume-and-side-button chord. The automation stack cannot press two hardware buttons at once, and, the sneaky part, a capture taken through the tooling does not fire the OS screenshot notification, because the OS correctly considers it an external capture. So the one gesture our flagship feature depends on looked undrivable.
The answer was sitting in the accessibility settings: AssistiveTouch, the floating on-screen button iOS offers for people who cannot press hardware buttons. Set its single-tap custom action to Screenshot, and one tap produces a genuine OS screenshot, indistinguishable from the chord, notification and all. With that, the full loop runs with no human in the room: one phone posts, a second phone screenshots it via AssistiveTouch, and the first phone's notification is asserted, two real devices, real detection, end to end. It was the first test in the suite that was physically impossible to fake on a simulator, and it is the reason the farm exists.
Android: the pad that ignores you, the dump that lies
Android driving is adb plus uiautomator, and the app's test IDs surface as resource IDs, so most driving is selector-based and pleasant. The traps are elsewhere.
- Animations are not cosmetic. A continuously looping animation in the app can starve uiautomator's idle wait until the device is rebooted. All farm devices run with the three animation scales set to zero; the day one device did not, we learned this the slow way.
- The credential pad that ignores input. On recent stock Android, the system credential sheet is a Compose view that silently ignores injected text and key events; focus stays on the app and digits never land. You have to read the on-screen pad out of the UI tree and tap each digit by coordinate. Samsung's One UI, meanwhile, renders a completely different sheet with a text field that accepts injected text just fine. Same OS, same dialog, two drivers.
- Type once, verify by screenshot. Credential entry must never be retried in a loop (failed attempts stack toward a lockout), and the accessibility tree lies for a few seconds around unlock transitions, so the screenshot is the only honest witness.
- The dump that lies. The nastiest bug in the lab: when a UI dump fails, it writes nothing, and a driver that then reads the dump file gets the previous screen. Ours once kept "seeing" a credential pad long after unlock had succeeded and blind-tapped the live app. The fix is an invariant, not a patch: delete the dump file before dumping, and treat a failed dump as an error, never as data.
- Secure screens render black. A capture of the system credential sheet, or of any screen the OS protects, comes back black. That is the security feature working. Every screencap-based assertion has to know which screens are supposed to be black, or it will file crash reports against correct behavior.
Patience as a recovery strategy
The subtlest fleet lesson: powered is not the same as drivable. The hub will happily keep charging a phone whose USB data interface has gone away, so "the port shows a device" means nothing; the signal that matters is whether the port reports a USB descriptor. And when a device disappears and returns, the adb server on the host does not re-notice it by itself; after every port power-cycle, you restart the adb server too, a two-line detail that once cost us ten minutes of misdiagnosis at a time we could not spare.
Best of all: after a reboot, one of our budget Samsungs takes fifteen to twenty minutes to re-present USB, far past the point where any reasonable retry loop concludes the device needs hands. It then comes back entirely on its own, authorized and unlocked. The fix was not technical, it was scheduling: reboot-class tests run last in a device's queue, and the recovery playbook for a rebooted Samsung is a long passive wait, not an escalation. Sometimes the correct automation is patience.
One more desk-scale distributed-systems lesson: sixteen phones behind one router share one IP address, and to your own backend's rate limiting that looks like a single wildly abusive client. The first time the whole fleet sat foregrounded polling for config, it rate-limited itself into a corner. Now idle fixtures get backgrounded, and the fleet is polite to the API by design.
What we would tell you to expect
- Buy the managed hub. Programmatic port power is the difference between a lab and a pile of phones.
- Decide your credential story before your driver: no biometrics, known passcode, and never loop an entry.
- Screenshots are ground truth; UI trees are fast but lie at transitions. Assert on the picture when it matters.
- Write resume-tolerant scripts. Every long flow should be re-runnable from whatever screen the device is actually on.
- Check the automation support window for every OS version you plan to keep on the bench.
- Watch battery health and temperature like production metrics, because in this lab they are.
- Anything recovery depends on must be license-free and human-free.
Why a small private app runs a hardware lab
boopr is a free private app with no ads and no algorithm, where only the friends you added can see what you share, and where screenshot detection is a promise made to users. Promises like that live in OS behavior: secure screens, hardware key storage, real capture events. The farm is how we keep them tested instead of assumed, on the same hardware our users hold. If that sensibility appeals to you, the app is on the App Store and Google Play.
Back to Blog