Configuring iOS real device infrastructure for in-house parallel mobile test execution

Agile development is putting a lot of pressure on traditional CI and testing approaches. It demands faster build and test times so that feedback can be provided quickly to the developers. It also assumes that test system related issues shouldn’t slow down CI pipelines and that tests in isolated, sterile reproducible environments offer the most reliable feedback. For mobile projects, that can also mean performance testing on real devices as well.



Let’s look at the particular case of testing iOS applications on real devices in CI. Execution time for tests is very critical, and testing on multiple devices sequentially would take a long time. The problem gets worse with a lot of test executions and the requirement to keep pace with rapid build frequencies. If the budgets are limitless, then hosted device farms are a great option. Most often teams discover that unlike other cloud services which have gotten cheaper over time, hosted device farms prices don’t come down, but only increase. It’s because new devices get added, and the frequency of test execution keeps growing. As a result, the total cost can get pricey. Additionally hosted device test farms may not benefit the security posture of some organizations that want to secure the protection of their intellectual property.



The approach users take in this scenario is to connect as many iOS devices as possible to available macOS hardware and run test sequentially through CI. But, anyone who is doing this is familiar with the challenges of maintaining and automating test execution on such an infrastructure. They include but are not limited to–cleaning up after a test, starting a new test in a sterile setup, executing tests in parallel on multiple iOS devices (not an issue in newer Xcode version) and more.



One alternate approach is to launch a clean macOS VM which has an iOS device connected for every test execution. Each test run gets a sterile environment and environments with particular test tool settings can be consistently reproduced for very execution. When you build a macOS self-service on-premise cloud with Anka Build, you can programmatically manage and automate association of physically connected iOS devices to macOS VMs for running parallel iOS real device tests.



Anka Build exposes the ability to manage iOS device association with the dynamically provisioned macOS VMs through a command line interface and enables automation of this task. Then, it can easily integrate with the existing iOS CI process.



How does it work?



Let’s say that you have a cluster of centralized build/test Apple machines (on-premise or hosted).



Step 1 – Connect one or more iOS devices to these machines.



Step 2 – Create a group using `anka usb-group` command. The Anka USB group feature eliminates the need to pre-define/attach individual iOS devices to test environments. A dynamically provisioned test environment/VM can, at run time, get a free device from the USB group (think of the group as a pool) attached to it, execute the test and release the device for use in the subsequent test.



Step 3 – Add the iOS devices to this usb group.




anka usb-group --help



Usage: anka usb-group [OPTIONS] COMMAND [ARGS]...



manage usb device groups, and you can attach a group to a vm so the vm could
pull an available usb device from that group



Options:
--help Show this message and exit. [optional]



Commands:



add-device : adds a device to a usb group, if the group...



create : create a new usb group



delete : delete a usb group and release all it's...



list : show all usb groups



list-group-devices : show all devices that belongs to a group



remove-device : remove a device from a usb group




Step 4 – Attach this usb group (pool of iOS devices) to the VM using anka modify command.




anka modify VM add usb-group --help



Usage: anka modify add usb-group [OPTIONS] GROUP_NAME




Step 5 – Push the VM as a base VM to Anka registry.



Step 6 – Now, when you provision test environments with this base VM on the build host with the devices attached, the VMs will claim the first available devices from the group, execute the test and release them for the next test job execution.



This approach provides a clean test environment setup for every test run, reduces the management overhead of claiming/releasing devices and offers a programmatic interface to do this automatically.



If you would rather work with the single iOS device instead of a pool, you can use the same USB pass through Anka USB command line interface.




anka usb [OPTIONS] COMMAND [ARGS]...



Do actions on USB devices



Options:
--help Show this message and exit. [optional]



Commands:



claim : make a device available for attaching to vms,...



list : list all available usb devices on this host



release : release a device back to host availability,...



anka modify add usb-device [OPTIONS] DEVICE_IDENTIFIER




For detailed information on how to setup a self-service macOS on-premise cloud, read this Anka Build blog.