T2 Chip and Anka co-existence

Since the release of new 2018 mac minis, we have received several questions from users on the impact of T2 security chip included in the new mac minis on Anka Build setup. This blog describes in detail how Anka technology foundation is agnostic to T2 and can co-exist with it.

There is enough written about the new T2 mac security chip, but as a quick summary, let’s look at the areas where it impacts existing mac administration and management workflows. T2 security chip is a security processor that is in charge of securing the following components of Mac hardware.

  • It validates the boot process (secure boot).
  • It does on-the-fly encryption of storage. Its full encryption with no performance loss due to software encrypt/decrypt I/O.
  • It protects the hardware from malicious usage scenario with hardware disconnect that ensures the microphone disables whenever the lid closes.


  • By default, all the new Mac Hardware including mac mini 2018 and newer iMac Pros have the T2 security chip enabled. So, what does it mean?

    1. Only certified operating systems, macOS and Windows, can boot on this hardware.

    2. NetBoot is not possible with Macs that have T2 hardware even when it is disabled.

    How does this affect the existing workflows?

      Installing other OS on top of Mac bare metal
    Before the introduction of T2 chip, it was possible to install software like ESXi, etc. on the Mac hardware. Now, even after disabling T2, access to SSD is still restricted, so booting from external USB is the only way to install other OS on the hardware. In this scenario, the OS can’t use the fast SSD inside the mac and will run on top of the slow USB device.

      Imaging
    Netboot has primarily been used to implement imaging in order to manage a group of macs. Now, with Netboot not working on the newer Mac hardware, it’s impossible to image a large group of macs consistently. It’s quite challenging now to set up and administer mac infrastructure for iOS CI.

    Anka installation on newer Macs is not impacted with T2 enabled. This is because Anka virtualization for macs is built on top of macOS hypervisor.framework. It works like any other mac application, which leverages and can work with full security on, utilizing all of the Mac hardware resources, including the fast internal SSD for maximum performance. So, Anka VMs can run with no impact of T2 enabled on the host machines.

    Anka VM on 2018 mac mini


    Anka Build is an alternative to explore if you wish to set up an agile and scalable mac infrastructure to provision immutable, container-like macOS VMs on-demand for iOS/macOS CI. Anka Build now supports Mojave and APFS.

    iOS build and test in Anka Build VMs on new 2018 mac mini – No performance compromise compared to running native on Mac Pro and Mac mini

    The new Mac mini 2018 which was released this month, with 8th Generation Intel CPU going up to six cores and support for up to 64GB of memory, is now a powerful contender for enterprise-grade macOS workloads. Here at Veertu, we were looking forward to this new mac hardware release to run Anka Build. Anka Build works out-of-box on the new mac mini hardware because it uses macOS Hypervisor.framework for virtualizing macOS VMS on top of macOS. What’s exciting us for is the performance of Anka Build on this new hardware and how it can change the game for iOS CI infrastructure. With new superpowerful CPUs, virtualization gets a boost, and we are seeing Anka Build VMs perform faster than running the same workload native on the older twelve core Mac Pro.

    Anka virtualized build/test iOS workload on new 2018 mac mini run faster than running it native on a twelve core Mac Pro

    We took the opensource Kickstarter iOS application and executed Xcodebuild build and test job in one Anka VM with 6vCPU (with HTT enabled) on the 6-core new 2018 mac mini and also native on a 12-core Mac Pro host. Anka VM performed ten percent better than native 12 core Mac Pro.
  • One Anka VM with 6 vCPU(HTT) on 2018 6-core mac mini – 1m21s
  • Bare metal Mac Pro 2013 with 12 cores (SSD) – 1m24s


  • Multiple Anka virtualized build/test iOS workloads on the new 2018 mac-mini run as fast as running it native on a quad core mac mini

    Again, we took the same opensource Kickstarter iOS project and executed Xcodebuild build and test job in two concurrent Anka VM with 6vCPU on the 6-core new 2018 mac mini and also native on a quad core mac mini. Both concurrent Anka VMs performed the same as a native quad-core mac mini.
  • Two Anka VM with 6 vCPU on 2018 6-core mac mini – 2m6s
  • Bare metal 4 core mac mini 2012 (2.3ghz SSD) – 2m4s


  • We also noticed that running concurrent Anka VMs on the new 2018 mac mini perform significantly better than running the same workload native on a dual core mac mini.
  • Two Anka VM with 6 vCPU on 2018 6-core mac mini – 2m6s
  • Bare metal dual-core 2014 (2.6 GHz, SSD) – 3m5s


  • So, what’s our takeaway from this?
    1. Devops and iOS software development teams are usually concerned about the performance impact of moving their iOS CI workloads to a virtualized environment. Since Anka Build virtualization on the new 2018 mac minis performs better or as good as native (better than native mac pro, significantly better than the native dual-core and as good as native quad-core), DevOps should explore running their workload on Anka Build macOS cloud on the new Mac minis.
    2. Most teams running virtualized environments for their IOS workloads are using Mac Pros (6 or 12 cores). They should explore the option of gaining a significant increase in the performance of their existing virtualized workloads by running them on Anka Build on top of the new 2018 mac minis.
    Contact us at info@veertu.com or https://slack.veertu.com/ to get more details on these benchmarks or with specific questions on running your iOS/macOS CI workloads.

    Anka Build Runner for GitLab CI

    Dynamically provision iOS build/test environments from GitLab CI on Anka Build macOS Cloud

    Anka Build software is used to configure an AWS like macOS cloud on mac hardware. Once Anka Build setup is complete, DevOps can create multiple build/test images for their iOS CI, and dynamically provision container like instances on Anka Build macOS cloud from these images.

    GitLab CI is a part of GitLab and is used by many dev teams as their platform of choice for CI. GitLab CI setup includes GitLab Runner and build cloud infrastructure to provision build containers or VMS.

    Today, we are announcing the availability of GitLab CI Runner for Anka Build. This will enable teams who are using GitLab CI to execute their iOS CI pipelines on their Anka Build macOS cloud. Check out detailed setup instructions at https://github.com/veertuinc/gitlab-runner.

    Setup and Architecture

    Anka Build Controller – This is the management module of Anka Build cloud software.

    Anka Build Registry – This is docker registry like registry module to store, manage and distribute Anka macOS images.

    Anka Nodes – Mac hardware on which the Anka VMs are provisioned and build and test jobs execute

    Anka GitLab CI Runner – Anka Build plugin which communicates with the GitLab CI APIs and Anka Build Controller.

    Steps
    1. Setup Anka Build Cloud on your mac build and test hardware.
    2. Prepare you macOS build Anka image/VM template with project dependencies.
    3. Install git on it.
    4. Push the Anka image/VM template to the Anka registry.
    5. Setup and configure GitLab CI Anka Runner. We recommend that you run it on the same instance where Anka Build controller is running. You can set it up as a docker container or install it manually. GitLab CI Anka runner should be able to communicate with both your GitLab CI service/server and Anka Build controller.
    6. Run GitLab CI Anka Runner.


    The setup is complete and your GitLab CI iOS pipeline should dynamically provision ephemeral Anka VMs on the Anka Build Cloud and execute builds.

    Project CI Pipeline in GitLab CI



    Anka GitLab Runner provisioning on-demand VM from an Image template on Anka Build macOS Cloud



    GitLab CI Pipeline job executing inside Anka macOS VM



    Job Completed and Anka macOS Vm is deleted

    Shared storage or local storage based macOS Cloud for iOS CI

    It’s obvious that mobile development is very mainstream now and as a result, the size and number of mobile development projects continues to grow. However, as projects and teams scale, and the number of dependencies increase, ensuring a consistent and stable build for all developers while ensuring code and test quality is a much bigger challenge.

    Mobile CI Infrastructure Requirements

    For a mobile CI system to scale, it needs to enable management of project dependencies, build and test environment dependencies and faster build times. While strategies for scaling build systems vary widely across use cases, most implementations focus on ephemeral approaches to managing the mobile job environment. This means the use of self-contained, immutable build environments to ensure proper versioning and verified stability. For many of these environments, typically virtual machines or docker containers, parity of performance becomes the chief concern. Management of these VMs and containers can prove challenging, however, and requires scalable architecture and a reduction in the number of system dependencies.

    Container and Container registry for iOS CI Cloud?

    In the container world, there are ample resources for building, scheduling, and deploying stateless applications and batch processes. As you know, containers cannot be used for the typical macOS CI use case, but there are examples to take from this highly scalable technology. These include projects like Mesos, Kubernetes, and others, which enable the ease of container management across very large environments on-premise or in public clouds like AWS or GCP. An additional hallmark of the container technology space is the use of registries to host base and additional container layers to add continued scalability using incremental approaches to infrastructure and application development.

    When scaling out infrastructure, registries can be used to download a container as an artifact, enabling workloads to be executed using local system resources, no matter the resource need. Further, the layering and artifacting of images allow this scale to expand across massive pools of resources, executing computations and compiles in more distributed fashion when required. Typically, containers downloaded by servers for running distributed compute, applications, or compile can perform moderately to extremely well using local CPU, memory, and especially storage. By using local storage to perform I/O operations within the container, directly on disk, the performance of these containers remains lightweight and easily scalable. Anka registry architecture is purpose-built around these concepts, and particularly well-suited for mobile CI systems. In contrast to this topology, the virtual machine world holds many different challenges.

    Storage setup for iOS CI Cloud

    Virtual machines often used to manage large stateful applications, can be a bit more daunting to manage. Contrasted with containers, their on-disk size is typically far greater, often exceeding 20Gb or more for a simple base operating system layer. In the macOS world, these images are often more difficult to manage because of limitations or requirements of the Apple ecosystem. Add to those additional dependencies, security tools, or large projects required to be included, these images can exceed 30Gb and can even grow beyond this when leaving room for a job or test execution, results exports, or caches to make the build faster. With a higher storage footprint, VMs become difficult to update and distribute at scale and typically have significantly higher I/O for compile jobs and other tasks. In traditional virtual machine architectures, stateful applications running inside of VMs are managed across a shared storage array, connected to the compute pool running virtual machines through various high-speed network connections. These storage arrays can be comprised of any number of technologies, including traditional spinning disks, flash storage types, or a combination of both with the in-memory cache as well.

    For the majority of use cases (like providing high availability and virtual machine migration when underlying hardware fails) this networked storage array can perform adequately to workloads exhibited. However, in build and CI environments, the I/O patterns of project compile and some test tasks can constrain these platforms and harm build performance. Because CI system scalability is achieved by a large number of ephemeral instances to execute build and test environments, the compile workloads in addition to VM distribution can add considerable load to the shared storage layer regardless of technology. This is because communication with the filesystem takes place both on the filesystem and over the network, adding latency and slowing throughput compared to local disk.

    Anka Registry allows for the download and distribution of VM images across a large number of underlying hosts. These hosts can be easily updated and can launch additional instances from local caches without requiring that the image is downloaded again. This architecture removes dependencies of network and storage during compiles run by virtual machines hosted on shared arrays and allows mobile build systems to more easily scale and perform faster. If you want the simplicity and the performance of running your iOS CI environment atop local SSDs, Anka is the platform for you.

    Provisioning on-demand macOS virtual machines with Jenkins and Anka Build for iOS CI

    Guest Blog Post By Peter Wiesner, Senior Software Engineer @Skyscanner

    Every year Apple releases a new version of Xcode. CI systems for iOS application development need to adopt it, so developers can take advantage of the new iOS features. CI systems using Anka Build have a head-start here. For folks not familiar with Anka Build, read more details here.

    You only need to create a new tag for the current Anka macOS virtual machine with the new Xcode installed. Anka ships with features helping to automate this process.

    In this blog post, I will show you how to do this. We will use the following solutions:
    • anka create to create the macOS virtual machine from a script
    • Jenkins pipeline to run the jobs on demand
    • anka run to execute batch commands on the virtual machines
    About the prerequisites:
    • Jenkins with pipeline plugin installed
    • a mac with anka-create label connected to Jenkins. We will use this native node to create and provision the virtual machine
    • Anka Build package version 1.4 current release installed on the mac
    Jenkins pipelineLet’s create a Jenkins pipeline project with following script. The key tricks are:
    • use Jenkins parameters to provide configuration (Xcode version to install)
    • use a separate repo for the actual provisioning files
    • always save the state of the VM on pipeline failure so next iteration is faster
    pipeline {
    
        agent none
    
        options {
    
            timeout(time: 240, unit: 'MINUTES')
    
        }
    
        environment {
    
            // Provide credentials here (anka username/pwd, keychain passwords,
            credentials
            for tool packages etc.)
    
    }
    
    parameters {
    
        //Provide mutable configuration for scripts (anka VM name, anka
        tag,
        Xcode version)
    
    string(name: 'ANKA_VM_NAME', defaultValue: 'anka_VM_node',
        description: 'Name of the Anka VM')
    
    //...
    }
    
    stages {
    
        stage("anka-create") {
    
            agent {
    
                node {
    
                    // There is a native mac connected to Jenkins with this
                    label
                    label 'anka-create'
    
                }
    
            }
            steps {
    
                create_task()
    
            }
    
            post {
    
                always {
    
                    // Create a vm that stores the state of the VM on exiting. When fixing
                    failures,
                    this can speed up the process a lot
    
                    sh ""
                    "#!/bin/bash
    
                    if [\$(anka list | grep current_state_vm | wc - l) == 1]
    
                    then
    
                    anka delete--yes current_state_vm
    
                    fi
    
                    anka stop\ $ANKA_VM_NAME
    
                    anka clone - c\ $ANKA_VM_NAME current_state_vm
    
                    if [\$(anka list | grep\ $ANKA_VM_NAME | wc - l) == 1]
    
                    then
    
                    echo "Deleting Anka VM: \$ANKA_VM_NAME"
    
                    anka delete--yes\ $ANKA_VM_NAME
    
                    fi
    
                    ""
                    "
    
                    cleanWs()
    
                }
    
            }
        }
    }
    }
    
    def checkout_provisioning_scripts() {
    
        // It is useful to decouple the actual provisioning script from this
        pipeline script
    
        // Easy to do updates and helps reading the pipeline script
    
        // Clone the provisioning script from a GitHub repo
    
        timeout(15) {
    
            checkout([$class: 'GitSCM', branches: [
                    [name: 'master']
                ], ,
    
                extensions: [
                    [$class: 'CloneOption', depth: 1, noTags: false,
                        shallow: true, timeout: 30
                    ]
                ],
                userRemoteConfigs: [
                    [name: 'origin', refspec:
                        '+refs/heads/*:refs/remotes/origin/*', url: 'SSH_GIT_URL'
                    ]
                ]
            ])
        }
    
    }
    
    def create_task() {
    
        cleanWs()
    
        credentials.withAllCredentials {
    
            checkout_provisioning_scripts()
    
            // Once we cloned the repository, print the environment variables
            to see
            if job parameters and variables are correctly passed to this task
    
            // provision.sh comes from the repo
    
            sh ""
            "#!/bin/bash
    
            printenv
    
            sh. / provision.sh
    
            ""
            "
    
        }
    }
    


    Creating the VM

    The Jenkins pipeline calls out to a provision script. This is responsible to utilize anka for provisioning. The key tricks are:
    • collecting all necessary packages beforehand
    • allowing the script to use previously saved virtual machine as starting point (`anka create` can take up to 20 mins)
    #!/bin/bash
    
    # Let's fail on error
    
    set -e
    
    # Retrieve macOS installer and necessary packages
    
    # It's beneficial to store them on a NAS that can be connected to the mac
    node
    
    # We can start the process with previous state of vm if we would like to
    
    # This saves a lot of time
    
    if [ "$START_FROM_BEGINNING" -eq 0 ]
    
    then
    
    #Create VM from the Installer on NAS
    
    ANKA_DEFAULT_USER=$USER_NAME anka --debug create --ram-size $VM_RAM
    --cpu-count $VM_CPU --disk-size $VM_DISK --app "$PATH_TO_OS_INSTALLER"
    $ANKA_VM_NAME
    
    anka modify $ANKA_VM_NAME add port-forwarding --host-port 0 --guest-port
    22 --protocol tcp ssh
    
    else
    
    #Clone VM
    
    anka clone current_state_vm $ANKA_VM_NAME
    
    fi
    
    # Iterate on the setup/install scripts
    
    for file in $files_to_execute
    
    do
    
    # Run script as root
    
    anka run --env $ANKA_VM_NAME sudo -E bash $file
    
    # Run script as created anka user
    
    #anka run --env $ANKA_VM_NAME sudo -E -H -u $USER_NAME bash $file
    
    done
    
    # Stop the VM after we are done, start it up and suspend it, so CI can use
    the super fast start-up feature of Anka VMs.
    
    anka stop -f $ANKA_VM_NAME
    
    anka run $ANKA_VM_NAME ls
    
    anka suspend $ANKA_VM_NAME
    
    # Save the new baseline virtual machine
    
    anka registry -a $REGISTRY_IP_ADDRESS push $ANKA_VM_NAME -t $ANKA_TAG
    Installing necessary tools

    The actual provisioning happens in the `for loop`, where we use `anka run`. These scripts install the tools like Xcode, Xcode CLI, Ruby, git or any packages. The content and order of the scripts are company/case specific, but let me list some general tricks. Keep in mind, when running the `anka run` command, the current working directory will be mounted to the VM and the files can be accessed with a relative path (sweet!).

    Installing certificates

    $P12_PATH contains the path to the exported p12 files location.
    echo $USER_PWD | sudo -S security import $P12_PATH -k
    "/Library/Keychains/System.keychain" -P "$P12_PASSWORD" -A
    Installing Xcode CLI

    $XCODE_CLI_PATH contains the path to the Xcode Command Line Tools package location.
    MOUNTDIR=$(echo `hdiutil mount $XCODE_CLI_PATH | tail -1 | awk '{$1=$2="";
    print $0}'` | xargs -0 echo)
    
    echo $USER_PWD | sudo -S installer -pkg "${MOUNTDIR}/"*.pkg -target /
    hdiutil unmount "${MOUNTDIR}"
    Installing Xcode

    You need to install the xcode-install gem for this. $XCODE_APP_VERSION contains the version number of the Xcode you want to install (like 10.0) $XCODE_REMOTE_URL is an URL from where the Xcode xip can be downloaded (it is worth to download it and upload to a private remote server to have bigger download speed).
    xcversion install $XCODE_APP_VERSION --url="$XCODE_REMOTE_URL" --verbose

    Build cache for faster iOS builds with Jenkins CI and on-demand macOS VM slaves

    Most of the existing iOS CI environments are configured to run the build and tests jobs directly on the Mac hardware. In such installations with large projects, some data like XCode derived data, and code repository get cached on the hardware and subsequent builds use these caches. In this blog, we will describe how you can set up your Anka Build macOS cloud in a Jenkins CI environment to build cache inside the VM and use them for on-demand provisioned Jenkins iOS CI slaves. This will provide the advantages of isolated, reproducible environments and build caches for faster build times.

    We have implemented a Jenkins plugin called “Anka Slave template Prepare” plugin. This plugin enables devops to load and update build cache on existing jobing Anka macOS VM templates in an automated manner. Devops can then use the Anka Build Cloud Jenkins plugin and setup their primary iOS CI Jenkins jobs to always run using the VM templates pre-loaded with caches.

    Visit https://ankadoc.bitbucket.io/using-jenkins/#using-the-anka-slave-template-prepare-plugin for detailed documentation to configure the plugin.

    Anka Slave Template Prepare Jenkins Plugin



    Post Build Action Configuration

    Run docker workloads in macOS VMs on Anka Build – DevOps Uploaded

    There is “Lunchable,” and then there is “Lunchable Uploaded”. Today, we are announcing nested beta support in Anka Build platform to run Docker workloads in macOS VMs. It takes DevOps up a notch for macOS ecosystem. So, why nested support in Anka? Nested for everyone who is familiar with virtualization, means to run a hypervisor inside of a virtual machine. In Anka Platform, this means that you can run docker workloads(container) in Anka macOS VMs. Docker on macOS uses xhyve virtualization and now, with nested support you can run xhyve hypervisor inside Anka macOS VMs.

    In the past few months, we have received a lot of user feedback on use cases to build macOS CI infrastructure which can also support nested workloads on macOS, in addition to the traditional iOS app and macOS app CI. We think of nested support as providing infrastructure to build and test infrastructure for macOS (think Russian nesting dolls).

    So, everyone who is building apps for macOS which uses a hypervisor layer in their application stack can now configure a private macOS cloud for CI and provision on-demand macOS environments for build and tests. Example, folks at Docker might find this interesting to build CI infrastructure for their Docker macOS client, security software macOS clients who use a virtualization layer in their stack could use this to configure CI infrastructure and probably many more use cases.



    We would love to learn and engage with all of you who want to look at this beta feature. Send us an email at info@veertu.com, tweet us (@veertu_labs) or connect with us on slack.

    iOS simulator tests in macOS VMs – XCUITest simulator tests on Anka Build macOS Cloud

    There is some confusion about the feasibility of executing iOS simulator based UI testing in macOS VMs. Anka macOS VMs provide pretty much all capabilities of macOS running on hardware to set up a scalable, agile CI infrastructure for building and testing iOS applications. Since iOS simulator uses software-based rendering for graphics, it can run and execute all UI and non-UI based testing inside Anka VMs.

    We have detailed steps to set up and run iOS builds and tests in Anka Build macOS cloud our blogs. Here is a quick summary of these steps – Setup Anka Build on a cluster of mac build machines, bootstrap your test Anka VM template with all relevant test framework dependencies, push this VM template to the Anka Registry and integrate Anka Build Controller to communicate with your CI tool.

    When you execute iOS simulator test jobs in your CI tool, the job will automatically provision multiple isolated, concurrent macOS VMs from the uploaded VM template and execute UI test jobs. The VMS can be configured to be deleted after the job or kept alive for troubleshooting purposes. Every execution of the UI test job provisions new VMs, hence there is no leakage of any environment variables or dependencies and the UI tests run in a predictable manner.

    View a demo of XCUITest execution in dynamically provisioned concurrent Anka Build VMs – https://youtu.be/wivotJuku24

    Container like ‘anka run’ interface for Anka macOS VM

    Anka run command line interface enables command execution inside Anka macOS VMs directly from the host. It’s very similar to container docker run.

    Why is this interesting – It allows Devops to take their existing container based Linux CI workflow automation, repurpose it for macOS and use it on Anka Build macOS Cloud infrastructure for iOS CI.

    anka run --help

    Usage: anka run [OPTIONS] VM_NAME COMMAND [ARGS]...

    Run commands inside VM environment

    Options:

    -w, --workdir PATH Working directory inside the VM [optional]

    -v, --volumes-from, --volume PATH Mount host directory (current directory by default) into VM . '--volumes-from' is deprecated form [optional]

    -n, --no-volumes-from, --no-volume Use this flag to prevent implicit mounting of current folder (see --volume option). '--no-volumes-from' is deprecated form [optional]

    -E, -e, --env Inherit environment variables. '-e' is deprecated form [optional]

    -f, --env-file PATH Provide environment variables from file [optional]

    -N, --wait-network Wait till guest network interface up [optional]


    ‘anka run’ by default mounts the current folder from the host into the Anka macOS VM and executes commands on this mount point as the current directory. Upon completion of command execution, the mount directory is automatically released. As a developer, this can be quite useful to be able to execute operations inside the VM environment in an isolated manner using artifacts/code residing on the developer machine. Example,

    anka run xcodebuild -sdk iphonesimulator -scheme Kickstarter-iOS build

    –volume (-v) – Mount directory into VM. Mountpoint could be specified explicitly in this form:–volume /host/dir:/mountpoint, or assigned dynamically: –volume /host/dir. If no –workdir option is specified, mountpoint will be selected as working directory for running command.

    Example : anka run -v ~/Projects/anka:/tmp/anka run build

    –no-volume (-n) – By default anka run command mounts current directory into VM, (implicit –volume $PWD option). Specify this option to disable implicit mounting. If no –workdir option is specified, working directory will be set to VM user’s home directory.

    Example :

    anka run -n xcodebuild -sdk iphonesimulator -scheme Kickstarter-iOS build - Will not mount the current host directory and execute build in the VM current directory.

    –workdir (-w) – Specify working directory inside the VM

    Example :

    anka run -n -w /tmp/workspace/project git clone ${URL} src


    anka run -n -w /tmp/workspace/project/src xcodebuild

    –env (-E) – Inherit host environment variables in non-overriding mode. Example : CFLAGS=-O2 CC=clang anka run -E ${VM} make test.c

    –env-file (-f) – Provide environment variables in overriding mode from file specified. Format of the environment file can be:

    VAR=VALUE

    VAR2=VALUE2

    –wait-network (-N) – Resumed VMs require some time to reconfigure networking interface, so scripts which use networking immediately (e.g: git clone) could fail on network inaccessibility. This option postpones execution of the command till network is up. Incorporate it in your VM bootstrapping automation before starting to access network from inside the VM.

    Scaling iOS CI infrastructure with Anka Build

    You start with one iOS app and then it morphs into multiple apps, and you are shipping a new version of all these apps frequently to the app store without compromising on quality. How do you do it?

    First, you build an application architecture that supports the scale – a large team of developers working on same codebase and flexibility and ability to expose new features quickly to the users. Then, you invest in test automation to have multiple sets of tests for different workflows and features in your app. And, you invest in CI and CD infrastructure that eliminates wait times for developers between builds, faster feedback from tests and quick deployment to the app stores.

    In this blog we will focus on how you scale and manage iOS CI build/test infrastructure to support a high velocity of development and increased code quality for releases.

    We see users adopting one of three approaches to address the scale challenge of iOS CI infrastructure and introduce a new method outlined in Option 4.

    Option 1 – Increase the bare metal mac hardware capacity for build and test machines

    This results in increased hardware costs, but also means significantly higher system management overhead. Devops and system management team need to plan for additional complexity in build and test environment configuration management across all the hardware, capacity planning and sharding strategies to test concurrently on multiple versions of application dependencies. Due to imaging-based approach, repeated set of these operations on the same hardware also results in stability issues.

    Option 2 – Consuming hosted CI iOS build services

    These services are provided as multi-tenant/shared basis and in most cases unable to meet the extensive scale requirements of a single user. Additionally, there is an exponential increase in cost as the usage goes up. You are no longer in the lowest/cheapest service tier but have to move to a custom tier.

    Option 3 – Scaling with legacy based virtualized infrastructure

    There are some supported and(some unsupported) approaches to set up a virtualized infrastructure on top of Mac hardware which can be used to spin up iOS build and test environments on-demand. The underlying technology to virtualize in such case uses a shared storage infrastructure to support scale(more than one host). With base VM images stored as snapshots in shared storage, new instances of a VM and all the build and test job IO operations execute on shared storage infrastructure. This places extremely short response times demands on shared storage and can only be achieved at a very high cost, low latency shared storage architecture. As you scale up this infrastructure, the prices for shared storage increases exponentially. The long-term TCO of should be considered before starting with this option.

    Option 4 – Setup cloud/virtualized infrastructure which can scale linearly and operates as infra-as-a-code.

    The concept of containers bring this concept to Linux and Windows infrastructure; however, it’s not yet possible to run macOS inside a container. Anka Build virtualization uses macOS native hypervisor framework to create and run lightweight macOS VMs. Similar to container provisioning, it provides provisioning of macOS environments from ‘macOS.app’ package in a fully programmable manner. Anka Build Cloud exposes features to store, version and manage multiple macOS images in a central registry, similar to container registry and also provides a programmable interface to pull and push these images across various host mac machines. Once the image is pulled on a Mac host machine, immutable VMs on-demand are provisioned by cloning the already pulled image and deleted after the build/test job execution. There is no need for a shared storage architecture attached to each mac host. Additional mac hosts can be added to an existing Anka Cloud setup linearly to scale the CI iOS infrastructure with minimal effort.

    Get more details on Anka Build macOS Cloud architecture and registry setup at www.veertu.com.