Build it and They Will Run

Build it and They Will Run

There’s no doubt about it: the modern software ecosystem has been transformed by the cloud. The availability of “infinitely scalable” infrastructure at a modest marginal cost has created an entirely new way of building, testing, deploying, and managing software. Fast build systems and test-driven development make it possible for distributed teams to build, test, maintain and operate highly available distributed systems.

Take a look at the recent Battery Ventures prediction around software growth (it assumes a 5% CAGR) and you can see that software (and the estimated 25 million developers who touch software as part of their job) is most definitely eating the world.

The Growth of Software

The Journey to Arm

All software runs on infrastructure. That infrastructure is ever more specialized and diverse.
Since “built in the cloud” software is taking over, it makes sense that we’re seeing a modern developer experience drive adoption for architectures like Arm. While it may be hard to see in the thousands of variants of cloud computing droplets, instances, or machines, start shipping software beyond the datacenter you’ll see what I mean. From mobile phones and laptops to cars and drones, once you go everywhere you start to touch a lot of Arm. Like, billions and billions of Arm cores.

This has been an early main drivers of Arm adoption in the datacenter. Simply put, developers don’t want to do anything special to ship their software for a range of infrastructure.

Everything Depends on Build Systems

It’s not enough that software works once. Long-term adoption requires attention to stability, which means repeatability and robust testing. You can’t jump to “at scale” usage without a solid foundation, no matter what the end devices may look like.

This is why build (or CI/CD) systems are the keystone of modern software development workflow. They provide:

  • Rapid feedback on change requests
  • Reproducible builds
  • Faithful tests that reflect real world conditions
  • Risk management for dependencies
  • Time-efficient builds of complex systems
  • Cost-efficient build infrastructure
  • Manageable complexity

Build Systems are Complex!

Build systems are often highly complex and opinionated in their own right, and rarely all that well understood by those who depend on them. This is a primary source of issues when trying to build software for new platforms or architectures.

Take as an example Bazel, a build system from Google that is used to build Tensorflow and adopted by companies from Asana and Dropbox to Braintree. Bazel, which is written in Java, is actually built with itself. Some Java pieces are built with Bazel.

And yet Bazel is not yet fully and completely ported to arm64, which makes things a bit tough if you’re a developer / company shipping software for a multi-architecture world!

Aside from complexity, other challenges include:

  • Lack of native support in hosted build systems
  • Shortage of high-powered cores in high-memory machines
  • Imperfect emulation in QEMU
  • Poor understanding of the depths of dependency chains
  • Complexity of cluster build environments
  • Ubiquity of x86-first, x86-only design thinking in cloud environments

These can all be overcome with diligence and effort. Once you get the flywheel spun up to power the software ecosystem, they are relentlessly powerful.

Build it and They will Come

A key part of our efforts at WorksOnArm over the past year has been to enable the foundational bits of the ecosystem: languages and build systems. Last November at Packet’s IFX event in Las Vegas, we jointly announced a service with Drone (https://cloud.drone.io) which provides free builds for open source projects, including on Arm (32-bit and 64-bit). A few months later, Shippable announced support.

Today we’re pleased to announce that TravisCI has expanded its support to include arm64. This feature rolls out first for Open Source projects (sponsored jointly by Arm, Packet and Travis), utilizing c2.large.arm servers from Ampere hosted at Packet. The first project built like a charm: OpenSSL.

What made the work especially interesting to watch is TravisCI’s use of LXD. Next up: production rollout for commercial projects.

The Squeaky Wheel Gets the Grease

Ecosystems are made up of people, and nearly everyone is good at making a bit of noise when needed! In order for the flywheel to really get going for Arm support in build systems, you should make your needs known.

Excellent Arm servers are available from Marvell and Ampere, as well as on AWS with Graviton. With this, there is no absolute requirement for you to handle the ongoing care and feeding that extra cluster of Raspberry Pi’s for the occasional build on your project. Instead, hook your software system up to one of the several continuous integration tools that are powered by fast arm64 build servers, and test every release (or even every commit) promptly and effectively.

You can unsubscribe at any time.

Explore Topics