Continuous Delivery for Hardware IP

Reducing developer frustration with an automated solution that works with a variety of circumstances and operating systems.

Background: ARM technology is at the heart of a computing and data revolution that is transforming the way people live and businesses operate. ARM’s energy-efficient processor designs and software platforms have enabled advanced computing in more than 180 billion chips and securely power products from the sensor to the smartphone and the supercomputer. Together with 1,000+ technology partners, ARM is at the forefront of designing, securing, and managing all areas of AI-enhanced connected compute from the chip to the cloud.

Senior Software Engineer Chaitanya Dwivedi outlined the two business problems that need to be solved simultaneously:

1. the need to deliver more products of increasing complexity.

2. the need to increase the quality of the products being delivered.

"The traditional IP delivery approach involves creating and checking deliverables right before the delivery milestone," explained Dwivedi. "This means the unsustained quality of deliverables between milestones and untracked risks for the next milestone."

On top of that, traditional solutions also rely heavily on manual reviews and don’t guarantee any consistency across different product lines. "We wanted to tackle the above challenges by developing a solution that standardizes our delivery mechanism, automates the review process, and enables us to adopt improvements to the deliverables," added Dwivedi. "And this has to happen while the projects are running to efficiently deliver increasingly complex products."

Goals:  Standardize flow for packaging IP deliverables.

"The extensibility of Jenkins was the prime reason we were able to make an excellent solution for our organization. We were able to offload a lot of work to plugins and that made a lot of difference."

Chaitanya Dwivedi, Senior Software Engineer, ARM

Solution & Results: Dwivedi worked with Productivity Engineering Manager Eric Vasquez and Senior Director HW Engineering Stephen Hill to undertake a solution using Jenkins. "The most efficient way to solve our problems was to incorporate Continuous Delivery across engineering teams," said Dwivedi. "Our solution had two parts. The first was the standardization of the CD pipelines. Then came the implementation of a YAML interface to Jenkins' pipelines."

"We used Jenkins as our preferred automation server, because of how extensible and flexible it is," he added. "However, we still had a challenge of porting our custom automation scripts to Pipelines."

Standardization was also an important part of the solution and relied heavily on communication and collaboration between various engineering teams. "We effectively charted out a template pipeline for teams to incorporate the delivery process," said Dwivedi, "which included build notifications, fast failures, bug reproducibility, and quality tracking."

With standardized pipeline definitions, all ARM needed was the codification part. Dwivedi, Vasquez, and Hill developed a solution where users could write specifications in YAML and Jenkins would interpret it as a scripted pipeline. This allowed the team to support all of the scripted pipeline features in the YAML interface.

"For the interface to work efficiently," Dwivedi continued, "we ported all of the commonly occurring Groovy coding patterns into a shared library. We also wrote adapter functions to interface with Jenkins' vast collection of plugins." Key capabilities and plugins included Pipeline as Code, Blue Ocean, Pipeline Utility Steps, Git, JUnit, Gerrit Trigger, Slack Notification, HTTP Request, HTML Publish, and SSH Agent.

"With Jenkins, we were not only able to standardize our process," said  Dwivedi, "we also reduced our development time and increased the confidence in our delivery process. Results included:

• improved deliverable quality and consistency

• bug discovery shortened from weeks to days

• reduced high touch error-prone steps

• reduced milestone risks to projects

• standardized checks and reporting for products