Powering Preply’s online education platform with CI/CD

By Amet Umerov

*SUMMARY: To support this online education platform — and over 100,000 students and 25,000 tutors — Preply’s engineering team set out to create a fully automated CI/CD process for its software and development projects. *

CHALLENGE: Maintain their 'everything as code' mantra and enhance their current Jenkins installation. Eliminate manual intervention by using a configuration-as-a-code only solution for all software development.

*SOLUTION: A flexible and scalable CI/CD pipeline that enables their team to introduce and accelerate the deployment of new builds without jeopardizing the entire service. *

RESULTS:

• Daily release rate increased from 1 to 40 and continue growing

• Rollback time decreased from 45 minutes to 60-120 seconds

• Deployment time reduced from 90 to 5-40 minutes (depending on service)

• Vastly improved conversion and user experience

From day one, Preply has focused on innovative learning. Their online educational platform makes it easy for over 100,000 students to find private language tutors based on their profiles.

===

With over 25,000 tutors to choose from, Preply leverages proprietary ranking algorithms – machine learning for recommendations and classifications of tutors based on student profiles – to ensure the perfect tutor/student match. Even during the Pokemon craze, Preply quickly spun up the ability for tutors to get in the game and make learning fun with Pokemon Go tutors. This created a brand new marketplace for players desiring help learning to play the game.

=== === It’s no wonder; their development and operations organizations are equally cutting edge, especially when it comes to building a CI/CD platform. Preply’s DevOps team developed a platform that can scale as quickly as the software their team innovates: from expediting multiple language processing to supporting online tutors from over 185 countries to launching hundreds of releases per day to keep up with demand.

Automating CI/CD processes to accelerate innovation

"When we talk about DevOps tools, a CI/CD system is the first thing that comes to mind," said Amet Umerov, DevOps Engineer with Preply. "We use Jenkins at Preply because we run hundreds of jobs every day, and we need to use many features that aren’t provided by other systems, or if provided, these features have stripped-down functionality."

"In our company, we are trying to use an 'Everything as a Code' paradigm which is about having reproducible infrastructure, monitoring, jobs, credentials, jobs, and many other things in code," Amet said. "Using Jenkins allows us to configure fully reproducible code and fuel a completely automated CI/CD process for all projects inside our company."

Preply’s team initially tried running Jenkins in Kubernetes. However, it wasn’t as compatible, nor did it completely address their specific requirements. To automate their platform, they needed to begin by building a more robust underlying infrastructure for Jenkins, and include Jenkins in more aspects, like configuration, jobs, and plugins.

Diagram of the new Preply’s Jenkins architecture

Enhancing and stabilizing Jenkins to accelerate the development

Preply’s technology team is continuously creating new product features and experiments for rapid product A/B testing, so speed-to-market is critical. "Having a more stable environment frees up developers to run more experiments. The faster you can get the desired outcome, the faster you validate your hypothesis. Stable deployment means that your work no longer blocks development processes for other product teams, and you can deploy independently."

In order to achieve a more stable Jenkins environment, the team knew the first step was to focus on infrastructure. "We opted to use AWS resources like EC2 instances, SSL certificates, and load balancers," Amet pointed out. "And for the Advanced Metering Infrastructure (AMI), we chose to integrate Jenkins with GitHub."

Preply’s team implemented this in combination with CloudFront to allow paths and token validation. With those steps, Amet said they had a ready-made infrastructure with AMI, giving them opportunities to monitor and access credentials within their corporate Vault. But to automate reproducible configurations, they had to tackle the ongoing requirement to update specific Jenkins plugins.  For that, they chose to pin versions  – using a declarative format – of those plugins by leveraging  Docker.

"Docker made it easier to manage Jenkins and its plugin versions," said Amet. "Now implementing Jenkins plugins is a breeze to update and configure. We can run them locally and easily deploy them on production. And the fun part is we can play with plugins or a new Jenkins version on the local level without disrupting other services: our engineers are free to innovate."

Customizing plugins to provide pipeline visibility and eliminate manual intervention

Another 'must-have' plugin for Preply was one that allowed them to store configurations in a human-readable text format. "In addition to text interpretation, the Jenkins Configuration as Code (JCasC) plugin also gives us the ability to describe security configurations, credentials, plugin settings, nodes, views, and many other things vital to our business," Amet noted.

Finally, the CI/CD makes use of the Amazon EC2 Plugin for bootstrapping agents on AWS. Its configuration can also be managed with this plugin. A matrix-based authorization allows them to manage users' permissions from code. With this implementation, Preply’s platform team explains, they have a flow for testing Jenkins changes locally, so they can find and fix bugs before they surface in production.

And while setting up a reproducible Jenkins configuration process was paramount in building their CI/CD, creating the job configuration was equally essential. As we mentioned earlier, we’re talking about managing a few dozens of releases per day. Jenkins Job Builder (JJB) and Job DSL plugins meet the need for the team to manage freestyle projects either using GUI or directly using the REST API.

"The plugins allow us to configure jobs in a human-readable text format (YAML or JSON). And it’s really comfortable to manage jobs with SCM," Amet said. "Basically, we can set up a CI/CD process for our CI/CD tool using JJB. That’s made our development easier. Every engineer can make changes in any pipeline through the Pull Request. No manual changes, only code!"

Reaping the rewards of a new CI/CD platform

More importantly, with a few code changes and by enhancing the plugin architecture of their existing Jenkins instance, Preply’s DevOps team was able to fully automate their CI/CD process for all projects.

"Our time to deployment has decreased drastically: we’ve managed to increase our software release rate up to 40 per day, as opposed to just one per day before we automated our processes," Amet said. "This has helped us to increase the rate at which we develop and launch the product experiments."

The ability to accelerate its product release schedule has also shown an improved conversion and user experience. But what they like the most is that their small DevOps team can introduce and deploy a new service in any way they want without jeopardizing the entire product.

By eliminating manual processes with Jenkins, development became easier and freed this global online education company to focus on innovative services for both their students and tutors around the globe.

Want to get the tech specifics, read Preply’s Engineering Blog on Medium or visit Amet’s * https://jenkinsistheway.io/user-story/jenkins-is-the-way-to-make-your-configuration-as-a-code-possible/.