Separating Build and Release Pipelines for Effective DevOps
Why should we separate Build, Release into different pipelines?
With modern DevOps tools, we have the flexibility to break Continuous Integration(CI) and Continuous Deployment (CD) into multiple stages.
And, we typically split the pipeline, looking at various factors such as:
- Performance — Reduce the execution time, e.g., Maximize resource usage with parallelism.
- Reuse — Reuse specific steps in a complex pipeline, e.g., Building the frontend for different languages where most of the build is reusable.
- Repeatability — Repeat specific steps, if required, e.g., Releasing the same build for a different environment.
- Fault Tolerance — Retry on partial failures in the pipeline.
- Cost Reduction — Reduce overall costs for execution.
Furthermore, some of these factors have shaped the way we set up CI/CD pipelines.
One such pattern is to split the Build and Release into separate pipelines.
So, in this article, let’s look at the reasons behind this separation and the benefits you will gain by doing it in practice. Let’s get started.
Why Separate Pipelines?
Technically, we can define both build and release in a single pipeline. However, doing so creates several challenges that affect the effectiveness of the pipeline.
The fundamental reason is that the build and release address two different things.
Some of you might wonder, isn’t the build, the prior stage before release?
Though it’s the prior stage, typically, we have multiple build stages before the release. One is when building Pull Requests (or Merge Requests), and the other is after merging a branch.
The former, is mainly focusing on giving the feedback to the developer as soon as possible regarding;
- Code quality.
- Unit tests.
- Test coverage.
- Vulnerability assessments.
It helps to detect and rectify any errors not found in the development environment before merging the code into the shared branch.
The latter is the stage that’s connected with the release.
Still, having the branch build and release separate is worthwhile if we plan to keep the build stage to generate artifacts deployable into multiple environments.
What Goes into Build Stage?
The build stage consists of all the tests we run on top of the codebase and scripts to Compile, Transpile, or Minify the code into a deployable artifact.
In some cases, we might have parameterized builds, targeting different locales. A good example would be building Angular applications, where we build the frontend artifacts per locale.
And the steps are pretty straightforward.
Pull Request Build
- Fetch the code of the PR to a build agent.
- Compile, Transpile, or Minify the code.
- Run code checks that are not time-consuming. (e.g., Unit tests, Code quality &, etc.)
- Update the results (to PR itself and coverage tools).
Branch Build
- Fetch the code from the branch to a build agent.
- Compile, Transpile, or Minify the code.
- Run code checks (quick checks + comprehensive tests like integration tests, e2e tests).
- Upload the artifacts to a release stage.
Deployment Stage is Different
As the name suggests, we do the deployment of the application to the specified environment at this stage. It also supports choosing environments like Dev, QA, Stage, Production, and rollbacks as safety measures.
You can include steps to inject the environment-specific variables in this stage.
Besides, there could be new environment creation (e.g., Blue-Green development) depending on your deployment strategy. So the focus of this stage can be more complex than just releasing the product.
What if it is Just a Single Pipeline?
Yes, we can use a single pipeline for both build and release, as I mentioned above. But the thing is, it comes with several limitations.
After going through what they are, you will probably understand why separate pipelines are the better solution, just like me.
1. Have to Build Everything from Scratch for Each Environment
Suppose we have once run the build pipeline, and this time we want to release it to another environment.
There is no other choice than to re-running the whole pipeline, including the build process. But, unfortunately, it’s not efficient and a waste of resources.
2. Artifacts may Change with Updates
Consider a situation where you run the single pipeline for one environment, and re-run the same build (same code) after some time for a different environment.
The build most likely installs dependencies like NPM, and sometimes the minor versions or patch versions get updated. Because of that, there is a possibility that the final artifacts for one environment will differ from the other.
Tip: Build with independent components, for speed and scale
Instead of building monolithic apps, build independent components first and compose them into features and applications. It makes development faster and helps teams build more consistent and scalable applications.
Bit offers a great developer experience for building independent components and composing applications. Many teams start by building their Design Systems or Micro Frontends, through independent components.
Give it a try →
Building Separate Build and Release Pipelines with Azure DevOps
When we consider popular DevOps tools like Azure DevOps, it offers a pipeline for the build and a distinct type of pipeline called release.
It adheres to the philosophy of separating build and release pipelines and facilitates releasing them into multiple environments. But in some tools, the process breaks down into steps where the last is the deployment step.
Let’s look at how we can configure Azure DevOps step by step to get a better overview.
Prerequisites:
- An Azure DevOps organization.
- A GitHub account (You can also use Azure DevOps for code repo).
- A React App.
You can create a free Azure DevOps organization to try this out. Besides, you can use GitHub actions too. But today, we are going to implement this with Azure DevOps. So, let’s get things automated.
Build Pipeline
1. New Project
At first, let’s create a new project in your Azure DevOps account.
2. Check-in Code
Now, add your local React project to a Git repository. You can use;
git initgit add .git commit -m “message”
To create the repository from Visual Studio Code. Next, you can use the below command with your credentials to push the app from the local machine to your Azure project.
git remote add origin https://user@dev.azure.com/organization/projectname/_git/build-release
git push -u origin --allAnd now, if you refresh the Azure DevOps page, the React app code should be there in the files section.
3. Modify the Pipeline Configuration YML
Let’s create the build pipeline. But, first, let’s open the pipeline configuration stored in the azure-pipeline.yml file.
Pipelines (Navigation Pane) -> Pipelines -> Create Pipeline -> Select Azure Repos Git -> Select the newly added repository -> Select Node.js with React if your project is a React app -> azure-pipeline.yml file
You can modify the file to include the variables defined as follows.
And after that, you can modify the script section to use these variables.
Now click, show assistant (at top right corner) -> Archive Files.
After changing the root folder name, click the “Add” button to continue.
4. Publish Artifacts
Now let’s see how to publish this artifact. From the same show assistant tab, select Publish Build Artifacts. Always remember to place your cursor at the end of the file. That is where the additional code gets added.
Click add, and we are ready to run. So, hit Save & Run.
You will see the job running. And if you get an error saying that you have no free parallelism grant, you can request it here.
Now you can find the artifacts under the Drop folder of the job. It contains the content of the build folder.
Release Pipeline
1. Choosing the Template
Create a new pipeline under the Releases in the Pipelines tab. Select Azure App Service Deployment from there.
2. Choosing the Artifacts
Click Add an Artifact, and select the artifact from our previous job.
Let’s enable the continuous deployment trigger by clicking on the lightning sign over the artifact we added.
3. Choosing the Deployment Environment
And after that, we can set up the Azure App Service details by clicking on job 1, the task link on the stage we created.
Fill out the fields, and save. Now you can go back and create a “new release” to deploy the application.
Finally
After reading the article, I hope you are fully aware of the value of separating the build and release pipelines in DevOps.
Besides, many tools like Azure DevOps encourage it, as it’s a well-known best practice. And, I hope you will use this separation in your next project.
If you have any questions, let me know in the comments below. Thanks for reading !!
Build composable web applications
Don’t build web monoliths. Use Bit to create and compose decoupled software components — in your favorite frameworks like React or Node. Build scalable frontends and backends with a powerful and enjoyable dev experience.
Bring your team to Bit Cloud to host and collaborate on components together, and greatly speed up, scale, and standardize development as a team. Start with composable frontends like a Design System or Micro Frontends, or explore the composable backend. Give it a try →
