Selecting the Right Tool for Your Monorepo
Monorepos used to be thought of as a solution suitable only to large enterprises, but recently, it seems, the need for them has become apparent to teams of all sizes.
This growth in demand has been met by the creation of numerous tools that offer simpler implementations for this type of code management architecture.
Before we review the most prominent tools, let’s first set out a clear definition for monorepos.
The Wikipedia definition refers to a monorepo as a
Software development strategy where code for many projects is stored in the same repository.
I aim for a more precise definition saying that a monorepo is a single VCS repository where you produce multiple artifacts.
Wait? What?
Isn’t a monorepo the whole codebase of your company? probably not. Some companies may have their whole codebase reside in a single repository, but it is also perfectly fine if you have multiple monorepos. If you use a single repository and publish multiple applications, multiple packages for NPM, or multiple microservices, this can be considered as a monorepo.
So that immediately leads to the next question: what is monorepo good for? Few answers come to mind:
Code Sharing
The most common use of monorepos is for code sharing. When multiple applications need to share a piece of code, creating them in a monorepo and using the code in all applications is the most straightforward approach. Referencing the code directly inside the monorepo saves the overhead of publishing or exporting the shared code to external artifact storage before the resident projects can consume it.
Publish smaller changes
Let’s imagine you have a library or a product that you publish inside your company, sell to customers, or share with the community. The library may be delivered in one of two ways to its consumers:
- As a single package that contains all the code
- As a set of smaller packages, each one with a specific responsibility.
The latter approach can rip high benefits to the consumers of the package:
- They can get smaller changes so the stability of their product is less impacted. Instead of replacing GIANT PACKAGE from 1.0.0 to 1.1.0, they might be able to upgrade only the parts they need, such as packages where bugs were fixed.
- They can decide what parts of the software they are willing to install. In tools that provide plugins-like architecture, the consumer has the power to decide which of the plugins are relevant and only install those.
Reduce the development burden
Splitting a codebase into smaller units that are self-contained can reduce the build and test times in large projects. If the repository is becoming large, breaking it into smaller pieces can optimize test and build time.
Since all packages reside in the same repository, they are aware of each other and a change in one package can trigger only the relevant test and build chain.
Look at the diagram below. A change in package C results in only the build of application A and there is no need to build and test application B.
Functionality
Monorepo tools should at least support 3 key functions:
- Bootstrapping or preparing the package environment, including linking the local packages among themselves.
- Build the code. That might include transpilation and code bundling.
- Test / validate the code: static analysis and test execution.
- Publish the code to the package registry
Let’s go over some of the tools that provide the needed functionality and understand how they do it.
Lerna
This piece of an architectural gem is a dependency for almost 40,000 repositories on GitHub and is almost a synonym for monorepos in JS. A Lerna based repository is built from multiple directories, each one with a package.json file. To configure Lerna, it is only required to define the top-level directories that contain Lerna packages. The loose structure of Lerna lets it support any framework and language flavor, as the actions that can be performed on each package are defined and configured inside the package.json of each package.
Lerna supports the installation of all packages separately under each package’s folder, but also the hoisting of all packages to the root of the workspace, so install and update times are reduced. Lerna is also responsible for linking packages inside the repository, so packages that depend on other packages are linked locally.
To build and test the code you can execute a command on each package with a single Lerna command. Lerna runs the command on each package but does not provide a dependency build.
Publishing is where Lerna excels. Lerna support unified versioning or independent versioning. when publishing packages in the independent mode, only packages that have changed are going to get a version bump.
Yarn Workspaces
Yarn workspaces is also mentioned quite often as a tool for managing monorepos. Yarn repository structure is similar to the Lerna structure (and often they work together).
The workspaces functionality of Yarn, in version 1, only supports installation and bootstrapping of packages. Following a rough start, it is now stable and effective.
Yarn 2, currently in active development, is extending this functionality with functions similar to Lerna such as running commands on each package including parallel run and topological sort.
NX
NX by nrwl is a monorepo for multiple applications, heavily inspired by the Angular CLI, but also extended to support React.
NX works in a centralized manner. An NX repository requires a centralized file (workspaces.json) that defines all the actions that can be performed on the workspace. To perform an action you need to define a dedicated code, called “builder” that runs on the package. A builder can build, test, lint, or any other action on each package.
NX also provides a schema that can be used as a boilerplate for each type of package inside the repository. Generating an NX package using the CLI also generates a boilerplate in the configuration file.
To support additional frameworks or tools, a dedicated builder needs to be configured for it. NX is geared towards bundling and compiling the full application, so it does not support the publishing of single packages.
NX supports distributed and incremental builds by executing only packages that are “affected” by changes. The NX paid cloud service is built to accelerate builds and tests by caching the already built packages and only building incremental differences.
Bit.dev
Bit.dev takes a different approach altogether which extends beyond the single repository. Bit works in a distributed manner where each package can reside in any repository. Bit server (self-hosted or a paid cloud service) is storing versions of the packages from all repositories that changed it, similar to NPM.
Bit does not require to have each package defined with a package.json, and it can generate a package.json based on the code and global configuration.
Each package in Bit is associated with a “compiler” which contains the full configuration required to build each component. Similarly, a component is associated with a “tester” that contains test configuration.
The SaaS version of Bit.dev also supports the publishing of packages so they can be consumed as NPM packages.
Honorable mentions
There are some additional tools worth watching:
Boltpkg, also under active development, is employing a similar approach to Lerna and Yarn 2.
Bazel is a google tool for large monorepos. Bazel goes beyond Javascript and supports multi-languages, and can also run on parallel machines (so it is useful if you are Google and own a cloud infrastructure).
RushJS, developed by Microsoft is another tool that should support the full lifecycle of monorepos, including bootstrapping, incremental and full builds, and publishing. Rushjs did not get a lot of traction yet, but with the Microsoft -> Github -> NPM acquisition chain, it might mature and become more popular.
Conclusion
There is not yet a clear winner in the monorepos arena, but a lot of interesting things are going in that domain. If you find any mistakes I have made, or a cool tool that should be mentioned, do not hesitate to leave a comment.
