11 Great Tools for a Monorepo in 2021
Probably the best tools to develop and build your monorepo.
These days, many tools can run “npm install” and “npm run build” in 20 different folders. But, not all tools can facilitate a proper monorepo.
Facilitating a proper monorepo development means solving challenges like operating the test and build processes for decoupled modules, being able to independently publish modules from the project, and managing the partial impact of changes on every affected dependant module in the project.
The list of challenges goes on, to include even “trivial” things like how you manage issues and PRs, which can get hard as you scale development.
Note that a monorepo is NOT a monolith application(!) — it is not built or deployed all at once. It is a group of applications developed separately.
In this roundup, I’ve gathered some of the best tools in the world to construct a “monorepo”, where you can build multiple modules inside a single project, with a decent developer experience that scales.
The list isn’t ranked and aims to outline the strengths of each tool based on its own merits. I hope this can help you save time and find the right tool.
Feel free to comment below and share your own insights. Cheers!
Also read:
1. Pnpm/Yarn Workspaces
Late addition: I recommend using pnpm workspaces as first choice. Both are great really.
Yarn Workspaces aims to make working with monorepos easy, solving one of the main use cases for yarn link in a more declarative way. Your dependencies can be linked together, which means that your workspaces can depend on one another while always using the most up-to-date code available. This is also a better mechanism than yarn link since it only affects your workspace tree rather than your whole system.
Workspaces helps with a few issues, making it a great monorepo setup:
- It sets up a single
node_moduleswithout repetitions or cloning dependencies throughout different packages in the project. - All your project dependencies will be installed together, giving Yarn more latitude to better optimize them.
- Yarn will use a single lockfile rather than a different one for each project, which means fewer conflicts and easier reviews.
- It allows you to change the code of one of your packages and have those changes instantly visible to the other packages that use it. Any modification to one’s source code is instantly applied to the others.
As such, Yarn workspaces makes a very powerful combo with pretty much any tool on the list, and especially tools like Bit, Nx, and Lerna, acting as a lower-level layer of your monorepo management abstraction.
However, you can also publish directly with workspaces. When a workspace is packed into an archive, it dynamically replaces any workspace: dependency with a package version, so you can publish the resulting packages to the remote registry without having to run intermediary steps — consumers will be able to use published workspaces as any other package. Cool!
2. Bit
Bit is a next-generation tool for building modular projects.
In short, it solves the key problems of a monorepo:
- Configurations on the workspace and component level
- Code sharing on the next level via composability
- Standardization of code, tech, and dev tasks
- Ownerships over components / features / experiences
- Build, test, and deploy changes on a graph
- Auto-defining and managing dependencies at app/component level
- Dev experience that stays simple at scale
It creates a workspace that lets you easily create and compose many components and applications, taking away the pains of configurations, and makes it easy to update changes, run build/test tasks, and even deploy changes across components and apps with next-to-zero configurations and in a super fast and modular way on the graph of components.
With just a few CLI commands you can setup an entire monorepo with React apps, a component library, packages, and super-performant build and test pipelines that work with just about any tool possible for testing etc.
$ bit new react my-workspace$ bit create react-app apps/my-react-app1 component(s) were created
my-name.my-scope/apps/my-react-app
location: my-scope/apps/my-react-app
env: teambit.harmony/<aspect> (set by template)$ bit run apps/my-react-appapps/my-react-app app is running on http://localhost:3000
With Bit, many of the common problems in a monorepo no longer exist. For example, code-sharing; You can consume and use any app, feature or component from any application or library to another with ease.
Bit also prevents and solves common pains with monorepo dependency management — here’s a short overview written by the maker of pnpm.
In essence, with Bit you can easily develop, version, manage, build, test, document, deploy, and publish many “components” in the same project. Bit “knows” which component and app depend on each other, how, and where — so every change you make propagates smoothly across the monorepo.
Bit’s workspace manages all the relationships between components in the project. When you make changes to any component, Bit builds and tests it in isolation, and propagates the change up the dependency graph.
Components can be bulk published, as independent packages, to NPM and/or to the bit.dev platform for collaboration, consumption, and documentation.
Bit’s UI helps you view the development of your monorepo. As you code, and each component is documented, tested, built etc, you can visually see what’s happening with live feedback and hot-reloading.
Bit provides decoupled dev environments — Reusable and customizable modules that configure and “bundles” together different services needed throughout the life-cycle of an independent component such as compiling, bundling, testing, linting, documenting, and more.
3. NX
NX is an advanced set of extensible dev tools for monorepos, with a strong emphasis on modern full-stack web technologies.
NX aims to provide a holistic dev experience via CLI (with editor plugins), and capabilities for controlled code sharing, and consistent code generation. It also provides incremental builds, so it doesn’t rebuild and retest everything on every commit you make, resulting in faster build times.
NX’s command execution allows for consistent commands to test, serve, build, and lint each project. It uses a distributed computation cache so if someone has already built or tested similar code, Nx will speed up the command for everyone else instead of rebuilding or retesting the code from scratch.
With Nx you can use your preferred framework, integrate with modern tools you’re probably already using. For example, NX lets you use out of the box integration with Cypress, Jest, Typescript, Prettier, and other tools.
The NX team also provides NX cloud, with smart computation memoization in the cloud and faster builds to help your team working with NX deliver faster.
4. Rush
Rush is a powerful monorepo infrastructure by Microsoft + Open Source. It aims to help you, well, build and publish many packages in one repository.
Some of the key features of rush include a single NPM install (also works with Yarn and pnpm) so you can install all dependencies for all projects into a common folder, using isolated symlinks to reconstruct an accurate “node_modules” folder for each project.
This also helps to ensure there are no phantom dependencies, so you won’t accidentally import a library that was missing from your package.json, and there are no dependency duplications where you find 10 copies of lib in your node_modules.
Automatic local linking means all your projects are automatically symlinked to each other, and when you make a change, you can see the downstream effects without publishing anything, and without any npm link headaches.
Rush’s unique installation strategy produces a single shrinkwrap/lock file for all your projects that installs fast. Rush detects your dependency graph and builds your projects in the right order, so if two packages don’t directly depend on each other, Rush parallelizes their build as separate processes.
If you only plan to work with a few projects from your repo, Rush provides subset and incremental builds so rush rebuild --to <project> does a clean build of just your upstream dependencies. After you make changes, rush rebuild --from <project> does a clean build of only the affected downstream projects. And rush build delivers a powerful cross-project incremental build. Rush even handles cyclic dependencies, by separating versions for projects.
When you want to release, Rush supports bulk publishing, so it detects which packages have changes, automatically bumps all the relevant version numbers, and run npm publish in each folder.
Rush also helps to implement and enforce development policies. For example, when a PR is created, you can require developers to provide a major/minor/patch log entry for the affected projects, which will later be aggregated into a changelog file upon publishing. It also helps you enforce stuff like pre-publish reviews, specific dependency versions, and so on.
5. Lerna
Lerna (named after the home of Hydra, the multi-headed beast) is a “tool for managing JavaScript projects with multiple packages”.
Lerna was created to solve the multi-package for Babel, to optimize the workflow of managing multi-package repositories with git and npm. It’s essentially tools and scripts to effectively manage and publish many independently versioned packages in a single Git repository.
my-lerna-repo/
package.json
packages/
package-1/
package.json
package-2/
package.jsonThe two primary commands in Lerna are lerna bootstrap and lerna publish.bootstrap will link dependencies in the repo together. publish will help publish any updated packages.
You can manage your project using one of two modes: Fixed or Independent.
Fixed mode Lerna projects operate on a single version line. The version is kept in the lerna.json file at the root of your project under the version key. When you run lerna publish, if a module has been updated since the last time a release was made, it will be updated to the new version you're releasing. This is the mode that Babel is currently using.
Independent mode Lerna projects allows maintainers to increment package versions independently of each other. Each time you publish, you will get a prompt for each package that has changed to specify if it’s a patch, minor, major or custom change. Independent mode allows you to more specifically update versions for each package and makes sense for a group of them.
The ‘lerna.json’ file is a list of globs that match directories containing a package.json, which is how lerna recognizes "leaf" packages (vs managing the dev dependencies and scripts for the entire repo). Example:
{
"version": "1.1.3",
"npmClient": "npm",
"command": {
"publish": {
"ignoreChanges": ["ignored-file", "*.md"],
"message": "chore(release): publish",
"registry": "https://npm.pkg.github.com"
},
"bootstrap": {
"ignore": "component-*",
"npmClientArgs": ["--no-package-lock"]
}
},
"packages": ["packages/*"]
}Even if you’re not intending to publish to NPM, Lerna can still be helpful in managing versioning and common development tasks in a monorepo.
6. Bazel Build System (Google)
Google introduced the Bazel build system. It is an open-source build and test tool similar to Make, Maven, and Gradle that uses a human-readable, high-level build language. Bazel supports projects in multiple languages and builds outputs for multiple platforms. It supports large codebases in a big monorepo or across multiple repositories, and large numbers of users.
Uber Developers uses Bazel to build their Go monorepo. Uber writes most of its back-end services and libraries in Go, which in 2018 were all grouped into a large Go monorepo which now have over 100,000 files. Bazel allowed this project to scale, cutting build times, and supporting its growth.
Here’s a nice small open-source project with Bazel as a demo:
Bazel is designed to work at scale and supports incremental hermetic builds across a distributed infrastructure, which is necessary for a large codebase. With Bazel’s remote cache, build servers can also share their build artifacts. Bazel caches all previously done work and tracks changes to both file content and build commands. A package is built and tested only when something has changed either in the package or its dependencies.
Bazel runs on Linux, macOS, and Windows. Bazel can build binaries and deployable packages for multiple platforms, including desktop, server, and mobile, from the same project. Many languages are supported, and you can extend Bazel to support any other language or framework.
7. Buck Build System (Facebook)
Buck is a build system that encourages the creation of small, reusable modules consisting of code and resources, and supports a variety of languages on different platforms.
It is developed and used by Facebook, and has gained its fame by serving as the official build system of the FB monolith and thanks to being used by teams like Uber Developers to greatly reduce build times. The team at AirbnbEng marked 50% faster builds and 30% smaller apps.
Buck is designed to build a monorepo, and support for the monorepo design motivated Buck’s support for cells and projects.
It is Facebook’s experience that maintaining all dependencies in the same repository makes it easier to ensure that all developers have the correct version of the code and simplifies the process of making atomic commits.
Buck is commonly used for Android and iOS development. You can find more use-cases for buck here in this showcase page.
Buck can help you and your team in many ways:
Like other tools on the list, Buck builds independent artifacts in parallel to take advantage of multiple machine cores. It reduces incremental build times by keeping track of unchanged modules so that the minimal set of modules is rebuilt. And, Buck only uses the declared inputs, which means everybody gets the same results.
More cool features include buck query, which helps you better understand your dependencies and what is required to build your product. And buck project helps your IDE better “understand” the project your building.
8. Pants Build System (Twitter)
In 2014, Twitter introduced its monorepo build system called Pants. Today, on v2, Pants aims to be a fast, scalable build system for growing codebases. It’s currently focused on Python, with support for other languages coming soon.
Pants uses a fine-grained workflow, and isolates each work unit from side effects, so it can utilize all available cores. Some of Pant’s best features include explicit dependency modeling, fine-grained invalidation, shared result caching, concurrent execution, remote execution, and extensibility and customizability via a plugin API.
The Pants engine is written in Rust, for performance. The build rules are written in typed Python 3, for familiarity and simplicity. The engine is designed so that fine-grained invalidation, concurrency, hermeticity, caching, and remote execution happen naturally, without rule authors' intervention.
Not using Python? Move on, nothing to see here. But if you do, you might want to take Pants for a test run before setting up your build system.
9. Please Build System
Please is a cross-language build system with an emphasis on high performance, portability, extensibility and correctness.
Please makes sure build steps are executed in their own hermetic environment, with access to only files and env variables that they have been given access to. Incremental builds means it only builds what it needs to, and it also provides task parallelism, and distributed caching, for a reliable and performant build system at scale.
Please also aims to focus on dev experience, so you can enjoy a famlar CLI and define aliases for common tasks leveraging auto-completions. Written in Go, Please provides all this user experience with no runtime dependency. And, there’s no single large workspace file with too much configs to handle.
It as an intriguing option to explore as your monorepo’s build system.
10. Oao
Oao isn’t the most mature, rich, or easily usable tool on the list, but it’s interesting none the less. It is a Yarn-based, opinionated monorepo management tool thatp provide monorepo features like installing all dependencies, adding/removing/upgrading sub-package dependencies, validating version numbers, determining updated sub-packages, publishing everything at once, updating the changelog, etc.
Oao lets you run a command or package.json script on all sub-packages, serially or in parallel, optionally following the inverse dependency tree. And, it supports yarn workspaces, optimising the monorepo dependency tree as a whole and simplifying bootstrap as well as dependency add/upgrade/remove.
Support for non-monorepo publishing: benefit from oao’s pre-publish checks, tagging, version selection, changelog updates, etc. also in your single-package, non-monorepos. Note that Oao uses a synchronized versioning scheme so a master version is configured in the root-level package.json, and sub-packages will be in sync with that version . You can try it out here.
11. Bolt
Boltpkg aims to be a a “ Super-powered JavaScript project management tool”.
Bolt implements the idea of workspaces on top of Yarn. Bolt CLI is largely a replacement of the Yarn CLI. You can use it on any Yarn project.
As we know, workspaces are nested within a larger project/repo. Each workspace can have its own dependencies with its own code and scripts. Workspaces can also be grouped into sub-directories for organization.
Using Bolt, you can install the dependencies for all of these packages at once (and you can do it really really fast). And, when you specify a dependency from one workspace to another, it will get linked to the source. This way, when you go to test your code, all your changes get tested together.
What about Git Submodules / Git Subtrees / Git Slave?
Honorable Mentions ✨
Watch out, some of these are no longer actively maintained. ✋
