When your composer build breaks

Yesterday a project on github was moved, causing Drupal’s own build process, and that of many other websites, to “break”. Here’s what happened:

  1. The coder library was removed from github (it’s main home is on drupal.org).
  2. Drupal core’s composer.json had a reference to the now non-existent repository.
  3. Anyone attempting to obtain Drupal by downloading it’s source and running composer install couldn’t, due to the broken link.
  4. Many other developers who tried to build their websites were similarly left disappointed.

This issue on drupal.org captures the problem in more detail.

We’ve been here before

In March 2016 a JavaScript library called left-pad was removed from the npm package manager, causing the builds of many front-end projects that used it to break.1

This seems to be a risk that comes with dependency management, and raises the question of should vendor be committed to version control? I’m hoping that this post will help you answer that.

Notice that I’m only talking about full applications or website codebases here, not libraries. If you’re working on some sort of standalone component for use within a larger project (like a contrib module), you definitely don’t want to ship vendor to your users. Nor do you want to include composer.lock.

A lean codebase

The keep-vendor-outside-git argument favours a lean codebase. There’s some merit to this. After all, upgrading a module is often considered a single, atomic change, and it’s nice when a pull request of the form “Upgrade the EVA module to 1.2” comes down to a single line: 2

diff --git a/composer.lock b/composer.lock
index 378e3be3fa..8f6d7d31a9 100644
--- a/composer.lock
+++ b/composer.lock
@@ -637,17 +637,17 @@
         },
         {
             "name": "drupal/eva",
-            "version": "1.1.0"
+            "version": "1.2.0"
         },

It’s not hard to cross reference that with a changelog in the other project, should you need to. And by downloading the code each time prevents you from modifying a module without auditing it. Every patches is explicitly listed.

It’s interesting that composer’s own documentation recommends this approach.3

Resilience

The commit-vendor-to-git argument focuses on resilience. You don’t want your build process to be dependent on external services that may or may not be available. Furthermore, you want to be able to track all changes to your source code in one place, both the bespoke code and any other libraries.

Pascal Morin articulates this well here: Do you really need composer in production? The CocoaPods dependency manager for iOS also leans towards this position.4

Can we have our cake and eat it?

We’re looking at two objectives - the resilience of not being dependent on packagist/github/drupal.org for building, plus the advantages that come with a lean codebase. By trying to achieve both, are we trying to have our cake and eat it?

In an ideal world, each project would have some kind of artefact repository. One that’s under your control and from where you can obtain every library/version combination you’ve ever used in the project. This is what Maven, a Java dependency management tool, suggests.5

Do I think the core Drupal repository should contain a vendor directory? No, I don’t. I think it’s Drupal’s job to be a component, augmented with contributed modules and perhaps custom code. Every Drupal website is different; as soon as you add a contributed module the contents of vendor change.

But unless you have the resources to manage an artefact repository, your website’s repository probably should contain vendor. You can still have the benefits that composer brings, but why introduce another point of failure at build time?

Notes

  1. How one developer just broke Node, Babel and thousands of projects in 11 lines of JavaScript - The Register

  2. In reality, composer.lock will have a few other changes, like download URLs and hashes corresponding to the version.

  3. Should I commit the dependencies in my vendor directory? - getcomposer.org

  4. Using CocoaPods - guides.cocoapods.org

  5. Best Practice - Using a Repository Manager - maven.apache.org