Erik Erskine

July Drupal contributions round-up

Thu, 31 Jul 2025 00:00:00 +0000

TLDR: July: lots of debugging, three comments linking a few issues together, and a sandbox module that I hope will soon be obsolete.

Layout builder

I had a Drupal project the other day involving layout builder and inline blocks. One such block had a media field that non-admins weren’t able to set until the page itself (and the block) had been saved first.

You shouldn’t need permission to create blocks in order to use layout builder. Except seemingly you do if you want layout builder and media to play nice together.

There’s an open issue about it. However, the suggested workaround fudges the answer to “can I create this thing?” depending on the circumstances:

// paraphrased:
function hook_ENTITY_TYPE_create_access() {
  if (/* inside a media library route? */) {
    return TRUE;
  }
  else {
    return /* whatever normally happens */;
  }
}

I don’t like that, it feels like a second bug trying to cancel out the first.

In this project there was no actual risk in granting the editors a few extra permissions. They are trusted and there isn’t any damage they can do with it. So the pragmatic thing to do is just to grant them and forget about it.

Group

But later we came across the same issue with the group module. You can’t use a media field properly if you don’t have permission to create entities of that field’s type.

There’s an open issue about that too, with an eerily similar workaround:

// paraphrased:
function hook_ENTITY_TYPE_create_access() {
  if (/* inside a media library route? */) {
    return TRUE;
  }
  else {
    return /* whatever normally happens */;
  }
}

This time there is no easy way out: we aren’t able to just grant permissions and forget about it. We don’t want people creating things outside of their group.

Media library

I’ve tracked the bug down to the media library module, and lo and behold there is another open issue about that too. This one identifies the root cause, so any way forward is going to be a clumsy workaround.

I’ve at least been able to tie these three issues together. Hopefully that will give other people who find these issues a bit of direction and prevent them going down rabbit holes. I’ve also created a sandbox module as a temporary measure until the main issue can be resolved properly.

Adding GitLab CI to one of my contrib modules

Sun, 02 Feb 2025 00:00:00 +0000

I had a few hours free at the end of January to get GitLab CI enabled for the daterange_compact module.

When I first wrote this module, Drupal projects could opt-in to automated PHPUnit testing. This module has always had plenty of test coverage, so having tests run for every patch was so useful. Somewhere along the line something changed and the automated tests stopped running (though the tests themselves still were still fine). Nowadays, projects can have their own GitLab CI pipeline, but I’d never got around to setting that up.

The first task to fail was cspell, a spell checker for comments. I had no idea such a thing existed, but I soon found out.

Drupal Core uses US English spelling for all source code, including comments and names.

🤷 Whatever:

- * behaviour of that field formatter, and the default configuration.
+ * behavior of that field formatter, and the default configuration.

Pick your battles.

Next was the PHP code sniffer. It checks coding standards, comments, whitespace etc. I tidied up a few things here, adding some missing functions comments and fixing whitespace issues.

Sometimes, it’s ok to be ~~a bit lazy~~pragmatic:

// phpcs:ignore
private static function isSameDay($start_timestamp, $end_timestamp, $timezone): bool

The last error was more interesting. An actual test failure!

What?! These tests have been working for years, and I’ve always been dilligent to run them locally whenever I work on this project. Moreover, they still passed locally. But one of them failed on GitLab CI.

The offender was a test that makes sure we revert to a fallback date format when something isn’t available. And that fallback date format has changed in Drupal 11.1. I was still running 11.0.

So the test needed updating to match. It now expects different results depending on what version of Drupal is in use.

$expected = \Drupal::VERSION >= '11.1'
  ? 'Thu, 1 Jan 1970 - 10:00 - Thu, 1 Jan 1970 - 11:00'
  : 'Thu, 01/01/1970 - 10:00 - Thu, 01/01/1970 - 11:00';

It was good to get the CI pipeline working. As a bonus, I got to write a line of code too!

August Drupal contributions round-up

Sun, 01 Sep 2024 00:00:00 +0000

A new release of the compact date/time range formatter module, with support for optional end dates. We now support core’s datetime & timestamp fields now, not just ranges, which makes sense if you want to render “3:00pm” as “3pm”.

Dived down a rabbit hole about regions being rendered even when they are empty. This affects LocalGov Drupal too. I don’t think this is something that can be “fixed”, per-se. It’s more a case of choosing between performance and design accuracy. I’ve started a sandbox module to expose that choice.

On giving Drupal development advice

Fri, 29 Jan 2021 00:00:00 +0000

In my Drupal development and consultancy work, I often advise development teams on how best to implement something in Drupal. I was asked recently how I—perhaps as the sole Drupal specialist in the room—validate the advice I give. I hadn’t really articulated that before, so I wrote down some thoughts.

With Drupal, there are often many ways to acheive the same end. If you’re only familiar with part of it, it’s natural to look there for a solution. But you might miss out on something easier.

So first of all I’ll strive to put possible options on the table that I think might work. That’s good for getting the nitty gritty of what a client really needs; when you propose options, some will make sense, others won’t. Ideally we’ll get to a point where one option just naturally stands out from the rest.

Let’s take an example: Drupal has templates for different content types. If you want a piece of content to appear differently depending on it’s context, you might be tempted to put some branching logic into the template. That’s ok, but Drupal content types also have view modes precisely for this scenario: a full page event, or the teaser of an article. Each content type/view mode combination has it’s own template, so you can skip the extra logic (plus the requisite twig syntax knowledge).

This kind of thing makes sense when you’re aware of it, but isn’t always obvious at first. By revealing it, the team is empowered to make that choice themselves.

Beyond that, I consider these principles:

1. Is this as simple as it can be?

I’m a big believer in progressive enhancement. Start by doing the simplest thing that does the job and then enhance it according to time/budget/desire etc. On the web, progressive enhancement is often misunderstood: “must work on IE6 and no javascript is allowed!” is a common misconception. But, when you can acheive what you want by serving up HTML content at a given URL, why not start there? Add the JavaScript as an when it’s needed and makes things better. The end result is much more resilient to unexpected circumstances (and obscure browsers).

2. Is this familiar?

Any code we produce needs to be maintained. Depending on one’s experience and knowledge of a platform, it can be advantageous to stick to what is already known, and what’s used in the rest of the project. Perhaps that means using a GUI-based layout manager over custom templates. Or an atomic CSS methodology like Tailwind over a global one like BEM. Or PHP over complex SQL queries. Sometimes it’s better to eschew the most technically superior methods (or the ones we think are superior!) for familiarity and consistency.

3. Is this replaceable?

This might sound like a strange one, after all we’re trying to produce code that brings long term value. Shouldn’t we be striving to build reusable code instead? Yes, but in my experience a piece of code becomes more reliable when its broken down into small discrete pieces, boundaries and dependencies clearly defined, and the implementation ring-fenced. It’s also more testable that way. If what we’ve built does need to change, that’s a lot easier and less costly to do, or swap out for something better without the burden of much technical debt.

That’s by no means an exhaustive list, but in general they can be summarised as trying to keep things as simple as possible.

ffconf 2017

Sat, 25 Nov 2017 00:00:00 +0000

Web components meet 8-bit video games

I recently attended ffconf - a conference in Brighton for web developers. This is the first time I’ve been. The event is broad and covers general topics about the web as a whole, rather than covering the latest trends in web development. That’s what drew me to it in the first place; as someone who has been primarily involved in “backend” development but wants to diversify a bit, I wanted something that I was sure would be a worthwhile investment, rather than a gamble on a framework I may never use.

It’s held once a year at the Duke of York’s cinema in Brighton, so I didn’t have far to go. We were welcomed with delicious pastries and a classic 80’s playlist which set the bar high for what was to come.

The rest the day didn’t disappoint. I made a lot of notes during the day, and it’s been helpful to go over them afterwards and write them up here.

Big thanks to Remy and Julie for organising such a great event - I got a lot out of it.

1. Rethinking the Web Platform

James Kyle spoke on the conflicts between making progress and technical correctness when learning web development. He helped a friend create an app using React without gaining any experience with HTML or CSS first.

He felt strongly that statements of the form “you need to learn … first” were off-putting to new developers. Instead, he saw all web sites as a set of components, and advocated for getting started with a framework early on that could provide those and get a project up and running in it’s entirety, at the expense of having parts you don’t understand.

James’ argument was that we need to rethink our approach to teaching, embracing tooling early on and placing a lot more emphasis on getting a result quickly. Only then should you go back and start learning individual parts in more depth. To really learn and understand a topic you need an appreciation of where it fits within the wider picture, and an understanding of how it relates to other things.

I was intrigued by this talk. Much of it resonated with me; some things I disagreed with; overall it was thought provoking. It was a challenge to hear that as an industry it’s not easy for new developers to pick things up. But I still think we need to be careful with tooling though, particularly a large framework like React. I think that comes too soon, learning a tool before knowing what that tool does for you.

I’ve taken away some things from this talk for use when I teach things. I like to learn in a thorough, methodical, bottom-up way, usually outside of a real world project. It’s inevitable that’s also going to be reflected in how I teach. But not everyone learns that way. Many people are happier with a big picture that they can delve into from the top-down, gradually understanding more and more. The challenge is to fit in with them, not the other way round.

2. If you’re going out of San Francisco, be sure to wear Web Standards in your hair

Slides

Bruce Lawson started by looking at some of the holy wars in web development. All industries have them, and our current one seems to be CSS vs CSS-in-JS. Pure use of the fundamentals of web development (HTML, CSS & JavaScript) is often out of line with what happens in the real world. CSS for components is difficult - everything is global, the order matters, and the CSS is tightly coupled to the structure of your HTML. But CSS-in-JS is only a methodology, not a new technology, and similar things have existed before.¹

We risk locking out many great designers who understand CSS if we use a different model just for our own convenience.

There’s also a performance cost. Bruce spoke of some of the work he’s doing with Wix. Traditionally everything was built with JavaScript. That’s been ok–performance wise–on desktops, but now has real problems on mobile. They are working on a project called Stylable to address this.

Utimately though, the web isn’t about computers or clouds, it’s about people. Technology is only worth caring about because it facilitates the communication between people across the world. During the talk Bruce shared the story of his diagnosis of MS and his first use of the internet as a result. Whilst abroad he was able to get online in an internet cafe, finding forums and information about the condition. Some of the founders of those sites have become lifelong friends.

Bruce went on to give some illustrations of the digital divide. In Africa, the lack of locally relevant content and digital skills are some of the biggest barriers to internet adoption.². Cost is often prohibitive. In Nigeria, the bandwidth for a 2 minute video clip can cost more than sending a child to school for a month.³ Landlocked African countries typically incur an extra $232 to the cost of monthly broadband.⁴

The web can provide opportunities for the poor, particularly for women, older people and disabled people, who generally benefit more from flexible online working. Progressive web apps are thriving in Asia and Africa, where native apps are too big.

The web is a lifeline for people in backward and despotic regimes. But we need to be more responsible in what we do, and ensure large companies like Google, Facebook, Twitter and Amazon are held to account in the way they make profits.

As our industry professionalises, we mustn’t lose the sense of craftsmanship, and the artisan mentality of caring about the end product. This is perhaps best described by the Japanese idea of shokunin:

Shokunin means not only having technical skill, but also implies an attitude of social consciousness... a social obligation to work best for the general wefare of the people, an obligation both material and spiritual.

If you haven’t already, I’d recommend reading Bruce Lawson’s article World Wide Web, Not Wealthy Western Web.

3. How to train your dragon web standards

Monica Dinculescu talked about how web standards come to be. Web standards are a compromise between browser vendors. We need them to stop browsers copying each other’s bugs.

Not all standards have been sucessful - <input type='date'> for example. Although a popular request, it turns out date pickers are a hard thing to standardise because of the variety of ways people use them. Which day of the week to start on? How do you omit day, month or year? How do you style it?

This example hasn’t worked because it was an attempt to standardise the component rather than the component model. The date picker is too high level. Web standards should focus on low level capabilities of browsers, leaving space for libraries and frameworks to build on this. WebGL is a good example - it facilitates graphics on the web, and tools like D3.js exist because of it.

A standards process should have these properties:

it should cover something that browsers already do
it should document and explain any existing browser magic
it should be extendable
people should already use and like the feature

Monica went on to talk about one particular standard - web components. The idea behind web components was born out of a desire to reduce the amount of copying & pasting JavaScript to produce common widgets, like date pickers. Web components are an attempt to standardise ideas behind many different models, rather than any one implementation. They provide the low level functionality for something like a date picker to exist - a lifecycle and integration with the DOM.

Despite the extensible web manifesto being made in 2013, progress has been slow. It’s a catch-22: developers are hesitant to use features that browsers haven’t fully implemented, and browser vendors won’t implement things people don’t use. Polyfills (named after Polyfilla!) are the answer to this. They are–quite literally–a way to plug holes in existing browsers. We shouldn’t be scared of using them!

4. Lessons learned sciencing the web

Slides

Addy Osmani looked at using data science to help with performance on the web.

Mobile devices are resource constrained. On a 3G connection, 600ms is taken up with the network overhead. The server response adds another 200ms, only after which can client side rendering take place. So it’s cruicial to load only what you need, when you need it.

JavaScript comes with a cost due to parsing and compiling. So don’t just consider download size - 100kb of JavaScript is not the same as 100kb of JPEG. We can do a lot by removing unused code, or deferring what we don’t need for the current request and lazy loading it later. An average phone (eg Moto G4) will parse 1Mb of JavaScript in 1-2 seconds, but this can vary from almost nothing on an iPhone 8 to over 6 seconds on a low-end device.⁵

Addy explained the PRPL pattern⁶: push the minimal code for the initial route, render it and become interactive as soon as possible. Then pre cache remaining resources using a service worker and lazy load anything else. He went on to give examples of this from CNN and Pinterest.

It’s also important to examine your caching. <link rel="preload"> is a way to specify that resources should be fetched early on. We can use this for any type of resource, not just CSS and JS. There are also prefetch (loads but doesn’t parse), dns-prefetch and preconnect declarations.

These declarative ways provide hints to the browser about ways in which performance might be improved. They are a lot easier to debug and reason about than something like HTTP2 server push. Although push can be powerful when you know what resources are needed, it isn’t cache aware, so the browser may download more than it needs to.

Images qaulity can make a big difference too. q=80 is a good baseline for JPEGs.⁷ Tools like ImageOptim and mozjpeg can give you a comparison of quality levels and their respective file sizes. Addy has also written a guide to image optimisation.

If you’re using webfonts, have a loading strategy. Preload can bring a big improvement; you don’t have to waiting for the CSS to finish loading. But if you can’t load fonts quickly, don’t load them at all. The font-display property dictates how long a browser should wait for a font to load–if at all–before showing a fallback.

Finally, it’s important to introduce workflows that force everybody to think about load times from the beginning. Performance budgets are a useful constraint to have.

5. My password doesn’t work

After lunch, Blaine Cook looked at passwords.

Much of his talk originated out of frustration in helping a friend–who is a nomadic guide in the Jordan desert–set up email on his phone. There were many hoops to jump through, not least configuring an iTunes account and billing address. Why did Apple need this information for someone to download free apps? What data did they really need? And why do password restrictions seem so arbitrary?

Email addresses are so often used as a primary means of identifying someone. And if that service also uses a password, it can usually be reset via email, thereby rendering the account only as secure as the email account.

Blain gave an overview of the forgotton-password-reset-email dance that we encounter so regularly. This process is only a means of verifying that someone is who they say they are.

He advocated for third party sign-in as a good way to improve your conversion, arguing that if you’re going to provide a password reset mechanism over email, you might as well just use that email provider as your authentication provider. This can improve conversion, because there’s one less password for a user to remember.

I’m always skeptical about logging in with Google/Twitter though! I think there’s some more information needed on the possible privacy implications of doing this.

There were some pointers for web developers too:

don’t require an account at all if you can help it
if not, leave sign in as late on in the process
long lived sessions reduce the number of times users need to enter passwords

And we should all take care to keep our primary email secure. So much is accessible via it.

6. Don’t forget to take out the garbage

Katie Fenn gave a overview of garbage collection in JavaScript, and illustrated how Chrome dev tools can highlight memory usage.

Memory is a finite resource, and apps will find a way to consume it. Memory is particularly constrained on older devices.

Languages like C require manual memory management - you ask for memory by calling the malloc function and return it for reuse by calling free. Unlike C, we don’t have to do this in JavaScript. The browser keeps track of what memory are in use and can release it as required.

JavaScript scope captures variables declared with var, let or const and makes them available to the functions within that scope. Once the scope no longer exists, the variables are no longer reachable, and can be garbage collected.

The memory heap is divided into two sections - a young generation area and an older generation area. The younger generation is small and fast, for short lived, frequently used data. The older generation is for longer lasting data.

Variables start off pointing to space in the young generation segment. Once that space is full, a minor garbage collection occurs. All the unreachable variables are considered dead, and can be discarded. Anything still alive after two minor garbage collections then gets moved to the older generation. A major garbage collection marks dead objects, sweeps the memory and compacts it, freeing it up for future use.

JavaScript does a good job managing memory for us, but there are some potential pitfalls. If we aren’t careful, variables can remain referenced longer than we expect, and end up not being released. This is a memory leak, for which there are a few common causes:

accidental global variables caused by a missing var - these will have the global scope rather than the function’s scope, so they will end up living indefinitely
forgotton timers and event listeners also keep a scope alive
detached DOM nodes

Katie illustrated using the Chrome task manager and dev tools to highlight memory usage. These give you an overview of the memory footprint of each tab, and an indicator of what objects are in use.

7. Abstract art in a time of minification

Software is for humans. It’s easy to forget or ignore that sometimes. To be a good engineer, you have to be a good member of society.

Jenn Schiffer learned by viewing the source of web pages. Nowadays, that method of exploring other’s work isn’t always enough to learn how things work; a lot of what we do is hidden behind developer tools. In treating those tools as only for developers have we raised barriers to entry for people? Have we unwittingly made it more difficult to learn to code?

There’s a sense of ephemerality about what we do online now. Things we produce and the tools we use to do so are often at the mercy of startups who disappear at any moment without our consent, leaving a steam of 404s in their wake. Ownership is another issue - who really owns your data? Which companies can you trust with it and what will they use it for? Do we understand our rights and what licences like GPL and MIT give us? Can we educate people about this?

In a capitalist society, companies have an incentive to do things faster, but not always better. Computers are very good at doing this fast, but not all problems are best solved with speed. Many problems require empathy, and if we’re not careful technology can often make things worse here. Jenn gave examples of how software can introduce racial bias to law enforcement.⁸ ⁹

Like the art world, the tech community has a problem with diversity and representation. But we make this worse for ourselves when we consider problems like accessibility as “too hard to solve”. Why do we argue this? Ok, it may take a tiny bit of work to understand the difference between a div and a button, but is that not worth it to avoid leaving technical debt for those less fortunate than ourselves.

That optimistic belief that every problem can be fixed sometimes leads people making tech to think they’re the only ones who can fix things. And all of these problems are severely exacerbated by tech’s persistent failings at inclusion, which means these problems affect marginalized communities even more acutely.
Anil Dash ¹⁰

Jenn left us with a challenge: What’s your legacy? Think! talk! Widen and diversify your network, and become role models.

8. Alpha, beta, gamer: dev mode

Joe Hart, descibed as “a comedian who codes”, closed the conference with something very different.

Regaling us with tales of his unfortunate Sims characters and their various demises, through communist Tetris to a nuclear weapon-wielding Ghandi (it happens when you represent agressiveness using an unsigned int…), he then got the audience playing games of cheering controlled Flappy Birds and home drawn Super Mario.

If you like get the chance to see this show, do! It was a lot of fun, and nice way to end a thoroughly useful and enjoyable day.

Notes

Netscape 4 had a form of stylesheets called JSSS. ↩
Slide ↩
Slide ↩
Slide ↩
Slide ↩
Slide ↩
Slide ↩
Courts are using AI to sentence criminals. That must stop now - wired.com ↩
State Governments Use Racially Biased Software to Predict Crimes - gizmodo.com ↩
Software Matters in the World ↩

The map is not the territory

Mon, 06 Nov 2017 00:00:00 +0000

I recently attended an excellent workshop on OpenStreetMap led by Jez Nicholson, and hosted by the Data Visualisation Brighton meetup. Entitled The map is not the territory, we looked at maps beyond being something that’s purely visual, and delved into the data behind them.

OpenStreetMap was started in 2004¹ and is often referred to as ‘the Wikipedia of maps’. The data is openly available, owned by all contributors and moderated by peers.

I’ve used OpenStreetMap many times before, but never done any editing. During the workshop we logged in and I suddenly found myself exploring a whole new world of information!

Lots more than meets the eye

The first thing that strikes me is the vast amout more detail available than you normally see. Of course, it isn’t possible to display everything on the browseable map, but that doesn’t mean it isn’t there. What you see when browsing a slippy map like the one on openstreetmap.org is just a subset deemed the most suitable for a general audience.

Editing a feature on OpenStreetMap.

Everything is made up of shapes and tags

The format of OpenStreetMap data is relatively simple. Lines, points and polygons form the basic shapes from which all physical features are recorded. Then each of these items is has one or more tags identifying it.² These tags have evolved dynmaically over time, no-one sat down at the beginning and worked out what all the tags were going to be. I find this rather fitting for the web. In the same way HTML has grown and adapted over time, the vocabulary of OpenStreetMap–and it’s usefulness–grows as more and more people contribute to it.

During the workshop I made a small edit, marking two bus stops as having a bench and/or shelter.³ A trivial change, but nice to come away having done something real and tangible.

What about Ordnance Survey & Google?

This organic growth is where OpenStreetMap differs from more “official” sources of mapping like Ordnance Survey. These organisations aim for very high consistency and accuracy, which they acheive at greater cost and a slower feedback cycle. Commercial providers have their own goals–Google maps and street view enhance their search offering.

These alternatives are fundamentally closed entities - you can’t alter the data, for good or bad. And both of these data sets come with terms and conditions dictating how you can use them.

There’s certainly room for both though. There are times when you want the assurance of thorough, unbiased surveying that’s gone into something like an OS map. You’d certainly take that sort of map out hiking, for example. But the existence of a global, freely-available rich dataset offers up so much potential and it’s exciting to see what can be done.

Notes

History of OpenStreetMap - wiki.openstreetmap.org ↩
Map Features - wiki.openstreetmap.org ↩
Changeset 53217525 ↩

When your composer build breaks

Wed, 01 Nov 2017 00:00:00 +0000

Yesterday a project on github was moved, causing Drupal’s own build process, and that of many other websites, to “break”. Here’s what happened:

The coder library was removed from github (it’s main home is on drupal.org).
Drupal core’s composer.json had a reference to the now non-existent repository.
Anyone attempting to obtain Drupal by downloading it’s source and running composer install couldn’t, due to the broken link.
Many other developers who tried to build their websites were similarly left disappointed.

This issue on drupal.org captures the problem in more detail.

We’ve been here before

In March 2016 a JavaScript library called left-pad was removed from the npm package manager, causing the builds of many front-end projects that used it to break.¹

This seems to be a risk that comes with dependency management, and raises the question of should vendor be committed to version control? I’m hoping that this post will help you answer that.

Notice that I’m only talking about full applications or website codebases here, not libraries. If you’re working on some sort of standalone component for use within a larger project (like a contrib module), you definitely don’t want to ship vendor to your users. Nor do you want to include composer.lock.

A lean codebase

The keep-vendor-outside-git argument favours a lean codebase. There’s some merit to this. After all, upgrading a module is often considered a single, atomic change, and it’s nice when a pull request of the form “Upgrade the EVA module to 1.2” comes down to a single line: ²

diff --git a/composer.lock b/composer.lock
index 378e3be3fa..8f6d7d31a9 100644
--- a/composer.lock
+++ b/composer.lock
@@ -637,17 +637,17 @@
         },
         {
             "name": "drupal/eva",
-            "version": "1.1.0"
+            "version": "1.2.0"
         },

It’s not hard to cross reference that with a changelog in the other project, should you need to. And by downloading the code each time prevents you from modifying a module without auditing it. Every patches is explicitly listed.

It’s interesting that composer’s own documentation recommends this approach.³

Resilience

The commit-vendor-to-git argument focuses on resilience. You don’t want your build process to be dependent on external services that may or may not be available. Furthermore, you want to be able to track all changes to your source code in one place, both the bespoke code and any other libraries.

Pascal Morin articulates this well here: Do you really need composer in production? The CocoaPods dependency manager for iOS also leans towards this position.⁴

Can we have our cake and eat it?

We’re looking at two objectives - the resilience of not being dependent on packagist/github/drupal.org for building, plus the advantages that come with a lean codebase. By trying to achieve both, are we trying to have our cake and eat it?

In an ideal world, each project would have some kind of artefact repository. One that’s under your control and from where you can obtain every library/version combination you’ve ever used in the project. This is what Maven, a Java dependency management tool, suggests.⁵

Do I think the core Drupal repository should contain a vendor directory? No, I don’t. I think it’s Drupal’s job to be a component, augmented with contributed modules and perhaps custom code. Every Drupal website is different; as soon as you add a contributed module the contents of vendor change.

But unless you have the resources to manage an artefact repository, your website’s repository probably should contain vendor. You can still have the benefits that composer brings, but why introduce another point of failure at build time?

Notes

How one developer just broke Node, Babel and thousands of projects in 11 lines of JavaScript - The Register ↩
In reality, composer.lock will have a few other changes, like download URLs and hashes corresponding to the version. ↩
Should I commit the dependencies in my vendor directory? - getcomposer.org ↩
Using CocoaPods - guides.cocoapods.org ↩
Best Practice - Using a Repository Manager - maven.apache.org ↩

DrupalCamp Bristol 2017

Fri, 07 Jul 2017 00:00:00 +0000

Drupal has a thriving community in Bristol and the south-west, and they’ve put on some excellent events over the last few years. Last weekend they had their third DrupalCamp Bristol, and I was fortunate to be able to attend and speak.

The day opened with a keynote by Emma Karayiannis on self care and supporting others within open source communities.

Emma shared some of her contributions to Drupal, where she is part of the Community Working Group and track chair for the Being Human sessions at DrupalCon.

Look after yourself. Don’t feel that you can only contribute a little bit or that your opinion doesn’t matter. Just find something rewarding and start small. Getting involved can be daunting, so get to know the part of the community that deals with your interest, ask how to get involved and ask for someone to help you.

Burnout is real, and happens much more when we work alone, taking on lots of responsibility without anyone to partner with. So look for someone to co-work or co-lead with rather than try to be a lone superhero. It gives you the freedom to step away if necessary. Ask yourself: if I had to stop this tomorrow, what would happen?

It’s easy to become overwhelmed without realising it. You need to regularly check you’re looking after yourself, are still motivated, and aren’t taking on too much. Be accountable to your family, colleagues and friends, and step back if necessary.

Look after others. It’s healthy for an open source community to have people who think differently to you. Be respectful of other people and aware that miscommunication is very easy online, particularly with people whose native language is different to yours. But also accept that you’ll never be able to make everyone happy.

Make sure the people are really ok even if they appear fine. Experienced contributors, remember that you were once a beginner, and provide opportunities and safe spaces to include new people. Appreciate people for who they are and not just the work they do.

After a short break we split into two tracks.

Deji Akala provided an interesting look into the technical details on what happens on each page request. Along the way he summarised various parts of Drupal and concepts such as the autoloader, symfony handlers, the service container and event handling.

It’s an interesting exercise to unpick the index.php file line by line and discover what happens behind the scene in a single line of code:

$response = $kernel->handleRequest();

I then gave a short talk about Composer and Drupal. I’ve spoken to a number of people recently and it’s become clear that there’s still a bit of confusion surrounding how to use Composer with Drupal. I certainly found it unclear and started to look into it.

I pitched this at beginner developers, and the main things I wanted people to go away understanding were:

what the require, update and install commands really do
the difference between Drupal itself and the various template projects available

That was the first time I’d given that talk. It felt a bit raw but led to some interesting Q&A time, and it’s given me valuable insight for enhancing this in future.

To others contemplating public speaking - do it! Events like this are an ideal place to start, everyone’s friendly and on your side. You’ll gain knowledge, experience and friends from doing it. I was really glad to see that several of the speakers here were first timers - well done!

Ross Gratton shared some insights into using front end task runners like Gulp with Drupal.

Ross has been working on a large Drupal site utilising several themes, in 24 languages and with over 125 custom modules. He discussed the pros and cons of different architectural decisions, such as where to put source code and assets, what to put in version control and how to manage conflicts on such a large site.

He then shared some of the process of separating assets out to the brand level as opposed to a project level, treating a style guide or pattern library as a separate deliverable.

After lunch, George Boobyer spoke on web security, a topic often overlooked in the planning and budgeting of projects.

Security is perceived as complex but isn’t that hard, and any effort you make is rewarded. Recently we’ve seen a lot of ransomware attacks, but often these just have the same impact as a disk failure, so alongside keeping software up to date, have backups and test them.

George gave some examples of websites that had been attacked and were now hosting spam content, very often not visible to the naked eye but only to search engines. Often user data is obtained by way of database dumps that have been left accessible to the world - don’t put these in the document root.

Ana Hassel shared some insights as a freelancer. As a site builder, Ana has been able to use Drupal to focus on her clients’ needs and come up with a repeatable process for estimating and selling her work.

Ana also shared how she had invested some time learning the learning the command line and setting up scripts for everyday tasks. This had given her a better, more repeatable workflow and more predictable deployment and hosting.

An interesting perspective on personal development came from Johan Gant. I felt it complemented Emma’s keynote well with some recurring themes, and gave the day a nice mix of technical and human elements.

Johan covered issues such as Imposter syndrome, depression and burnout. Burnout often comes from a lack of engagement, and seems to be a particular risk for knowledge workers. If the values you have aren’t aligned with those of your employer or project, you can burn out very quickly.

Be selective about what you learn—patterns and techniques will last for a long time, whereas frameworks come and go. You need to make time to explore new things, but make sure you are following your interests rather than trends. Avoid stagnation—ask yourself if what you’re doing is satisfying. It’s healthy to seek new challenges, but means getting out of your comfort zone.

Lee Stone finished by sharing about how his organisation does extensive code reviews.

As well as preventing bugs, code reviews aid in training. New developers can learn the business by reading code, and junior developers can grow by asking why something is the way it is, or by asking about things they don’t understand. They often bring fresh ideas this way.

It’s important to review the code, not the person writing it. So don’t make these things too personal, and don’t take them personally! Prefer terms like “we” rather than “I” and “you” to foster a sense of team, and provide solutions rather than just stating something’s wrong.

After the talks we headed to ZeroDegrees in Bristol for a social time. It was great to catch up over dinner with people I hadn’t seen for a while, and make some new friends.

Thanks to everyone who helped make DrupalCamp Bristol such a great event. See you next year!

Nicer date ranges in Drupal – part 3

Mon, 05 Jun 2017 00:00:00 +0000

This is the last part of a series on improving the way date ranges are presented in Drupal, by creating a field formatter that can omit the day, month or year where appropriate, displaying the date ranges in a nicer, more compact form:

24–25 January 2017
29 January–3 February 2017
9:00am–4:30pm, 1 April 2017

The first post, looked at porting some existing code from Drupal 7 to Drupal 8, adding an automated test along the way. In the second post, we made the format configurable.

There’s currently no administrative interface though, so site builders can’t add and edit formats from Drupal’s UI. We’ll add that in this last post.

Routing

According to the routing overview on drupal.org, a route is a path which is defined for Drupal to return some sort of content on.

For our administrative interface, we want to define a number of routes:

/admin/config/regional/date_range_format - show a list of the formats, with links to:
/admin/config/regional/date_range_format/add
/admin/config/regional/date_range_format/*/edit
/admin/config/regional/date_range_format/*/delete

There are two ways in which our module can provide routes. We could include a routing.yml file along with our module. This file contains the same kind of information as would have been in hook_menu in Drupal 7. But it’s a static file—if we want something that’s dynamic we can provide it at runtime using a route provider.

For dealing with entities, it’s often much easier to use Drupal’s bundled AdminHtmlRouteProvider class. This examines various properties on the entity annotation—we’ll look at those next—and provides suitable routes for us automatically.

To use this route provider, we add the following to the entity annotation:

@ConfigEntityType(
  …
  handlers = {
    "route_provider" = {
      "html" = "Drupal\Core\Entity\Routing\AdminHtmlRouteProvider",
    },
  },
  …
)

At this point we need to run the drupal router:rebuild command from Drupal console. We must do this whenever we change a routing.yml file or any of the properties in the entity that affect routes.

The collection view

An entity can define a collection view—typically a page showing a list of entities with links to edit them. Drupal provides a list builder which can be used to show a list of entities with buttons for common add/edit/delete type tasks. We’ll create one of these for our new configuration entity:

<?php
namespace Drupal\daterange_compact;

class DateRangeFormatListBuilder extends ConfigEntityListBuilder {
  function buildHeader() {
    /* return an array of column headings */
  }
  function buildRow(EntityInterface $entity) {
    /* return an array of column values for the given entity */
  }
}

We then associate this list builder with our entity by declaring it within the @ConfigEntityType annotation:

handlers = {
  "list_builder" = "Drupal\daterange_compact\DateRangeFormatListBuilder",
}

The actual list builder is quite a rich, showing examples of different ranges. You can see the full implementation here.

The collection page

Once we have the list builder in place, we can add the collection link to our @ConfigEntityType annotation. The route provider will pick up on this link template and provide a route for the entity collection page automatically.

links = {
  "collection" = "/admin/config/regional/date_range_format"
}

By defining the link, our page appears at the appropriate URL. Note that the add/edit/delete links won’t show just yet—we still have to define those.

The screen for listing date/time range formats, provided by the entity list builder.

Updating the main configuration page

In order to reach this new page, we’ll create a menu link on the main configuration page, within the regional and language section. We do that by supplying a daterange_compact.links.menu.yml file:

entity.date_range_format.collection:
  title: 'Date and time range formats'
  route_name: entity.date_range_format.collection
  description: 'Configure how date and time ranges are displayed.'
  parent: system.admin_config_regional
  weight: 0

That link gives us the starting point for our interface:

Date/time range formats are accessed via the main configuration page.

We can now view all the date and time range formats from the main administrative interface in Drupal. Next we’ll build some forms to maintan them, after which the add/edit/delete links should start to appear on our collection page.

Forms

We need a form to be able to edit date range formats. The same form is used to create new ones. Drupal provides a lot of built-in functionality via the EntityForm class which we can extend. Drupal will then take care of loading and saving the entity. We just need to provide the form elements to map values on to our entity’s properties.

Adding & editing

We can add any number of forms, but we only need one to edit an existing format, and we can reuse the same form for adding a new format. This form is defined as a class, and lives in src/Form/DateRangeFormatForm.php:

<?php
namespace Drupal\daterange_compact\Form;

class DateRangeFormatForm extends EntityForm {
  /* implementation */
}

Configuration entities don’t use the field API, so we need to build the form ourselves. Although the form looks quite complicated and has a lot of options, it’s reasonably easy to build—each property in the configuration entity can be populated by a single element, like this:

$form['label'] = [
  '#type' => 'textfield',
  '#title' => $this->t('Label'),
  '#maxlength' => 255,
  '#default_value' => $this->entity->label(),
  '#description' => $this->t("Name of the date time range format."),
  '#required' => TRUE,
];

The full implementation of the form is here.

We also need to tell Drupal about this form, which we can do by adding the following to the @ConfigEntityType annotation:

"form" = {
  "add" = "Drupal\daterange_compact\Form\DateRangeFormatForm",
  "edit" = "Drupal\daterange_compact\Form\DateRangeFormatForm",
}

We also add some links, to match up operations such as add and edit with the new form. These are also defined in the @ConfigEntityType annotation:

links = {
  "add-form" = "/admin/config/regional/date_range_format/add",
  "edit-form" = "/admin/config/regional/date_range_format/{date_range_format}/edit",
}

If we look at the collection view again we see that alongside each format there is a link to edit it. That is because of the edit-form link declared in the annotation.

We also want a link at the top of that page, to add a new format. We can do that by providing an action link that refers to the add-form link. This belongs in the daterange_compact.links.action.yml file:

entity.date_range_format.add_form:
  route_name: 'entity.date_range_format.add_form'
  title: 'Add format'
  appears_on:
    - entity.date_range_format.collection

At this point we have a means of adding and editing formats. Our form looks like this:

The screen for editing date/time range formats.

Deletion

Deleting entities is slightly different. We want to show a confirmation page after a before performing the actual deletion. The EntityDeleteForm class does just that. All we need to do is subclass it and provide the wording for the question:

<?php
namespace Drupal\daterange_compact\Form;

class DateRangeFormatDeleteForm extends EntityDeleteForm {
  public function getQuestion() {
    return $this->t('Are you sure?');
  }
}

We declare this form and link on the @ConfigEntityType annotation in the same way as for add/edit:

"form" = {
  "delete" = "Drupal\foo\Form\DateRangeFormatDeleteForm"
}
links = {
  "delete-form" = "/admin/config/regional/date_range_format/{date_range_format}/delete",
}

Conclusion

That’s it. We’ve got a field formatter to render date and time ranges in a very flexible way. Users can define their own formats thorough the web interface, and these are represented as configuration entities, giving us all the benefits of the configuration management initiative, such as predictable deployments and multilingual support.

The module is available at https://www.drupal.org/project/daterange_compact.

I hope you found this write-up useful.

Want to help?

I’m currently working on getting this module up to scratch in order to have coverage from the Drupal security team. If you want to help make that happen, please review the code following this process and leave a comment on this issue. Thanks :-)

Nicer date ranges in Drupal – part 2

Wed, 03 May 2017 00:00:00 +0000

This is the second part of a series on improving the way date ranges are presented in Drupal, by creating a field formatter that can omit the day, month or year where appropriate, displaying the date ranges in a nicer, more compact form, e.g.:

24–25 January 2017
29 January–3 February 2017
9:00am–4:30pm, 1 April 2017

The first post dealt with porting some existing code from Drupal 7 to Drupal 8, adding an automated test along the way.

In this post, we’ll do some of the work to make the the format customisable:

a new config entity to store formats
moving the rendering logic into a service

This is a long post, and we’ll cover a lot of ground. You may want to make a cup of tea before we start.

Configurable formats

Drupal’s core date formats are stored as configuration entities. Each one consists of a single pattern made up of PHP’s date & time formatting symbols. We’ll create another type of configuration entity for date range formats. Each format will need several different patterns:

a default date pattern. This is used to show the full single date if the start and end dates are the same. It’s also used to show both dates fully where the range spans several years.
patterns for the start and end dates, where a range spans several days within the same month
patterns for the start and end dates, where a range spans several months within the same year

We can also support times in a similar fashion:

a default pattern. This is used to show the full single date & time if the start and end values are the same. It’s also used to show both dates & times fully if the range spans several days.
optional patterns for the start and end values, where we have a range contained within a single day

All of these 8 patterns can be stored within a single configuration entity, alongside custom separator text.

Defining a configuration entity

To create the configuration entity, we need some custom code. We can use Drupal console to generate a certain amount of boilerplate code as a starting point. Be aware that Drupal console will produce a lot of code all in one go, including the admin interface which we’ll look at in part 3.

I find it helpful to generate this boilerplate into a temporary location, then copy the files over one at a time, editing them as I go. It slows things down in a good way, and forces me to understand the code I’m going to be responsible for maintaining.

Entity class and definition

Let’s start with the entity class, which lives in src/Entity/DateRangeFormat.php:

<?php
namespace Drupal\daterange_compact\Entity;

class DateRangeFormat extends ConfigEntityBase implements DateRangeFormatInterface {
  /* implementation */
}

In order to tell Drupal that this is a configuration entity, we need to add an annotation to the class. Similar to the annotation on the field formatter in part 1, we’re telling Drupal about the existence of a new date range format configuration entity, plus some information about it.

/**
 * @ConfigEntityType(
 *   id = "date_range_format",
 *   label = @Translation("Date range format"),
 *   config_prefix = "date_range_format",
 *   entity_keys = {
 *     "id" = "id",
 *     "label" = "label",
 *     "uuid" = "uuid"
 *   }
 * )
 */
class DateRangeFormat...

The class adheres to a corresponding DateRangeFormatInterface. We’ll refer to the interface, rather than the entity class directly, elsewhere in the code.

The complete implementation is here.

Schema

End users will create instances of the configuration entity—one per format. These can be represented as a single YAML file, and imported and exported as such. The schema describes the structure of these YAML files, dictating the type of data we’re storing inside the entity.

The schema definition is itself a YAML file, and lives in config/schema/daterange_compact.schema.yml:

daterange_compact.date_range_format.*:
  type: config_entity
  label: 'Date range format config'
  mapping:
    # properties of the config entity

The complete schema implementation is here.

Updating entity definitions

If we look at the status report of our site, we’ll see that there is an error: Mismatched entity and/or field definitions. Each time we add or change code that defines entities, we need to update Drupal’s internal copy of the definitions. We can do this with the drush entup command, and we should get the following output:

The following updates are pending:
date_range_format entity type :
  The Date range format entity type needs to be installed.

Do you wish to run all pending updates? (y/n): y
 [success] Cache rebuild complete.
 [success] Finished performing updates.

That’s the bare minimum for defining a config entity. Right now the only way to manage them is by editing YAML files by hand, but that’s enough to start working on the improved functionality for now. In the next post we’ll look at an administrative interface for editing these formats.

Providing a default entity

Any Drupal module can provide configuration entities as part of their install process. The contact module is a good example—enabling the module will create the feedback and personal contact forms, each configuration entities. You can then change those forms, remove them or add new ones.

Let’s make our module provide a date range format called Medium, following the same naming convention as Drupal’s standard date formats. We do that by providing a file called {$modulename}.{$config_entity_type}.{$machine_name}.yml in the config/install directory.

So the module will contain a config/install/daterange_compact.date_range_format.medium.yml file that looks like the following. You’ll see the properties follow the schema defined earlier. The patterns are made up of PHP date & time formatting symbols.

langcode: en
status: true
dependencies: {  }
id: medium
label: Medium
date_settings:
  default_pattern: 'j F Y'
  separator: ' - '
  same_month_start_pattern: j
  same_month_end_pattern: 'j F Y'
  same_year_start_pattern: 'j F'
  same_year_end_pattern: 'j F Y'
datetime_settings:
  default_pattern: 'j F Y H:i'
  separator: ' - '
  same_day_start_pattern: 'j F Y H:i'
  same_day_end_pattern: 'H:i'

We’ll need to re-install the module for this to take effect, but if we do, and then export the site configuration, we should get a copy of this YAML file along with the rest of the configuration.

A note on cacheability

Normally, whenever we produce some sort of output in Drupal 8, we need to provide its cacheability metdata, which describes how it may be cached. Whenever something changes, the render cache can be examined and anything that depended on it can be cleared.

Certain items, like date formats, are so widely used they a treated differently. From drupal.org:

The DateFormat config entity type entity type affects rendered content all over the place: it’s used pretty much everywhere. It seems appropriate in this case to not set cache tags that bubble up, but to just clear the entire render cache. Especially because it hardly ever changes: it’s a set-and-forget thing.

Invalidating the rendered cache tag should be done sparingly. It’s a very expensive thing to do, clearing the entire render cache. But it’s appropriate here to follow what the core date format entity does.

We need to add this to the @ConfigEntity annotation:

list_cache_tags = { "rendered" }

and this to the DateRangeFormat class:

public function getCacheTagsToInvalidate() {
  return ['rendered'];
}

Refactoring

Until now, the code for rendering date ranges has been part of the field formatter. As it’s getting more complicated, it makes sense to move it out of there into it’s own, distinct location. That means we’ll be able to use it outside of the context of a field.

We’ll use a Drupal service for this. A service is a separate class in which we can handle the business logic independently of field formatters. When Drupal manages a request, it will take care of creating an instance of the service class and making it available to other parts of the system.

Our service class it quite straightforward:

<?php
namespace Drupal\daterange_compact;

class DateRangeFormatter implements DateRangeFormatterInterface {

  function __construct(…) {
    /* implementation */
  }

  function formatDateRange($start_timestamp, $end_timestamp, $type = 'medium', …) {
    /* implementation */
  }

  function formatDateTimeRange($start_timestamp, $end_timestamp, $type = 'medium', …) {
  /* implementation */
  }

}

The complete implementation of DateRangeFormatter is here.

We then tell Drupal about the service by including it in the module’s daterange_compact.services.yml file. We can also specify dependencies on other services, and these will be passed to our object’s constructor.

services:
  daterange_compact.date_range.formatter:
    class: Drupal\daterange_compact\DateRangeFormatter
    arguments: ['@entity_type.manager', '@date.formatter']

Now our formatting functions are available to use within other parts of Drupal via the daterange_compact.date_range.formatter service. We’ll access it from the field formatter next.

You can find more documentation about Drupal 8 services on drupal.org.

Pulling it all together

We need to revisit the field formatter and make a couple of changes. First we remove the hardcoded formatting logic that was there previously, and instead delegate that work to the service. Second, we need a way to let the site builder choose a particular format.

Dependency injection

The field formatter needs to be able access to the new service. Drupal 8 makes use of dependency injection for this sort of thing—a way to access dependencies at runtime without being tied to any particular implementation.

There are a few steps involved in getting to use our service this way:

First, we want it for the lifetime of the field formatter, so it needs to be in the constructor. The constructor for FieldFormatterBase takes quite a lot of parameters. We need to accept them all as well, plus our formatter. Some of the other parameters are hidden with … here for clarity.

function __construct(…, DateRangeFormatterInterface $date_range_formatter) {
  parent::__construct(…);

  $this->dateRangeFormatter = $date_range_formatter;
}

Next, we need to state that this formatter makes use of dependency injection. We do that by making the class implement ContainerFactoryPluginInterface, which declares a create function that should be used to create instances. The create function is passed the container, an object from which we can get services by name. In the create function we get the service by name and pass it to the constructor:

static function create(ContainerInterface $container, …) {
  return new static(…,
    $container->get('daterange_compact.date_range.formatter')
);

Now we will always have access to a formatter via the $this-dateRangeFormatter variable.

Field formatter settings

Whenever we use this formatter, we can choose a particular date range format. We store that choice in the field formatter settings, once for each time a field is displayed.

Adding field formatter settings is documented quite thoroughly on drupal.org, but it involves a YAML file describing the type of data we want to store, some default settings, a form and a summary.

The YAML file is named field.formatter.settings.{$formatter_name}:

field.formatter.settings.daterange_compact:
  type: mapping
  label: 'Date/time range compact display format settings'
  mapping:
    format_type:
      type: string
      label: 'Date/time range format'

The extra functions we need to implement in the DateRangeCompactFormatter class are as follows:

public static function defaultSettings() {
  /* an array of default values for the settings */
}

public function settingsForm(array $form, FormStateInterface $form_state) {
  /* form from which to choose from a list of formats */
}

public function settingsSummary() {
  /* text describing what format will be used */
}

Now when anyone opts to use the compact formatter to render a date range field, they will be prompted to choose a date range format.

Display

Finally, we have everything we need to render the date range using the chosen format. We can change the viewElements function, removing the hardcoded stuff we had before, and delegating to our date range formatter service:

$format = $settings['format_type'];
$formatter = $this->dateRangeFormatter;
$output = $formatter->formatDate($start_timestamp, $end_timestamp, $format, …);

The complete implementation of the formatter is here

Test it!

We’ve added substantial functionality, so we need to make sure there have been no regressions on what we had before. We also want to test the new configurable formats.

The test should pass as before, with one small tweak. In the setUp function, we need to load the configuration for the daterange_compact module, so that the medium format is present.

We’ll also define a new usa format, for US-style month, day, year display. That is created in the setUp function:

protected function setUp() {
  parent::setUp();
  /* existing set up code */
  /* create a new date range format called "usa" */
}

We’ll add another test specifically for rendering the USA format:

function testUSAFormats() {
  $all_data = [
  ['start' => '2017-01-01', 'end' => '2017-01-01', 'expected' => 'Jan 1, 2017'],
  ['start' => '2017-01-02', 'end' => '2017-01-03', 'expected' => 'Jan 2–3, 2017'],
  ['start' => '2017-01-04', 'end' => '2017-02-05', 'expected' => 'Jan 4–Feb 5, 2017'],
  ['start' => '2017-01-06', 'end' => '2018-02-07', 'expected' => 'Jan 6, 2017–Feb 7, 2018'],
];

  foreach ($all_data as $data) {
    /* 1. programmatically create an entity and populate start/end dates */
    /* 2. programmatically render the entity */
    /* 3. assert that the output contains the expected text */
  }
}

Whilst the goal should be to have as much test coverage as possible, is isn’t feasible to cover every combination of dates, formats and settings. But we should try to test lots of variations of date and datetime ranges, edge cases, possible formats and results that would vary by timezone. And if something doesn’t behave as expected later, we can write a test to demonstrate it before changing code. Later we can verify any fixes via the new test, and make sure there are no other regressions too!

You can find the full test implementation here.

Phew, that was a lot!

In the final post, we’ll provide an admin interface for editing the date range formats and look at some implications of using this in a multilingual environment.