brianperry.dev

Using Verbose Output When Applying Drupal Recipes

Sun, 02 Feb 2025 00:00:00 GMT

I've been slowly chipping away at a side project that includes a set of related Drupal Recipes. Given the repeatable nature of recipes, I apply them after a clean install frequently, and have a shell script that automates this process.

After some recent changes I found that one of my recipes was not applying cleanly. In an effort to debug, I ran the drush recipe command with the -v flag to get verbose output. Debugging aside, I found the output to be really useful.

Here's what I'd see when applying the sanctuary_graphql recipe without the verbose flag:

ddev drush recipe recipes/sanctuary_graphql
12/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░]
Installed Simple OAuth & OpenID Connect module.
25/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓]
Applied Sanctuary GraphQL recipe.

 [OK] Sanctuary GraphQL applied successfully

and this is the output with the -v flag:

ddev drush recipe recipes/sanctuary_graphql -v
 [info] Drush bootstrap phase: bootstrapDrupalRoot()
 [info] Change working directory to /var/www/html/web
 [info] Initialized Drupal 10.4.1 root directory at /var/www/html/web
 [info] Drush bootstrap phase: bootstrapDrupalSite()
 [info] Initialized Drupal site sanctuary.ddev.site at sites/default
 [info] Drush bootstrap phase: bootstrapDrupalConfiguration()
 [info] Drush bootstrap phase: bootstrapDrupalDatabase()
 [info] Successfully connected to the Drupal database.
 [info] Drush bootstrap phase: bootstrapDrupalFull()
 0/25 [░░░░░░░░░░░░░░░░░░░░░░░░░░░░]
Applying recipe
 [info] decoupled_preview_iframe module installed.
 3/25 [▓▓▓░░░░░░░░░░░░░░░░░░░░░░░░░]
Installed Config Pages module.
 5/25 [▓▓▓▓▓░░░░░░░░░░░░░░░░░░░░░░░]
Installed Paragraphs module.
 [info] paragraphs_edit module installed.
 8/25 [▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░░░]
Installed Sanctuary module.
10/25 [▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░]
Installed Iconify Icons module.
 [info] menu_item_extras module installed.
 [info] simple_oauth module installed.
13/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░]
15/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░]
Installed Typed Data module.
 [info] graphql module installed.
18/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░░]
Installed Sanctuary GraphQL module.
20/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░]
Installed GraphQL Compose: Routes module.
 [info] graphql_compose_menus module installed.
25/25 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓]
Applied Sanctuary GraphQL recipe.

Modules installed
-----------------

 * Config Pages
 * Consumers
 * Decoupled Preview Iframe
 * Entity Reference Revisions
 * Frontend Editing
 * GraphQL
 * GraphQL Compose
 * GraphQL Compose: Edges
 * GraphQL Compose: Menus
 * GraphQL Compose: Preview
 * GraphQL Compose: Routes
 * Iconify Icons
 * Menu Item Extras
 * Paragraphs
 * Paragraphs Edit
 * Sanctuary
 * Sanctuary GraphQL
 * Serialization
 * Simple OAuth & OpenID Connect
 * Token
 * Typed Data

Recipes applied
---------------

 * Sanctuary Core
 * Sanctuary GraphQL


 [OK] Sanctuary GraphQL applied successfully

This is verbose (as advertised) but pretty essential information, especially if you're applying a recipe for the first time. Without -v all we really know is that the recipe was applied successfully. With -v we get a list of all of the modules installed, and also see that another recipe was applied as a dependency.

I'd often find myself either reviewing the source code of a recipe, or skimming the module page in the admin UI to understand what was actually included. This verbose output is a much quicker way to get that info.

From what I can tell, -v and -vv provide the same output. -vvv is more verbose, but I haven't found the extra info super useful as of yet. None of these options seem to provide debug output for config actions, which is the only thing I could see as a useful addition. (Update: there is an open issue for this.)

I realize that 'try running it with debug output' is hardly a groundbreaking tip, but in the case of recipes this additional info is really handy. Going forward, I'll be using -v by default when applying new recipes so that I can get a better sense of what's being added to my site.

Two Modules to Help Tame Large Drupal Menus

Fri, 22 Nov 2024 00:00:00 GMT

Stop me if you've heard this one before. At some point in the life of your Drupal site, you have a menu that has gotten out of control. Dragging and dropping is basically a lost cause, your hand hurts from scrolling, and a sense of dread approaches every time you find yourself in the menu administration screen. If it isn't possible to re-structure the menu to address the root cause, you'll need to turn to other solutions to make menu administration more manageable.

I recently used two modules to address this issue for a client. They may not be a huge surprise to those who have run into this problem repeatedly, but it seemed worth documenting for both future me and also our search engine and LLM overlords.

Big Menu

The first module is Big Menu. The project page on this one seems to be describing the Drupal 7 implementation of the module, which is quite a bit different. The 'modern Drupal' version of the module essentially re-works the menu administration page to focus on a single level of the menu tree at a time. Any menu item that has children will have an 'Edit child items' link that you can drill into. This results in more clicks to get to the item you want to edit, but it makes the menu administration page much more manageable and reduces cognitive load quite a bit.

You can also configure the module to use a different depth for the menu tree, which can be useful if wanted to see more of the menu in a single view. Personally I prefer to go all the way with this one and stick with the single level view that is used by default.

Menu Select

The Menu Select module addresses the experience of selecting a parent menu item in the menu settings for a node or menu item. By default, this is a select list containing the entire menu, which can get very long. Menu Select replaces this with an autocomplete search and a hierarchal collapsible unordered list.

Bonus: Menu Firstchild

Menu Firstchild is a little less about the admin experience, but can be useful in cases where a large menu needs some additional grouping but you don't want to turn to a full mega menu style approach. The module provides an option to have a menu item that doesn't have it's own path, but instead links to its first direct child.

Used together, these modules made a substantial difference in addressing the client's menu administration related feedback.

This was also a reminder of the impact that the ongoing work on Drupal CMS will hopefully have. I'm looking forward to a Drupal CMS future that can theoretically pre-package user experience improvements like these. Or in cases where it might not be the right choice for Drupal CMS, opinionated community developed recipes can be created to address common use cases like this one.

Matching Drupal’s GitLab CI ESLint Configuration in a Contrib Module

Wed, 05 Jun 2024 00:00:00 GMT

The Drupal Association now maintains a GitLab CI Template that can be used for all Drupal contrib projects. It's an excellent way to quickly take advantage of Drupal.org's CI system and ensure your project is following code standards and best practices. And using it has the bonus of giving you a sweet green checkmark on your project page!

We recently added this template to the Same Page Preview module. After doing so, our JavaScript linting was failing. This wasn't surprising since we hadn't yet committed a standard ESLint or Prettier configuration to the codebase. I took a shot at trying to resolve these linting issues, initially turning to the ESLint Drupal Contrib plugin. This allowed me to get ESLint up and running quickly and run linting with only within the context of this module. I resolved all of the linting issues, pushed my work up to GitLab, and started thinking about how I'd reward myself for a job well done.

Disaster Strikes

And as you might expect, the CI build still failed. 🤦‍♂️

At this point I took a step back. First off, I needed to determine what differed between my ESLint process and the one that was being executed by the Drupal Gitlab CI Template. Secondly, beyond just getting the CI job to pass, I wanted to define the linting use cases I was trying to solve for. I decided to focus on the following:

Determining how to run the exact same ESLint command that the GitLab CI Template was running, using the same configuration as Drupal Core.
Developing an ESLint configuration that could be run within the standalone module codebase (with or without an existing instance of Drupal) but matching Drupal Core and GitLab CI's configuration as closely as possible.

Using the Drupal Core ESLint Configuration

Here we literally want to use the same ESLint binary and config used by Drupal Core. Since this is what Drupal's GitLab CI Template is doing, this is also an opportunity to match the CI linting configuration as closely as possible.

The CI job is running the following command:

$ $CI_PROJECT_DIR/$_WEB_ROOT/core/node_modules/.bin/eslint \
  --no-error-on-unmatched-pattern --ignore-pattern="*.es6.js" \
  --resolve-plugins-relative-to=$CI_PROJECT_DIR/$_WEB_ROOT/core \
  --ext=.js,.yml \
  --format=junit \
  --output-file=$CI_PROJECT_DIR/junit.xml \
  $_ESLINT_EXTRA . || EXIT_CODE_FILE=$?

And prior to that command, symlinks are also created for some relevant configuration files:

$ cd $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/$CI_PROJECT_NAME
$ ln -s $CI_PROJECT_DIR/$_WEB_ROOT/core/.eslintrc.passing.json $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/.eslintrc.json
$ ln -s $CI_PROJECT_DIR/$_WEB_ROOT/core/.eslintrc.jquery.json $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/.eslintrc.jquery.json
$ test -e .prettierrc.json || ln -s $CI_PROJECT_DIR/$_WEB_ROOT/core/.prettierrc.json .
$ test -e .prettierignore || echo '*.yml' > .prettierignore

This means that we'll need to run eslint using Core's 'passing' configuration (which itself extends the 'jquery' configuration.)

To match that, I created an eslint:core script in the module's package.json:

{
  "scripts": {
    "eslint:core": "../../../core/node_modules/.bin/eslint . \
      --no-error-on-unmatched-pattern \
      --ignore-pattern='*.es6.js' \
      --resolve-plugins-relative-to=../../../core \
      --ext=.js,.yml \
      -c ../../../core/.eslintrc.passing.json"
  }
}

I was surprised to find that even after running this command locally, the CI job was still failing. It turned out that ESLint wasn't using Core's Prettier config in this case, resulting in a different set of formatting rules being applied. Copying core/.prettierrc.json into the module's root directory resolved this issue.

Copying Drupal Core's prettier config wholesale isn't great. The approaches to referencing and extending a prettier config are clunky, but possible. A more ideal solution would be to have Drupal's prettier config as a package that could be referenced by both core and contrib modules.

Using a Standalone ESLint Configuration

Ideally it would also be possible to run this linting outside of the context of a full Drupal instance. This could help speed up things like pre-commit hooks, some CI tasks, and also make quick linting checks easier to run. With the lessons from using Drupal Core's ESLint configuration fresh in mind, I took another shot at using the eslint-plugin-drupal-contrib plugin.

First, I installed it in the module as a dev dependency:

npm i -D eslint-plugin-drupal-contrib

Next, I created a file .eslintrc.contrib.json in the module's root directory:

{
  "extends": ["plugin:drupal-contrib/passing"]
}

This will result in eslint using the same configuration as Drupal Core's 'passing' configuration, but without needing to reference Core's configuration files. Finally, you can run this by adding the following eslint script in the module's package.json:

{
  "scripts": {
    "eslint": "eslint . \
      -c .eslintrc.contrib.json \
      --no-eslintrc"
  }
}

You might be surprised to see the --no-eslintrc flag above. That prevents ESLint from looking for any other configuration files in the directory tree. Without it, ESLint will find the Drupal Core configuration files if this happens to be run from within a Drupal project. This will result in ESLint attempting to resolve plugins using Drupal Core's node_modules directory, which may or may not exist.

Also note that ESLint 8.x uses the --no-eslintrc flag, while the ESLint 9.x equivalent is --no-config-lookup. Drupal core is currently on ESLint 8.x, which is the previous major release.

Happy Linting

I ran into a few more hiccups than I expected along the way, but now feel confident that I can have consistent linting results between my local environment and the Drupal.org CI system in all of the JavaScript code I write for contrib modules. Hopefully this can help you do the same.

Resources

Drupal GitLab CI Template
ESLint
Prettier
ESLint Drupal Contrib plugin
Checking custom JavaScript with ESLint - an alternative approach that can be run from the conext of /core.

Drupal API Client 1.0 Release

Wed, 24 Apr 2024 00:00:00 GMT

We're extremely excited to announce the 1.0 release of the Drupal API Client. This release includes a fully functional JSON:API client and completes our commitment as a result of funding from the 'Pitch-burgh' innovation contest.

Before diving into the details of some recent updates, let's recap the state of the project now that it has reached 1.0.

What is the Drupal API Client?

The Drupal API Client is a set of JavaScript packages that simplify the process of interacting with common Drupal APIs. Most commonly, developers will use our JSON:API client to interface with Drupal's JSON:API endpoints, but we also publish a base API Client package that can be extended, a client for Decoupled Router, and may support other Drupal APIs in the future.

The Drupal API Client takes great care to be framework-agnostic and universal. It can be used:

with your JavaScript framework of choice, vanilla JavaScript, or even in Drupal itself.
with or without TypeScript.
on the server, or on the client.
with a bundler, or as a script import from a CDN.

Recent Developments

We've completed our Pitch-burgh commitment and released JSON:API Client 1.0. Thanks to all who contributed along the way!
We've published detailed documentation on GitLab Pages. The docs include live code examples, and is itself an example of using the API Client with Astro.
We've opened an issue proposing that the Drupal API Client packages be promoted to the Drupal namespace on npm. We'd love your feedback and support on the issue.
We're refining our 1.x roadmap and soliciting community feedback. One proposed priority will be TypeScript improvements and automatic type generation.

See you at Stanford WebCamp and DrupalCon!

Moving forward, we hope to prioritize additional features for projects that could use our libraries as a dependency. Catch up with us at community events in May to learn more and share your use cases.

The Drupal API Client will be featured in a session at Stanford WebCamp. Sessions are free and virtual, so this is a great way to get a more detailed overview of the project.
We'll be holding a Birds of a Feather discussion and participating in contribution events at DrupalCon Portland. The BOF will be on Monday, May 6 in room G129 at 4PM.

Beyond these events, we're always available in the #api-client channel on Drupal Slack and monitoring our issue queue. Hope to see you there!

Extending The Drupal API Client

Wed, 07 Feb 2024 00:00:00 GMT

import RadCallout from '../../../components/rad/RadCallout.astro';

As a result of our Pitch-burgh funding, the current focus of the Drupal API Client is to create a fully featured client for Drupal's JSON:API implementation. Even with that goal, we've focused on making our work extensible for other API formats in the future through the implementation of an ApiClient base class. Functionality that could apply to any API client is added to the base class, while anything specific to JSON:API is added to the JsonApiClient class (which extends ApiClient.)

Recently, we have been working on adding Decoupled Router support to our JSON:API Client. I found this implementation to be a great example of the extensibility of the library, so I wanted elaborate on it in a blog post for those who may want to extend the API Client in the future.

The existing JsonApiClient has the following method to retrieve data for a resource:

await client.getResource('node--article', '3347c400-302d-4f6c-8fcb-3e74beb002c8');

Ideally, users of Decoupled Router could also get an identical response by resolving a path:

await client.getResource('/articles/give-it-a-go-and-grow-your-own-herbs');

To achieve this, we first needed to provide a way to reliably get data from Decoupled Router.

The Decoupled Router Endpoint

With the module enabled, Decoupled Router exposes an endpoint with the following structure:

/router/translate-path?path=<path>

Given a path like /articles/give-it-a-go-and-grow-your-own-herbs the endpoint could provide a response similar to:

{
  "resolved": "https://dev-drupal-api-client-poc.pantheonsite.io/en/articles/give-it-a-go-and-grow-your-own-herbs",
  "isHomePath": false,
  "entity": {
    "canonical": "https://dev-drupal-api-client-poc.pantheonsite.io/en/articles/give-it-a-go-and-grow-your-own-herbs",
    "type": "node",
    "bundle": "article",
    "id": "11",
    "uuid": "3347c400-302d-4f6c-8fcb-3e74beb002c8"
  },
  "label": "Give it a go and grow your own herbs",
  "jsonapi": {
    "individual": "https://dev-drupal-api-client-poc.pantheonsite.io/en/jsonapi/node/article/3347c400-302d-4f6c-8fcb-3e74beb002c8",
    "resourceName": "node--article",
    "pathPrefix": "jsonapi",
    "basePath": "/jsonapi",
    "entryPoint": "https://dev-drupal-api-client-poc.pantheonsite.io/en/jsonapi"
  },
  "meta": {
    "deprecated": {
      "jsonapi.pathPrefix": "This property has been deprecated and will be removed in the next version of Decoupled Router. Use basePath instead."
    }
  }
}

While easy to make sense of, this response technically doesn't follow the JSON:API spec, which prevents us from using our existing JSON:API Client without modification. We could write a small amount of custom code in JsonApiClient to fetch and handle data from this endpoint, but this case is exactly what our ApiClient base class is intended for. With a similarly small amount of code we can extend the ApiClient class to add only what is unique to the Decoupled Router endpoint, while getting access to all of the features of the base class at the same time.

So rather than writing code specific to JsonApiClient, we decided to create a new DecoupledRouterClient class that our JsonApiClient could then make use of.

Extending ApiClient

For the sake of example, a simple Decoupled Router client could look like this:

// DecoupledRouterClient.ts
import {
  ApiClient,
  type ApiClientOptions,
  type BaseUrl,
} from "@drupal-api-client/api-client";

export class DecoupledRouterClient extends ApiClient {
  constructor(baseUrl: BaseUrl, options?: ApiClientOptions) {
    super(baseUrl, options);
    const { apiPrefix } = options || {};
    this.apiPrefix = apiPrefix || "router/translate-path";
  }

  async translatePath(path: string) {
    const apiUrl = `${this.baseUrl}/${this.apiPrefix}?path=${path}`;
    const response = await this.fetch(apiUrl);

    return response.json();
  }
}

In our constructor, the only modification we need to make is the default value for the API prefix. While the base class doesn't have a default, Decoupled Router uses router/translate-path. Now when instance of DecoupledRouter is created without this option, it will use the default.

We then define a translatePath method that:

Takes a path of type string
Uses the fetch method provided by the base class to make a request to Decoupled Router
Returns a promise with the provided json data

Using an instance of this class would look something like:

// main.ts
import { DecoupledRouterClient } from "./DecoupledRouterClient.ts";

const decoupledRouterClient = 
  new DecoupledRouterClient("https://dev-drupal-api-client-poc.pantheonsite.io");

const translatedPath =
  await decoupledRouterClient.translatePath(
    "/articles/give-it-a-go-and-grow-your-own-herbs"
  );

<RadCallout>Check out this code sandbox for a live version of the example above.</RadCallout>

Taking Advantage of Additional ApiClient Features

With this example we already have a functional client, but quite a bit more is possible using the features of the ApiClient class we extended. For example, We can already make authenticated requests using any of the supported authentication methods:

// main.ts
import { DecoupledRouterClient } from "./DecoupledRouterClient.ts";

const decoupledRouterClient = 
  new DecoupledRouterClient("https://dev-drupal-api-client-poc.pantheonsite.io", {
    authentication: {
      type: "OAuth",
      credentials: {
        clientId: "client-id",
        clientSecret: "client-secret"
      }
    },
  });

// API requests will now be authenticated
const translatedPath =
  await decoupledRouterClient.translatePath(
    "/articles/give-it-a-go-and-grow-your-own-herbs"
  );

Our example Decoupled Router client could be updated to take advantage of built in caching, logging, or locale support. For example, the following modification would allow us to make use of the defaultLocale option if our Drupal site supports multiple languages:

// DecoupledRouterClient.ts
import {
  ApiClient,
  type ApiClientOptions,
  type BaseUrl,
} from "@drupal-api-client/api-client";

export class DecoupledRouterClient extends ApiClient {
  constructor(baseUrl: BaseUrl, options?: ApiClientOptions) {
    super(baseUrl, options);
    const { apiPrefix } = options || {};
    this.apiPrefix = apiPrefix || "router/translate-path";
  }

  async translatePath(path: string) {
    // If it exists, incorporate the default locale
    // into the apiUrl
    const apiUrlObject = new URL(
      `${this.defaultLocale ?? ""}/${this.apiPrefix}?path=${path}`,
      this.baseUrl,
    );
    const apiUrl = apiUrlObject.toString();
    const response = await this.fetch(apiUrl);

    return response.json();
  }
}

Routing is a common problem, so we've added a fully featured getResourceByPath method to our latest @drupal-api-client/json-api-client release. We've also published the Decoupled Router client as a standalone package for anyone who wants to use it separately.

While the caching functionality of the client can lessen the impact, getResourceByPath still makes multiple API calls for uncached data, which leaves room for improvement. We could optimize this in the future by providing support for the subrequests module. That is yet another client for a type of Drupal API that could use the ApiClient base class as a starting point.

We're closing in on the 1.0 release of @drupal-api-client/json-api-client. If you’re interested in contributing, check out our project page on Drupal.org, and join us in the #api-client channel on Drupal Slack.

An Update on The Drupal API Client

Thu, 09 Nov 2023 00:00:00 GMT

At DrupalCon Pittsburgh, a team of interested community members submitted The Drupal API Client Project to be considered for funding as part of the Pitchburgh Innovation Contest. (If you missed it, check out our pitch video below, along with the original proposal.)

While competing community maintained API clients exist, Drupal does not offer any official JavaScript utilities to simplify the process of consuming data outside of the CMS. Our goal is to assemble a group of contributors to combine the best of existing Drupal API clients into a set of utilities that can both address common use cases with little configuration, and also be extended to support the needs of a diverse JavaScript ecosystem.

We were lucky enough to be selected by a panel of judges and the Drupal community to recieve funding. Since then, we've made some exciting progress on making our pitch a reality and wanted to share some updates.

Initial Goals and Vertical Slice POC

Our initial focus is what we've been calling the vertical slice POC for our JSON:API Client. As a checkpoint on the way to our eventual 1.0 release, we saw value in focusing deeply on a narrow portion of the client. Specifically, we decided to focus on the ability to get a collection of resources from the API.

Alongside that functionality, we also wanted to cover some level of implementation for all of our planned configuration options:

Custom fetch method #3376929
Authentication (basic authentication for the POC) #3376937
Local caching #3377144
Deserialization of API responses #3377191
Basic localization support #3377803
Adding query parameters to API requests #3383578

Going this deep on our method to get a collection should provide groundwork that will help simplify the remaining implementation on the way to 1.0. And possibly more importantly, having a functional POC early should allow us to engage with the community and gather real world feedback.

0.1.0 Release

Speaking of the POC, we're excited to announce that we've published our first release! You can install it right now using npm (or your package manager of choice):

npm i @drupal-api-client/json-api-client

API documentation for the package is available on GitLab pages.

Included below is a CodeSandbox including a simple example of using the client to get a collection of nodes from a Drupal site. Feel free to fork it and play around!

</iframe>

For the more adventurous, here's a live TypeScript example from our project readme that demonstrates more of the available configuration options:

</iframe>

This 0.1.0 release represents the initial POC, but we're planning on having continued 0.x releases as we approach 1.0.

Thanks to all who contributed to this release, including: coby.sher, pratik_kamble, mitchellmarkoff, shrutishende, elber, abhisekmazumdar, alexmoreno and the many others who provided feedback in the issue queue or dropped in on one of our Slack meetings.

Obligatory Call for Help

The Drupal API Client Project is rolling along, but we're still looking for help from the community to make it a success. With the release of this POC, one of the biggest areas we need help is getting general feedback on our work thus far. The wider the feedback we get from the community, the better chance we have to ship something that meets a wide array of decoupled Drupal use cases.

We've created a feedback issue to help guide the conversation, but we're also open to feedback in any form. We're you able to get up and running? Does the current API make sense to you? Is there a particular use case you're hoping to see supported? Let us know!

You also may have noted that many of our primary contributors work together at Pantheon. While we greatly appreciate Pantheon's support and sponsorship of this project, we're also looking for help from the broader community. We're doing a decent job keeping focused on the generic use cases that this project is intended to solve, but having a more diverse array of contributors can only help the end result.

If you're interested in contributing, check out our project page on Drupal.org, and join us in the #api-client channel on Drupal Slack.

And if you happen to be interested and on the East Coast, I'll be at NedCamp - both at the contribution day, and presenting on the API Client project during session day. Come say hi!

Decoupled Menus (and Beyond) at DrupalCon Portland

Tue, 19 Apr 2022 13:21:43 GMT

DrupalCon Portland is almost here! Getting back to an in-person DrupalCon will take some getting used to, but I'm especially excited for the ability to collaborate face-to-face during contribution events throughout the week. There is quite a bit going on in the world of Decoupled Drupal - here are some of the things I plan on focusing on...

Decoupled Menus Initiative

We’re aiming to get a point where we can declare this specific initiative complete and could use help in a few specific focus areas.

Migrating the decoupled menus linkset endpoint to Drupal Core - we're very close to getting this functionality included in core, but will still likely need some final assistance at DrupalCon.
Documentation improvements - first and foremost, we need to finalize the documentation specific to the new decoupled menus endpoint. We have made good progress there and mostly just need to incorporate some proposed revisions. We've also migrated the existing Decoupled Drupal documentation to a new home within the main Drupal Developer docs. There is a huge opportunity to flesh out the overall documentation for Decoupled Drupal, and hopefully this will evolve into a focused effort beyond the Decoupled Menus Initiative.
Decoupled Menu Parser - the Decoupled Menus Parser is a JavaScript library to simplify consuming data from the new decoupled menus endpoint. There are a few open issues to update the library due to recent adjustments to the API response.

Drupal State

Drupal State and the Need for a JavaScript SDK - bright and early Thursday morning I'll be presenting both about the Drupal State project, and perhaps more importantly my belief that an official JavaScript SDK is important for the future of Drupal.

Drupal State also has a healthy issue queue, so if you're interested in contributing or learning more about the project, I'll be around during the contribution events.

Generic Drupal Web Components

This project has had a long and winding road. Initially created in support of the Decoupled Menus hackathon at last year's DrupalCon, Generic Drupal Web Components (GDWC) aims to create a set of web components that can work well with data sourced from Drupal. But as you'll hear in my talk, finding a better solution to source data from Drupal resulted in the considerable tangent that was the creation of the Drupal State project.

There is an in-progress issue for GDWC that introduces two new components that can be used to make Drupal data available to any web components. I think this has the potential to open the floodgates for future components. I'd love to hear your thoughts.

Pantheon

That's a lot of stuff! I'll also be around the Pantheon booth and would be happy to talk about these projects, or anything relevant to decoupled Drupal on Pantheon. I'll also be at the Pantheon DrupalCon Party and almost certainly leaving early to rest up for my session the next morning :)

How Drupal's Preview Works

Fri, 08 Oct 2021 01:28:50 GMT

I've been thinking quite a bit recently about Drupal's options for decoupled preview with other JavaScript front ends. As part of some related experimentation, I found myself needing to understand more about how Drupal's standard preview functionality works. To be specific here - I'm talking about when you're editing a node and click on the preview button to see a full rendering of the page you're currently editing.

I realized I had never really had any reason to think about how that actually happens. Like many things on the web, it just kind of magically does. My general assumption was that it was some variation of Drupal's revision functionality. Something along the lines of an unpublished version of the node is created to represent the preview, and then is deleted after some period of inactivity.

As I dug into the code a bit I found out that my assumption was pretty far off.

Surprise number one was that the data for this is stored in Drupal's tempstore. Specifically Drupal's private tempstore. The tempstore is a key/value collection that can make data available across requests and the private tempstore also includes checks to ensure that the data is only available to the current user. Here's a short post with a good comparison of the private tempstore and the shared tempstore.

Surprise number two was that the form state of the current node edit form is what is stored in the tempstore, not the node entity itself. As we'll see in a second, it is possible to derive the node entity from the form state, but technically only the form ends up in the tempstore.

Let's take a peek at some of the relevant code.

When the node edit form is submitted, the following form submit handler is used:

// From core/modules/node/src/NodeForm.php

/**
 * Form submission handler for the 'preview' action.
 *
 * @param $form
 *   An associative array containing the structure of the form.
 * @param $form_state
 *   The current state of the form.
 */
public function preview(array $form, FormStateInterface $form_state) {
  $store = $this->tempStoreFactory->get('node_preview');
  $this->entity->in_preview = TRUE;
  $store->set($this->entity->uuid(), $form_state);

  $route_parameters = [
    'node_preview' => $this->entity->uuid(),
    'view_mode_id' => 'full',
  ];

  $options = [];
  $query = $this->getRequest()->query;
  if ($query->has('destination')) {
    $options['query']['destination'] = $query->get('destination');
    $query->remove('destination');
  }
  $form_state->setRedirect('entity.node.preview', $route_parameters, $options);
}

Here the form is being set in the private tempstore, and the user is redirected to the preview route.

Before the preview route is rendered, the node preview service will be invoked. Within this service the convert method gets the form state from the tempstore and then derives the node entity form that.

// core/modules/node/src/ParamConverter/NodePreviewConverter.php

public function convert($value, $definition, $name, array $defaults) {
  $store = $this->tempStoreFactory->get('node_preview');
  if ($form_state = $store->get($value)) {
    return $form_state->getFormObject()->getEntity();
  }
}

Finally, within the node preview controller, the view method assembles a build of the node.

// core/modules/node/src/Controller/NodePreviewController.php

public function view(EntityInterface $node_preview, $view_mode_id = 'full', $langcode = NULL) {
  $node_preview->preview_view_mode = $view_mode_id;
  $build = parent::view($node_preview, $view_mode_id);

  $build['#attached']['library'][] = 'node/drupal.node.preview';

  // Don't render cache previews.
  unset($build['#cache']);

  return $build;
}

Pretty much everyone I ran this by was surprised*, but now that I've worked through it a bit this approach makes more sense to me. Many of these previews will be abandoned, so the tempstore actually is an appropriate home for them due to its built in expiration and cleanup. Same for the private tempstore - previews are specific to the current user, and could contain sensitive data. Finally, I'd imagine that not processing this as a full entity/revision saves some overhead considering that this preview could be updated frequently while editing.

I now understand the 'how' and possibly the 'why', but that doesn't make it any easier to access this data for other preview related purposes. I think that is the reason why this specific use case is covered by few, if any, decoupled Drupal preview solutions. From my initial experimentation it can be solved, but it will take quite a bit of work to do right.

I'd love to hear more from those managing decoupled Drupal sites. Is previewing in your decoupled front end while editing in Drupal really the holy grail it seems like sometimes? Or are other preview workflows meeting your needs?

* I'm sure there are some Drupal lifers out there who were well aware of this, and are yelling at their screens right now.

Configuring Tugboat Live Previews For Drupal General Projects

Tue, 27 Apr 2021 18:23:13 GMT

I've written previously about my excitement that Tugboat now offers live previews for core and contrib merge requests on Drupal.org, so I was especially happy to see live previews added to the Generic Drupal Web Components (GDWC) project recently.

GDWC is a general project on Drupal.org and runs using NodeJS rather than PHP. Since Tugboat runs on Docker it seemed likely that we could also run Node, but the existing documentation is unsurprisingly focused on Drupal PHP projects.

With a little experimentation we found that Tugboat Live Previews could in fact be easily configured to run for Node based projects as well. Almost all of the credit here goes to Joe Parsons who took the lead on this issue. We ended up adding .tugboat/config.yml to the project with the following contents:

services:
  apache:
    image: tugboatqa/httpd:2.4
    default: true
    commands:
      init:
        - curl -fsSL https://deb.nodesource.com/setup_lts.x | bash -
        - apt-get install -y nodejs
      build:
        - npm install
        - npm run build-storybook
        - ln -snf "${TUGBOAT_ROOT}/storybook-static" "${DOCROOT}"

As you can see above, this first installs Node on Tugboat's httpd image. We can then install our dependencies and run a static build of Storybook, which is the primary development tool used by the project. The last step symlinks our build asset to the document root on the image so that it will display when someone views a live preview.

It feels like a more lightweight Node image could serve a similar purpose, but this solution meets our needs for now and builds in a reasonable amount of time.

Thanks again to all who helped make these live previews possible - they have already simplied the review process for this project.

We're In The Golden Age of Contributing to Drupal

Wed, 21 Apr 2021 02:06:16 GMT

I spent some time recently looking at the new general project type on Drupal.org. As someone who spends quite a bit of time working on Drupal adjacent things, a home for Drupal related projects that aren't modules or themes opens up some exciting (and important) possibilities. As I thought more about how this could be used for an upcoming project, it dawned on me that this was just latest in a series of developments that vastly improve the experience of contributing to Drupal.

I'm sure other Drupal lifers can think of older milestones, but my personal turning point goes back about four years ago to when the project application process was updated so that any user would have the ability to create a full project on Drupal.org. Prior to that, the process to create a new project felt insurmountable (at least to younger me.) Drupal activity will never reach the 50 NPM packages that were probably created while you read this paragraph, but reducing the barrier to entry makes a big difference.

Fast forward a few years and the hits just keep on coming.

Unquestionably the biggest improvement is the introduction of the merge request workflow on Drupal.org for both Drupal Core and Contributed projects. Rather than generating and uploading patches, users can now create an issue fork and submit a merge request. This process is likely to be much more comfortable to developers who are familiar with creating pull requests on Github. And this workflow wouldn't have been possible if Drupal hadn't migrated to a platform like Gitlab in the first place.

The process to set up a local environment for Drupal development has improved over this time frame as well. Docker based solutions like Lando and DDEV have prepackaged environments to automate the process of setting up a local environment to contribute to Drupal. Looking forward, cloud based environments could further simplify this set up process.

Simplytest.me continues to be a great option for contributors to evaluate modules and even test patches, and it has recently been modernized with a backend powered by Tugboat. Speaking of Tugboat, it also now provides live previews for merge requests for Core and Contrib projects.

That brings us all the way back to the recent addition of general projects.

A ridiculous amount of work went into all of these developer experience improvements. Thanks to the many people who put in the time and effort to make this possible.

Update:

During his keynote at DrupalCon this year Dries shared a walkthrough from Matthew Grasmick that provided quite the counterpoint. I'll let the video speak for itself:

This is a completely fair reminder that while we've come a long way, we also still have a long way to go to modernize our contributor experience.

While I've spent years getting to a point where I can benefit from the improvements that Drupal has made to the contribution workflow, countless others still get turned away at the door. Both things can be true: the contribution process is vastly improved, and it also isn't good enough. I'm still hopeful though. Based on the improvements outlined in my original post I have faith that the Drupal project will be successful continuing to adopt the features of Gitlab and I'll be able to collaborate with countless new contributor friends who are able to make it through the proverbial Drupal door.

Help Build a Decoupled Menu Web Component at DrupalCon

Tue, 06 Apr 2021 21:43:45 GMT

Next week at DrupalCon North America as part of the Decoupled Menus Initiative I'll be participating in the Decoupled Menu Hackathon. Members of the community will be coming together to build a variety of menu components using different approaches and frameworks as a way to exercise a new menu endpoint provided by the Decoupled Menus module, along with the infrastructure supporting the new general project type on Drupal.org.

Leading up to DrupalCon I've also begun work on the Generic Drupal Web Components project. Starting with a menu component, it aims to create a library of generic web components that are accessible, framework agnostic, possible to style, and easy to use with data provided by Drupal. Here's a quick video overview:

If you're interested in getting involved, we could use your help! The Decoupled Menus Initiative Keynote on Tuesday April 13 is a great opportunity to catch up on the current status of the initiative. Following the keynote there will be related contribution efforts throughout the week of DrupalCon.

If you're interested in working with the Generic Drupal Web Components project, there is a meta issue of tasks tagged for DrupalCon. Many of them involve seeing how far we can extend and re-theme the generic version of this component, which should leave room for a lot of interesting experimentation. Have other ideas? Leave a comment on the issue.

Web components not your thing? There is a similar meta issue for a menu component being built using React. Prefer another framework or have another wild idea? Try out the demo endpoint and see what you can come up with.

Looking forward to collaborating with everyone and seeing how many different ways we can represent data from a single menu endpoint. 🚀

Resources:

Gearing Up For Midcamp 2021

Mon, 15 Mar 2021 19:56:40 GMT

Midcamp, my local Drupal Camp, is back March 24 - 27. Midcamp went virtual at the beginning of lockdown last year and while it was a rousing success, we were definitely making it up as we went along. This time around we're trying a new format with an increased focus on community building which we're also hoping will help combat the Zoom fatigue we're all feeling. The schedule breaks down as follows:

Wednesday, March 24: Get Started with Drupal - content focused on introducing attendees to Drupal.
Thursday, March 25: Meet the Drupal Community - lightly structured community building activities.
Friday, March 26: Share Your Knowledge - a one day unconference format
Saturday, March 27: Give Back to the Project - Our traditional contribution day.

Within that schedule, here are a few things I'm extra excited about:

In addition to our traditional Zoom rooms, we'll also have a space in Gather.town for us to virtually socialize. I've been spending some time moving around sprites, and expect to have some surprises in store.
On Thursday, Matt Glaman will be leading a workshop on writing JavaScript tests using Nightwatch.js
And on contribution day, one of the featured initiatives will be adding additional test coverage to Drupal's new Olivero theme.

Registration is pay what you want, and we'd love to see you there.

With any luck we'll have an in-person component at Midcamp next year, which will also overlap with St. Patrick's day. I'm workshopping the following tagline: Midcamp 2022: IRL in IRL.

Recent Drupal Podcast Appearances

Sat, 13 Mar 2021 22:56:40 GMT

I've been lucky enough to be a guest on a few Drupal related podcasts recently, continuing my long standing trend of talking to anyone who will listen.

Drupal Easy Podcast: Back in January, Mike Anello had me on to talk about Front End Components on Episode 238. We mostly focused on the basics, which was a nice change of pace compared to the component integration talk I had been giving recently.

Talking Drupal: the folks at Talking Drupal recently decided to have a rotating co-host seat for four week stretches and were nice enough to invite me to be the first to occupy the seat. During my four week run I was lucky enough to chat about:

Starting a Drupal Career with Mike Anello (I swear Mike is following me)
Tugboat with Matt Westgate
Iterative Approach To Decoupling with me!
Decoupled Menus with Gabe Sullice, Baddy Sonja Breidert, and Liam Hockley.

The Decoupled Menus discussion was especially enlightening as it clarified some of my misconceptions about the project (they aren't shipping a menu component, instead they are making enhancements to Drupal to better support many menu components) and also gave me some ideas about how I might be able to contribute.

A huge thanks to Stephen, John and Nic for having me - it was a great experience. Looking forward to hearing from their future guest hosts.

On the topic of podcasts, I'd recommend the following recent non-Drupal podcasts that I thought were especially great:

Shop Talk Show 451: JavaScript and Web Components with Nolan Lawson
Syntax 330: React Query + More React with Tanner Linsley

Talks: Florida Drupal Camp 2021

Sat, 20 Feb 2021 00:00:00 GMT

Slides / Session Page

This was my second time giving this talk, and I kind of forgot how dense of a topic this is. One major update was a re-worked example of approaches to scoped styling:

</iframe>

(The sandbox probably doesn't make a ton of sense without having someone walk through it. Might try to do a quick recording of it someday.) Next time I do this styling demo, it should go later in the presentation. I think people needed more of a foundation on web component concepts before we get this deep into styling specifics.

I also added an example of using web components with a JS framework - React in this case:

</iframe>

I think this example ended up being a little confusing because React and Web Components don't play all that nice together. I'll try to work up an example with a more web component friendly framework next time.

Also saw that I somehow found my way into being listed as a featured presenter on Noti.st where I post my slides. Cool! (Less cool is the fact that my wife called me out on using a photo that is too old...)

Other odds and ends:

The way the camp used gather.town was really fun. Hoping we can borrow some of FLDC's great ideas for Midcamp.
Found out about Promet Source's Provus project which is a component based Drupal stater site. We are working on a similar effort at Bounteous.
Hoping to check in on contribution day for a bit.

Configuring gatsby-source-drupal to only Import Referenced Images

Fri, 19 Feb 2021 00:00:00 GMT

We've recently started using Gatsby on Bounteous.com, my company's existing Drupal 8 site. Rather than attempting to rebuild the entire front-end at once, we're starting iteratively with small portions of the site and leaving other sections rendered by Drupal (possibly forever.) This approach posed a few interesting problems, one of which was configuring the gatsby-source-drupal plugin to only import the content we needed for our build.

The gatsby-source-drupal plugin pulls data from Drupal’s JSON:API endpoints and makes this data available to React components via Gatsby’s GraphQL API. By default, the plugin imports all data from the source Drupal site. Since for this initial phase Gatsby would only be used to build a small subset of pages, most of this data was unnecessary and also would have the side effect of greatly increasing our build times.

As an initial attempt to solve this problem, we used Drupal’s JSON:API Extras module to only expose the resources that our Gatsby build needed to depend on. This helped, but we still eventually needed to enable the file resource, which pretty much immediately sunk our build times. Gatsby was now importing (and worse yet processing) local versions of years worth of images that we didn’t need to support our new content. We eventually found that it was possible to configure gatsby-source-drupal to only import the files referenced by content that was necessary for our builds, but it required a combination of configuration options that wasn’t completely obvious from the documentation.

The first step was to add the file resource as a disallowed link type:

// In your gatsby-config.js
module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-drupal`,
      options: {
        baseUrl: <your_url>,
        /// Disallow the full files endpoint
        disallowedLinkTypes: [`self`, `describedby`, `file--file`],
      },
    },
  ],
}

This alone would result in all files being ignored by the plugin. A little bit further on in the disallowed link types documentation is the following note:

When using includes in your JSON:API calls the included data will automatically become available to query, even if the link types are skipped using disallowedLinkTypes. This enables you to fetch only the data you need at build time, instead of all data of a certain entity type or bundle.

This essentially allows us to re-allow specific files if they are referenced by other content. What makes this feature potentially easy to miss is the fact that it uses the plugin’s filter option, which typically further restricts the data sourced from the plugin. The resulting configuration ended up looking like this:

// In your gatsby-config.js
module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-drupal`,
      options: {
        baseUrl: <your_url>,
        /// Disallow the full files endpoint
        disallowedLinkTypes: [`self`, `describedby`, `file--file`],
        filters: {
          // Use includes so only the files associated with our decoupled content
          // types are included.
          "node--decoupled_home_page": "include=field_dhp_components",
          "paragraph--dhp_hero": "include=field_dhp_fg_img",
          "paragraph--dhp_animation_cards": "include=field_dhpac_images",
          "paragraph--featured_post": "include=field_dfp_bg_img",
        },
      },
    },
  ],
}

With this configuration, if a featured post paragraph is used on the homepage, any associated background images (field_dfp_bg_img) will be sourced by Gatsby as well. While it does require some maintenance as we incorporate new entities into our builds, it has kept our build times quite reasonable thus far.