Render Arrays, Twig, and Drupal 8 Programming

Submitted by Kevin on

Over the course of today and yesterday, I went through a pretty amazing journey.

A few years ago, I created a custom Drupal 7 module to generate a block of news headlines that renders in the "featured top" section of a Drupal content page.  When I ported this and other modules over to Drupal 8, I figured out the necessities and got the code working, but I'd left a large chunk of the module in Drupal 7 style or worse.  For example, I was collecting the data for these headlines through direct database queries, which even in Drupal 7 was frowned upon, but figuring out how to pull data from Paragraphs entities the right way had always proved too elusive.  On top of that, my output was generated directly within the block class's build() method, as I'd simply not had time to untangle the inner workings of Twig templates and render arrays.  I knew these techniques were not going to be terribly forward compatible, but they were purposeful quick and dirty kludges to get my sites working.

While this had served our needs for a couple of years, the time had come to dive into the deep, deep end of the pool and figure these things out once and for all.  I had little hope of succeeding – I'd tried before and each time become too frustrated with the lack of decent documentation and tools to get close to finishing.  The difference this time, besides hoping for a little more maturity in documentation (which proved a failed hope), was giving myself two full afternoons to just go for it and see what would happen.

A Real Development Environment

Perhaps the biggest problem in figuring out how to properly code my project was the need to delve into extremely large and complex objects, since Drupal 8 has become as object-oriented as you can get.  Fortunately, I quickly found the perfect tool, and I cannot recommend it highly enough.  It's called "kint", and is provided as part of the third-party Devel module.  You don't even have to enable any other part of Devel – just load it into your site and enable the "Devel Kint" sub-module, and you're ready to go.  Any time you want to explore an object, just do the following in your code:

  kint($object_variable);

Reload the page that uses that code, and you'll get a nifty color-coded explorer at the top of the page.  Unlike print_r() and var_dump(), kint can handle recursiveness in Drupal objects with ease.

It's also useful to understand how to turn on other debugging tools in Drupal 8.  Some are easy to enable, such as disabling CSS and JavaScript aggregation (done via Configuration -> Development -> Performance) and uninstalling BigPipe to make debugging faster and less confusing.

Additional debugging features can be enabled in your site configuration files.  This is a little complex to explain here, but to get started, look at the following files in any Drupal distribution for instructions:

  • /sites/example.settings.local.php
  • /sites/default/default.services.yml
  • /sites/default/settings.php  (Look for "Load local development override configuration" near the bottom of the file)

In particular, the 'services' file, when activated, lets you enable Twig debugging, which provides several useful features:

  • Extra comments in your HTML let you know which template files are being used, and you even get suggestions for template file names that can be used to override the current template file.
  • A dump() function that you can use in Twig templates to see what data is being passed to the template.
  • Twig caching is disabled, letting you immediately see the effect of changes to template files without having to flush caches every time.

You can also use the files listed above to disable some of the internal caches and further speed up seeing the effects of changes you are making.  Just keep in mind that some caches always have to be rebuilt after changes are made, such as the caches for hook functions, routes, and libraries you define in your custom modules.

Data Access, the Drupal 8 Way

As tempting as it can be (and oh, is it tempting when moving to Drupal 8 from Drupal 7) to just resort to MySQL database calls to get the data values you need, this is never a great idea, since database schemas can be changed without warning in future versions of Drupal.  The reason it's so tempting is that it can be incredibly difficult to grasp and visualize how to find the data you need amongst a sea of objects and method calls.  So, below is a short summary of some of the calls I found quite useful.

Retrieve the Current Page's Node Object

$nid = \Drupal::request() -> attributes -> get('node') -> id();
$node = \Drupal\node\Entity\Node::load($nid);

Retrieve a Field from the Node Object

$node -> get('field_something');

Retrieve a Multi-Value Field from the Node Object

foreach ($node -> get('field_something') -> getIterator() as $something) {
  // $something will be one of the values stored in field_something
}

Retrieve a Paragraph Object from a Paragraph Entity Reference

While these calls are written for paragraphs, this command structure can be adapted to anything using entity references - the key is using the correct entity class name and path.

// Get all of the paragraph entities defined in the system
$paragraphs = \Drupal\paragraphs\Entity\Paragraph::loadMultiple();

// Get a specific paragraph entity
$paragraph = \Drupal\paragraphs\Entity\Paragraph::load($paragraph_id);

// Get a specific paragraph entity - using the $something object from above
$paragraph = \Drupal\paragraphs\Entity\Paragraph::load($something->getValue()['target_id']);

// Get the paragraph object's type machine name
$paragraphType = $paragraph -> getType();

// Get a field value from the paragraph object
$value = $paragraph -> get('field_something') -> getValue();

Mastering Render Arrays

Compared to figuring out how to use render arrays and twig templates, the data access stuff was a breeze!  In spite of Drupal 8 having been out over three years, there still isn't very good documentation available about these subjects.  I'll do my best to demystify them a bit, but it's no easy task.

First of all, render arrays are not new to Drupal 8 - they were available in Drupal 7, but in 8 they became all but mandatory.  Even so, you can get by with the most basic of array:

$results = array(
  '#attached' => array(
    'library' => array('my_module/library_name'),
  ),
  '#cache' => array(
    'max-age' => 0,
  ),
  '#markup' => $buffer,  // Raw HTML code in  $buffer
);

return $results;

// Or, simplified as far as you can take it, assuming you don't need any custom CSS or javascript

$results = array(
  '#markup' => $buffer,  // Raw HTML code in  $buffer
);

return $results;

This is basically just wrapping raw HTML that you generate in a shell of a render array and then sending it on to the default Twig template for output.  It works, but with some big caveats, the biggest being that Drupal forces the Twig sanitizer on with no way to override that setting, making it difficult to use forbidden HTML tags and impossible to use forbidden properties.  This is because the developers want you building HTML structure in Twig templates, not in your regular PHP code.  That said, the allowed HTML tag list can be modified by including a '#allowed_tags' value in your render array:

'#allowed_tags' => array_merge(\Drupal\Component\Utility\Xss::getAdminTagList(), array('input', 'form', 'label'),

This example line adds the 'input', 'form', and 'label' HTML tags to the existing master list, letting you use them in your raw HTML code.  There doesn't seem to be an equivalent call for changing the allowed property list, so you have to use modern techniques at attach JavaScript to any controls you include in your output (i.e. no "onclick", "onchange", etc.)

The preferred method for generating your page or block output is to pass data objects to Twig in your render array and let a Twig template manage the actual rendering.  This may seem convoluted at first, but there is a benefit.  If you've spent time building Drupal 8 sites, you may have become familiar with the new set of field display formatters, or just "FieldFormatter"s in Drupalese.  These useful little plugins now handle the rendering of field data for output to the screen, and when you embrace render arrays in your custom code, you can make use of these formatters and save yourself a lot of manual rendering work.

Once again, the biggest headache is just knowing what to put in your render array, so here are a few useful examples:

Include a Formatted Text Field in a Render Array

$renderArray['#textfield1'] => array(
  '#type' => 'processed_text',
  '#format' => 'full_html',  // substitute your preferred text format
  '#text' => 'Formatted Text Here',
),

// An example using a formatted text field object stored in $body, which might have been pulled from a node object

$renderArray['#textfield1'] => array(
  '#type' => 'processed_text',
  '#format' => $body[0]['format'],
  '#text' => $body[0]['value'],
),

Include an Image Field in a Render Array

// Retrieve the file entity using the target_id from the image field - $field contains the image field as pulled from something like a node object
$file = \Drupal\file\Entity\File::load($field[0]['target_id']);

$renderArray['#image1'] = array(
  '#theme' => 'image',
  '#uri' => $file -> getFileUri(),
);

// Optionally set width, height, and image style to use

$renderArray['#image1'] = array(
  '#theme' => 'image',
  '#uri' => $file -> getFileUri(),
  '#width' => 640,
  '#height' => 480,
  '#style' => 'thumbnail',
);

My ultimate goal was to make use of the responsive image field formatter, which isn't as well documented, but after much searching and trial and error, turned out to be fairly simple:

// Retrieve the file entity using the target_id from the image field
$file = \Drupal\file\Entity\File::load($field[0]['target_id']);

$renderArray['#image1'] = array(
  '#theme' => 'responsive_image',
  '#responsive_image_style_id' => 'my_responsive_style',
  '#uri' => $file -> getFileUri(),
);

This assumes that:

  1. You've enabled the core Responsive Image module
  2. You've gone to Configuration -> Media -> Responsive image styles and created a responsive style
  3. The value you give for "#responsive_image_style_id" matches the machine name of the style you created

See the documentation for Responsive Image if you need help with any of these steps or use of the module in general.

You can hard code a defined responsive image style ID where you see "my_responsive_style" above, but you might want to test that it exists, in case you should you want to use your module on different instances of Drupal.  In fact, you should really start by testing to see that the Responsive Image module is loaded, albeit if you make that module a dependency of your module then in theory the module test isn't needed.

// Check to see if the responsive image module is enabled *and* the 'my_responsive_style' style is defined
// If not, then fall back to regular image rendering

$themeList = \Drupal::service('theme.registry')->get();
$imageStyle = 'image';

if (isset($themeList['responsive_image'])) {
  $styleList = \Drupal\responsive_image\Entity\ResponsiveImageStyle::loadMultiple();
  if (isset($styleList['my_responsive_style'])) {
    $imageStyle = 'responsive_image';
  }
}

Now, as you build your render array, you can check $imageStyle to see which render format you can/should use.

A Finished Render Array

Ultimately, a finished render array is a collection of these hash arrays (referred to as Twig variables) as I've shown above, along with various configuration keys like "#attached" and "#cache" and possibly others.

What can be confusing about this is that from your PHP code side, you don't actually know how your variables will be used on the Twig side, and they can be used in one of two ways:

  1. The direct value of a variable can be output as part of the page, part of an HTML tag (e.g. as the value of a tag property), or even be used in making a calculation or manipulation in Twig logic structures.

  2. A variable can be processed as a render reference for a sub-object, causing the appropriate Twig template for that object to be loaded and the variable value (the hash array) passed to a new copy of the Twig processor along with that template.  This is how you get the HTML for an image placement or a formatted text field included on your final page / block output.

To know what's going to happen, you have to review the Twig template you are using.  To find that template, you have to look at the "#theme" key for a render array (or sub-array, in the case of a sub-object).  That key, however, doesn't point you directly to the file – you have to look it up in the master theme registry (see the code example above for how to retrieve the registry for perusal) to find the location of the Twig file.

Making Your Own Twig Templates

Passing a bunch of variables in a render array is pretty useless in a normal context.  For example, passing them as page output or block output does nothing useful, because the default templates for pages and blocks won't know what to do with those variables.  Thus, to make custom render arrays useful, you have to build a custom template that uses them.

There are multiple approaches to doing this, but the one I used starts with giving the template a unique name.  I went with the format of "my-module-templatename.html.twig", where "my-module" is the machine name of my module (with underscores changed to dashes) and "templatename" is the unique name of this particular template within my module's scope.

Configuring Your Module for a new Template

Next, in your my_module.module file, add the following:

function my_module_theme($existing, $type, $theme, $path) {
  return [
    'my-module-templatename' => [
      'variables' => ['varname1' => NULL, 'varname2' => NULL],  // Declare all variables that you want to be able to pass to this template
    ],
  ];
}

Make sure your key name matches your template name exactly, minus the ".html.twig" part.  Don't forget to flush your caches after this, as you must do after adding any new hook functions to your custom module.

You're now all set to add the following to the top level of your block or page's render array:

'#theme' => 'my-module-templatename',

Building the Template

Fortunately, there are some pretty decent guides to Twig itself, even in the Drupal context, so I won't go very deep into the subject of building the actual template.  The main thing to note is the two ways that variables can be used.  But, first, here's the basic syntax for using a variable:

// Top level variable
{{ variablename }}

// Sub-value of a top-level array or object variable
{{ variablename.key }}

The important point to understand is the difference in outputing a variable that has a scalar value (integer, string, boolean) versus outputing a variable that contains an array.  In the former case, the value is simply output inline with any surrounding static content.  In the latter case, if there is a "#theme" key, then the array is treated as another render array, and it is processed as such using the specified Twig theming template.

Thus, if you passed a proper image render array inside of a variable identified in your top-level hash array as "#image1", and you placed "{{ image1 }}" in your Twig template, then in that location in your template output the HTML generated by the Image renderer will be inserted.

In Summary

If your head is spinning right now, don't feel bad - mine was too by the end of my first day tackling this stuff.  I'm still not 100% certain that I did everything the "right" way, but in the world of Drupal you have to recognize that there can be variants on what is truly "right", so 100% may not be truly possible.  In any case, my code works to my satisfaction, and it's a lot more future proof than my old code.  It's all going to get a lot more testing and review before it goes out to any production sites, but overall I've made a huge amount of progress for two afternoons of hard work.

Hopefully the examples here will help others to figure this stuff a lot faster than I did, but I must offer the disclaimer that all of this is based on my own research and experimentation and may not work for everyone in all situations, and I in no way guarantee that these examples really are the best way of doing the indicated tasks.  In other words, enjoy, but use at your own risk!