For some reason, the new internal page caching system has been the hardest thing for me to wrap my head around in Drupal 8, and dealing with it properly has eluded me for the better part of two years. I think part of the problem is a lack of clear, understandable documentation about how it all works, so here's my attempt to sort it all out:
A Whole Host of Caching Tools
One of the more confusing parts to Drupal 8 is that there are now several ways to control caching, but the names of these components are not terribly intuitive and documentation is still kind of sparse.
External Caching Controls
Drupal 8 has an external cache control, on the Development -> Performance page, named confusingly "Page cache maximum age" (in Drupal 7, it was called "Expiration of cached pages", and the visible description included the phrase "external cache"). Contrary to what you might think, all this control does is to set an HTTP header that systems like Varnish can read and use to decide how long to cache that page. The setting has absolutely no effect on the internal page cache.
For those who use an external cache system like Varnish and want to tell it more about their content, the third party module Advanced Page Expiration is supposed to help with that.
Internal Page Caching Modules
To add to the confusion, Drupal 8 now has two internal page cache modules, both of which are enabled by default:
- Internal Page Cache - caches pages for anonymous (not logged in) users only
- Internal Dynamic Page Cache - caches pages on a per-user basis (for logged in users)
(Just FYI, you can disable either module if you wish, as neither is required to be active. However, you should probably only do this on a production site if you have it sitting behind an external caching system like Varnish.)
The Drupal documentation site has a page briefly describing the Internal Page Cache that gives a nice comparison to how the Drupal 7 page cache worked. Here's the quick summary of the Drupal 8 module that highlights its key points:
-
Internal Page Cache works on the basis of cache tags, which are just machine-name style strings assigned to visible components on pages, such as blocks.
-
When an anonymous user visits a page, it is rendered based on anonymous access permissions, and the rendered HTML code stored in the internal page cache based on the tags assigned to the various components of the page.
- UI and back-end mechanisms are able to call a system cache control function to invalidate a tag. When a tag connected to any of a page's components is invalidated, the whole page is re-rendered on the next anonymous request.
What kept me confused for a long time is that the Drupal 8 version does not use expiration timestamps, which was how the Drupal 7 version worked. Thus, if you don't have the right triggers in place, a dynamically generated page (e.g. a news feed page) will stay cached forever – or at least until you manually flush all of your caches.
The Internal Dynamic Page Cache is a completely different beast. I haven't dug into it, but it lets you add additional user-role based tags so that pages are only regenerated for a user if something that the user can see on the page has changed. The core code uses this a lot to control caching of things like the main navigation bar: if a new menu item is added to the main navigation menu (or an existing one updated), but the item has access controls to prevent users with role 'xyz' from being able to see it, then the dynamic cache won't re-render anything for those users, since it would be a waste of time to do so.
So far, I haven't encountered anything I've worked on getting stale for logged in users, so I don't think this cache comes into play unless you specifically ask to have your dynamically generated page or block cached.
BigPipe
For the record, there's a third component loosely related to caching called BigPipe, which is supposed to help speed up browser rendering. It works by sending page structures first, then the content that goes in those structures, allowing the browser to set up the page layout while it is still receiving content from the server (via AJAX calls). It's supposed to be production ready, but the last time I tried playing with it, it broke some of the HTML structures on my test site. That was a few months ago, so I need to run another test and see if it's gotten better with fixes I've made to my site theme code and any updates in the most recent release of Drupal 8.4.x.
Update: BigPipe is now on by default in Drupal as of Drupal 8.5.0 if you are installing Drupal from scratch, but will still be off if you are upgrading to Drupal 8.5.0 (or later) from an earlier version of Drupal 8.
Additional Update: There's an important aspect of BigPipe that can come to bite you, as I discovered when I started trying to enable BigPipe on my production sites. If you have JavaScript routines that manipulate the DOM, they'll likely have to be wrapped with special Drupal JavaScript routines to have them run after each chunk of the page has been loaded. This is because part of the BigPipe magic is letting the page load technically "complete" so that the browser will render to the screen. Doing that calls any BODY.onload event handlers, which means those handlers won't be able to see all of the page content.
This adds another twist: your DOM modifying JavaScript has to be able to handle being called multiple times, since pages are loaded in multiple chunks. So, any custom code will need to be rewritten to keep track of which DOM elements its processed so that it doesn't reprocess those elements. A simple way of doing this is by adding a custom data property to each processed element, and checking for that property as you skim through looking for elements to process.
Taming the Internal Page Cache
If you should get into writing custom modules that provide dynamic content, you'll have to learn how to tame the Internal Page Cache, as otherwise you'll end up with stale content showing in those blocks and pages for anonymous users.
Dynamic Pages
For dynamic pages, there is a simple shortcut. Just add the following before you return your render block:
\Drupal::service('page_cache_kill_switch')->trigger();
You should also be able to get the same effect by adding the following to your routing file for the page's route:
options:
no_cache: TRUE
The downside to either is that the page is never cached, so every access to it regenerates the whole page. You may want this in some cases, such as where you want the latest news always showing. In other cases, it might be perfectly fine to let the page get cached for up to an hour or two. To do that see the information below on cache tags and cron jobs.
Dynamic Blocks
When it comes to dynamic blocks, the coding gets more complicated, as you aren't controlling the entire page of output – only one section of it. While you could include a call to the "page cache kill switch" in your block build() code, doing this for a block that appears on all pages (e.g. a custom main menu block) effectively disables the entire Internal Page Cache. Even for blocks that only appear on one or two pages, killing the cache completely for those pages is usually not the best way to go if any of the content is static and there's not a need to have up-to-the-second dynamic updates showing. The smart way to handle caching in these cases is as follows:
In Your Block Build Code:
Add the following to your render hash array:
'#cache' => array(
'max-age' => 0,
'tags' => array('<YOUR TAG NAME HERE>'),
),
"<YOUR TAG NAME HERE>" should be replaced with a unique tag. If your block may be used in multiple instances with different output, then the tag should include an instance identifier. If all instances of the block will have the same output, then you can just specify a single static identifier. There's no standard syntax, but it must be machine-name style (i.e. no spaces or special characters other than '.' or '-'). A good approach is to use a prefix not likely to be used by anyone else, such as the machine name of your block's module, followed by a '.' and a unique name for the block type itself, if your module provides more than one block type. As an example, for the 'sample' block of a module named 'gt-demo', a good valid cache tag would be "gt-demo.sample".
The "max-age" setting is another point of confusion, as you would think setting it to '0' would turn off caching. Well, it does, sort of – but not the way you think. "max-age" only applies to the caching of the block, not to any page the block is placed on. So, setting it alone does nothing for anonymous users. For anonymous users to see regular updates to your dynamic block content, you have to use both a cache tag and a suitable "max-age" setting.
In Your General Module Code:
Updating your build code only gets you half-way there, and the next half is the tricky part. You have to find a place to add an invalidation trigger at the right time, though in some cases, this may not be necessary. For example, if you are writing a new renderer for a main navigation menu, you can just specify the cache tag as "config:system.menu." followed by the machine name of the menu object being rendered. This will map the block to the menu object, and then anytime the menu object is updated (an item is added, deleted, or modified), that tag will be invalidated and the block and any pages it is placed on will get invalidated and re-rendered.
If you are providing an interface for updating the data used in the generation of the block or page, then in your code that handles saving out updates, you can add the following line:
\Drupal\Core\Cache\Cache::invalidateTags(['<YOUR TAG NAME HERE>']);
Be sure to replace "<YOUR TAG NAME HERE>" with the tag you set in your render code. This will invalidate the tag, causing the pages or blocks that reference that tag to be regenerated the next time they are accessed.
In cases where you are pulling data from an outside source, you may not know when that data has been updated. If you're just reading data from an external source on demand, and don't know when that source is being updated, you can just add a cron job to your module as shown below, which will cause your cache tag to be invalidated every time cron runs, replicating the behavior of Drupal 7. Of course, if you are already using a Drupal cron job to pull your data, simply add the "\Drupal\Core\Cache\Cache::" line shown below to it.
function my_module_cron() {
\Drupal\Core\Cache\Cache::invalidateTags(['<YOUR TAG NAME HERE>']);
}
Be sure to replace "my_module" with the machine name of your module, and replace "<YOUR TAG NAME HERE>" with the tag you set in your render code. If you want a longer time period between refreshes, you can add something in there to store the time of the last invalidation, then check the system clock to see if enough time has elapsed, and if so, invalidate the tag.
A word on cron jobs: By default, Drupal uses "poor mans cron", using page accesses to trigger cron jobs the first time a page is accessed after a fixed amount of time has passed (three hours by default, but this can be changed at Administration -> Configuration -> System -> Cron). The key here is that the trigger is a page access - if your site gets very little traffic during any part of the day or night, it's possible that more than three hours will elapse before cron is invoked. Of course, for remote data retrieval applications, this is a moot point, since no accesses to the site means that no one is missing anything, and the cron run on the next page access should catch you up with any new remote data. However, if you're doing something that needs more cron precision, you should look into setting up your web hosting account to call Drupal's cron process on a precise periodic basis (making it a true cron job). The Drupal.org website has instructions for setting up outside cron jobs (see step number four on that page).
As with all new techniques, test your code out thoroughly before assuming it just works. That is, make sure your invalidation trigger is being called at the right time, and test to see that the page with the associated block is then being re-rendered on the next anonymous view. You can set up a testing environment with two different browsers (or the private browsing function of one browser), where one window is logged into the site and the other is not. Instant validation (such as changing a menu item) should just work immediately: change an item in your logged-in window, and then refresh your "anonymous" window and you should see the change. For invalidation done by a cron job, you can make a change to your source data, then use the button on the Cron administrative page (/admin/config/system/cron) to immediately run all cron jobs and see what happens in your "anonymous" window.