Emergency Alert Infrastucture

Submitted by Kevin on Tue, 03/31/2020 - 17:09

In today's blog post, I will dissect the emergency alerts system I built for my college.  Details are somewhat generic for security reasons, but this should give a clear idea of the infrastructure.

Any emergency alert system is really yet another collection of clients and a server to broadcast messages out.  The key is trying to make the system robust and reliant on a minimum of technologies and connections, so that it stands a decent chance of actually working when it's needed.

Client Architecture

Since all of our major websites run our own implementation of the official Georgia Tech web theme, it seemed most logical to integrate an alert feature into that theme module.  I integrated at the core theme level, allowing the alerts to work with any of our theme implementations (Drupal, WordPress, Omeka, PHP Application).

For Drupal sites, I provide the Georgia Tech gold header bar as a block that is placed in a region designed for it at the top of the master page template.  So, I just extended that concept and have the theme module provide an alert block that can be placed in that same region, just above the gold header bar block.  The backend code for the alert block is effectively the client code.

Since it only makes sense to use a push methodology (i.e. a central server pushes alerts to clients), I built the client component to simply look for a specific file in the web root directory of the website.  If that file exists, then it is read in and displayed on every page above the Georgia Tech gold header bar.  If that file does not exist, then pages are rendered in their normal fashion.  All CSS for formatting the message is a part of the theme module, so that messages only have to include the main content marked up with appropriate CSS classes.

Drupal presents an extra hurdle in that it caches all pages for public viewing, so simply changing the alert file does not cause a new message to be immediately displayed on pages that have been cached.  I ultimately found it worked best to simply have the alert module register it's own "cache tag" and then implement a service API path which, when accessed, invalidates that tag.  Once invalidated, the cache will regenerate any pages that include that tag, causing the new alert message to be pulled into the cached copies of those pages.

function mymodule_clear_alert_cache() {

Server Architecture

To push alerts out, I started with a simple command line infrastructure using SCP and SSH keys.  It's simple, but effective, and less likely to break over time compared to other possible methods.  I created a unique SSH key just for alerts so that it wouldn't be tied to any one person and made sure the key's email address reflected its purpose.  That way, we would be able to easily recognize it when doing future systems maintenance.

With a simple script, I could SCP the alert file into the right place on a particular web hosting account, then (for Drupal sites) use "wget" to connect to the service API path and trigger a cache refresh.

/usr/bin/scp -q -i /path/to/key/folder/my-id_rsa my-alert-file.html ssh-account@servername.gatech.edu:/path/to/webroot
wget --spider https://servername.gatech.edu/path/to/cacheclearing/service

A few notes:

  • For SCP, '-q' is quiet mode to suppress extra messages and '-i' is pointing SCP to your special private key.  Do note that the private key has to be readable only by the current user, or SSH and SCP won't use it.

  • For wget, '--spider' is a simple way to keep wget from actually writing out what it retrieves from the web server.

This works well enough for more IT savvy people, but I wanted to make it even easier to use, so that someone with less IT knowledge, like a communications manager, could send out an alert if necessary.  So, I put together a simple PHP application using my standard web application framework.  The application maintains a data table with basic details for each site that can be managed (site URL, server hostname, site SSH account, path to the site's web root on the server).  Another data table holds the messages and a few options for them (type of alert, date/time stamp option).

The user interface allows an authorized user to select a site and enter a message, configure the options, and then save the message.  An API module in the application then creates the alert message file and SCPs it to the appropriate web server, after which (for Drupal sites) it runs the appropriate "wget" command to trigger a cache refresh.  If the user has removed the message (by erasing everything from the message box), then the API module uses SSH to send a command to delete the existing alert file, which causes the website to go back to normal page rendering.

Only authorized administrators (i.e. top level IT and web staff) can add, delete, and update website configurations, since there's no reason to expose that functionality to those users who should not be using it.  This also keeps the interface simpler for communications people who may need to post a message in a hurry.

While not implemented at this time, it's fully possible to create a master alert function that would cycle through all websites that the system knows about and send the same message to them all.