Here’s how you correctly move Drupal sites between environments
Before we dive in, this particular article is highly technical, and unless you are familiar with the Drupal settings.php file and apache’s virtual host setup, and possibly have a bit of scripting under your belt, you may find your eyeballs rolling back into your head as you try to read on. For that, I apologize. If you use Drupal for your website and this is too technical, I do urge you to pass this on to your IT group, as it contains some information which might just make their lives easier.
This essay is an expanded version of a lecture I gave last year at a Drupal meetup on Long Island. IT is always a work in progress, and even if you were there, you might find a few new tricks in this version.
Drupal is a great tool. There are so many Drupal 7 sites out there, and with even more Drupal 8 sites being deployed, anything to help deployment is a boon to IT. One of the biggest problems recurring in the Drupal community is the need to deploy different settings.php files within your company’s different environments.
After all, your settings for Development, Staging/QA and Production are not going to be the same. They certainly are not going to all be pointing at the same database, right? Please tell me that the answer to that last question is yes, if not, please call me — we need to talk.
This is one of the problems that we faced a few years ago when Drupal development spun into high gear at Millennium. We had those three environments, and sometimes even more, as there were often parallel paths of development happening at the same time, and they often needed separate databases, and even separate staging environments for some.
The first way we approached this was by looking at the hostname in the URL, which was being used to call the web site. Based on that we had a set of “if” statements which would set the various variables. While this worked, it gave away more information than we were comfortable with disseminating. Specifically, it gave the developers access to the credentials of the production database. While the databases were in protected segments of a highly secure set of networks, we were still uncomfortable with having the credentials available.
What I ended up devising was a set of procedures using our existing puppet configuration system, Jenkins deployment system, and some custom scripting. We now put the database credentials, and many other aspects of configuration, which change between environments, into the puppet YAML files. These then get written into the apache virtualhost file, which writes them into the apache environment. Here is a snippet of the puppet YAML file:
There is obviously a lot more to it, but for this example, that is all we need. Based on the information above, we write out a virtual host configuration file named “path_staging_sitename.conf” which contains, among other lines, the following:
SetEnv DBHOST stagingdb1
SetEnv DBUSER staging_sitename_user
SetEnv DBPSWD staging_sitename_password
SetEnv DBNAME staging_sitename_database
In order to read these into the Drupal system, we use php’s getenv function to read those environment variables and set the database array using them, which is all fine and good for web requests, but leaves much to be desired when you need to use drush.
The solution for us with regard to drush was to write our own script, which we call “milldrush” short for Millennium’s drush, which parses the virtual host configuration file, inserts the proper environment variables into place, then calls drush. Puppet helped us with this as well. By having puppet write out the configuration file, as well as having it set up the directory structure for the site and maintaining the proper document root for the site, you have the advantage of being able to have a scripts directory at the same level as your document root, “/www/path_staging_sitename/scripts.” Using the example above, scripts can run, which look at the directory name above it and know the exact location of the site’s configuration file, something like /etc/httpd/conf.hosts.d/path_staging_sitename.conf, where it can look for the SetEnv line and populate the environment accordingly.
This method can be used for other settings as well. Through puppet, you can put a marker in place for the type of environment, and put a red bar across the top of the page for development, yellow for staging, and none for production, if you have users who work in more than one environment. I recently fielded a complaint from a third-party developer that a site was sending out the staging link for a “forgot password” request. When I looked in the mail logs on the various servers involved, it turns out that they were actually on the staging environment and not the production environment as they had been convinced they were. Having some sort of marker for the environment can help avoid such mistakes.
Knowing your environment can help ease other problems as well. We deploy our code using Jenkins. Using the environment marker lets the system know which set of deployment scripts to run for which environment. For instance, when deploying to production, we block all users who are part of the “administrator” group, as nobody should have that much access in production. We have a number of other roles to handle the responsibilities which are production-appropriate.
By using a combination of puppet, Jenkins and some custom scripting, we have made our deployments easier and more secure, using some of what I have laid out here, hopefully you can as well.