Automating Remote Administration

This feature has long been a goal for OPTASY; it’s been more challenging than we expected. You see, automation is easy if all sites are built from the same base repository using the same folder structure and implementing the same base modules. Yet OPTASY supports fabulous client sites that, because they leverage the complexity and extensibility of Drupal they are creative sites., each unique, blending modules, custom code and content. A generic one-size-fits-all update process has never worked.

In the past, we have manually updated sites using local scripts and patches, but this method was never going to succeed in providing security updates within 24 to 48 hours of a security release.

Early on, we decided that well-built automation needs to:

• Identify and provide security updates released on drupal.org in a timely manner.
• Implement a clear path of deployment and testing which starts from and, after validating each environment (Dev; Staging, and Production), and then returns to retest production site code to ensure all updated components have been propagated across all environments.
• Allow for testing independently of ongoing development by the customer.
• Provide clear and consistent communication to all stakeholders about pending changes.

RA Automation has (mostly) arrived!

Finally, we are well on our way to achieving exactly those standards, providing automated security updates to RA subscriptions within 48 hours of a security release, to both the core and contributed modules. We expect to reach this goal in the first half of 2015, but the beta functionality is already updating RA subscriptions.

With the introduction of automated patch and update deployment, there are minor but meaningful changes going on with some of the features you are familiar with today.

RA Environment

In order to avoid interrupting ongoing client development, OPTASY provides a free RA environment. For OPTASY Cloud Enterprise (ACE) clients, the RA environment is on a completely separate server to ensure that RA deployment and testing does not overburden customers’ dedicated testing environments. The redundant server allows security updates to be deployed into client environments as soon as they are available and readied for client testing.

RA Preferences

Last year, client-facing RA preferences were made available in your RA administrative settings. Clients can control RA automation on a per-subscription level. We plan to continue to refine these to work best with the script and the unique needs of clients.

Using APIs

OPTASY RA has utilized the internal OPTASY Cloud API in order to take advantage of the tools built by our fabulous cloud team. The API allows automation to read RA Preferences as well as other centralized subscription data that is used to create human-readable tickets and monitor progress. Optasy RA also takes advantage of the Zendesk API, and has contributed improvements back to Zendesk.

Drush and LiveDev

Since automated updates use Drush, it takes advantage of the Cloud Live Development feature. All updates take place on the RA Environment first, then are switched to Live Development for the update and code commits that will be deployed for the particular customer.

While drush typically does all updates at once, we ensure that each module update is a discrete git or svn commit. This allows much more efficient troubleshooting down the line, as load order can impact how the whole instance works due to custom code. The additional staging step, in the RA Environment, allows OPTASY to revert a single module update. It also allows customer development teams to cherry-pick the updates that best fit into their ongoing development process.

Drush Backend

OPTASY has also contributed back to the Drupal community by sponsoring development in Drush, eliminating a problem that interfered with automated updating.

RA clients will know that enabling the Update module is essential in order to get security updates. Drush 6 requires this module to access module status information without which Support teams have no way of knowing the status of a site’s modules. Asking clients to enable the module so we could scan always took extra time, often delayed the process.

In Drush 7 the Update module unnecessary. The fabulous-to-work-with F.L. Jonathan Araña Cruz developed the code, while Samuel Mortenson tested on it OPTASY servers. The result is the ‘Drush Backend’ which has been committed to the current dev version of Drush 7.

This work has streamlined initial versions of our automation, removing a significant amount of code that worked around Update cache issues (pm-enable update, pm-refresh, truncate cache_update, etc.). We reduced the total number of pm-updatecode calls. This also means that enabling the update module is no longer a requirement for RA subscriptions.

Automation now and in the future

Currently, OPTASY automation requires manual initiation via a queuing system. The OPTASY RA team always initiates the queue when a core security update is released. Otherwise, we initiate a periodic scan. Based on the improvements described here, RA customers can look forward to the following:

• Daily automated scanning which will create a new branch every two weeks, so as not to flood you with new updates.
• The ability to approve and automatically initiate all stages in the update process. This means that, unless you need troubleshooting from our team first, you can apply a security update without any required intervention by OPTASY.
• The ability to specify bug-fix updates that can be included in a security update, or done independently. This will be a feature of RA Premier only.
• Support for patching via Drush make files.

Finally, since Automation is under active development, this is a great time to let us know what features and capabilities you would like! Please feel free to comment below, I look forward to hearing your thoughts.