Our developer workflow for MODX CMS projects

Discover the strategies a MODX agency uses to effectively manage large, complex projects with multiple developers.
Maple's Tech Blog | By Peter Bowyer | Published:

Here at Maple we work with large and complex MODX websites. Many projects involve multiple developers and in-house teams working on them, and this exposes a weakness of MODX: it’s not designed for multiple developers to work on the same project at the same time.

MODX stores its data and much code in the database. This is great for content editors and quick tweaks in production, but it’s not so great for MODX developers. We need to be able to work on the code in a version-controlled environment, and we need to be able to deploy changes without affecting the live site.

Here’s how we do it.

Working with & not against MODX

Let’s start with a couple of definitions:

  • LIVE means the live site, the one that the public sees
  • DEV means the development sites, the one that developers work on and which run on their local computers
  • STAGING means the staging sites, which are used for testing changes before going live. Not all our workflows have these, but I’m including them for completeness.

Two rules underpin our workflow:

1. Content is always edited on LIVE

We made the decision that content changes are always made on LIVE and pulled to DEV. Having content changes flow in a single direction simplifies things, because there is no good workflow we found for bidirectional content sync. No matter what you do it’s going to need human input to merge conflicting changes.

2. Code is stored in source control

We use Gitify to keep our chunks/snippets/plugins/templates/ContentBlock configurations and more under source control. Gitify is a MODX CLI tool that can extract and import elements from MODX to files on disk, and vice versa. It’s a great tool and is central to our workflow.

We’re fans of DDEV for managing local development environments. Everyone on our team and the client’s in-house team has a local DDEV development environment set up on their computer. Using a DDEV console command we can fetch the latest LIVE database at any time (personal information in the database is automatically anonymised) to work locally on the latest version of the LIVE site.

Every developer then develops like they would with any PHP web project, in their own branch. At regular intervals they run gitify extract and commit their changes to Git. Git handles the sync errors and conflicts.

When changes have been tested and merged, we’re ready to deploy to LIVE.

How we deploy our MODX changes to LIVE

Deployment strategies are project-specific: there is no one size fits all here.

At its simplest it’s a bash script that is run on the hosting server(s) and does a git pull to get the latest changes, runs gitify build and clears the MODX cache.

In practice the deployment process is more complex than this because there’s CSS/JS assets to build and minify, composer packages to install, other caches to invalidate and so on. Sometimes these deployment scripts run within Ansible to provision servers - our clients haven’t yet moved to Dockerising environments so that is yet to come, and we’re looking forward to the day it arrives!

The nicest deployment process is when we push to a named branch in the Git repository and it automatically triggers the deployment.

Example deployment script

Here’s an example deployment script. It’s a bash script that is run on your local machine and connects to the server via SSH.

#!/bin/env bash

# Enter your SSH connection details here
SSH_STRING="username@192.168.0.100"
WEBDIR="/var/www/mysite/public_html"

# Build the production assets in DDEV
ddev npm run build

# We exclude uploaded assets etc. Adjust exclusions to suit your project
rsync -avz --progress --exclude=core/cache --exclude=assets/uploads/ \
  --exclude=node_modules/ --exclude=vendor/ --exclude=core/packages/ \
  --exclude=.ddev/ --exclude=.git/ --exclude=.idea/ --exclude=core/config/local.config.inc.php \
  --exclude=.gitify_backup/ --exclude=*.sql.gz \
  . ${SSH_STRING}:${WEBDIR}

ssh -p 2020 ${SSH_STRING} "cd ${WEBDIR} && composer install"
ssh -p 2020 ${SSH_STRING} "cd ${WEBDIR} && php vendor/bin/gitify build"
ssh -p 2020 ${SSH_STRING} "cd ${WEBDIR} && rm -rf core/cache/*"
# If your server isn't preventing access to the .gitify_data directory, you can remove it
#ssh -p 2020 ${SSH_STRING} "rm -r ${WEBDIR}/.gitify_data"

# Optional: delete the production assets in DDEV. You'll need to adjust this path to suit your project
rm -r assets/dist

What’s the developer workflow like?

Let’s say I’ve been assigned a ticket to add a contact form to a site. My workflow (assuming I already have a DDEV environment, local checkout of the site etc) would be:

  1. Commit any changes I have in my local environment, so they are backed up
  2. Download and load the latest copy of the LIVE database into my DDEV environment, to get everyone else’s changes
  3. Run ddev gitify extract to see what’s different from the files in Git. This is a helpful reminder if I’ve forgotten to fetch the latest changes from Git and someone else has made changes
  4. Now to work! Edit the chunk to add that extra field; edit the snippet; test locally
  5. Run gitify extract
  6. Check the changes in my Git client (I use Fork, other Git clients are available) and commit them
  7. Deploy to STAGING and test
  8. Deploy to LIVE

But the client’s team want to make edits on LIVE/STAGING!

Maple sponsored the update of Gitify Watch to MODX3. Gitify Watch allows elements to be edited in LIVE and changes committed directly to Git. It works well but it’s been hard to get buy-in from customers to use it, because they have security concerns about their web servers having write-access to their source code repository.

On a recent project we’ve excluded chunks from Gitify so they are edited direct on LIVE like content. As we’re not adding too many chunks now this works okay, and we use static resources for those chunks we want to version control.

Managing MODX site settings outside Gitify

We exclude site_settings from Gitify, because they will often be different between LIVE and DEV (e.g. SMTP connection details, Elasticsearch URL) and contain passwords and sensitive data which we don’t want visible in our Git repository.

I wrote a MODX Plugin called System Settings Override to remove the pain of having the wrong (LIVE) settings in your dev environment. It lets you set any system_settings from a file on the filesystem. This means you don’t have to edit the system_settings every time you import a new copy of the LIVE your database, because the details in the file have priority over the values in the database.

Handling Gitify conflicts

This workflow described is great until two developers add a new chunk (or template, or snippet, or plugin, or…) at the same time in their local development environments. Both new chunks have the same database ID, which causes a conflict. Gitify will try to resolve this automatically by giving one a new ID, but it’s not always successful and you have to manually fix the conflict (a painful task!).

Our workaround is to require developers to coordinate new additions ahead of time. This is not ideal and we have ideas for modifying Gitify to handle this better, but the real isssue is MODX using incrementing numeric IDs for these objects instead of random UUIDs. Changing the core of MODX would prevent conflicts in the first place.

Lessons Learned

Through our experience, we’ve gained valuable insights:

  • MODX’s database-centric architecture requires careful management of content and code changes.
  • Version control is essential, but tools like Gitify have limitations that need to be addressed.
  • Automating deployments and managing environment-specific settings are key to an efficient multi-developer workflow.

Conclusion

In the next article I’ll cover the changes we’ve made to the MODX core to make it work better with our workflow.