Note: I wanted to write up a piece that reflected on a year of Moodle admin, talked about how I tackled my biggest upgrade project so far, and also laid out my current approach to performing upgrades. Obviously, that is a lot and turned into too much for a post. So here is part 1 with some reflections. I’ll lay out my upgrade process in a shorter, pt. 2 post, coming (hopefully) soon.

I’m approaching a year of Moodle sys-admin-life and wanted to take a moment to reflect. I remember the uncertainty (sheer terror?) I felt just starting out: I had a lot to learn. Getting used to Linux CLI, Git, working with webservers and MySQL databases were all pretty new. I remember struggling through the first minor point upgrade I ran on my institution’s production Moodle site. I scheduled downtime for the evening hours and worked into the late night. A year later, this process takes me just a few minutes…there was definitely some newbie confusion and just a need to blindly go in and figure things out on my part that led to things taking way longer than they should have (especially figuring out those Git submodules). Yup, I was sweating the details….but I also remember the satisfaction and relief I felt, and the rush of seeing the update script complete and my upgraded site loading for the first time.

Throughout the last year, I kept practicing. I set up sandbox environments to get more experience on test sites (thanks AWS). I ran two more minor updates on production and became more confident each time. I got to experience installing, removing, and updating plugins, and worked with a contract developer to update a custom plugin. I even built out a brand new production Moodle site in the cloud, using Moodle with the Snap theme for a MOOC flavored side-project.

This is all to say that this Summer when it came time to plan not a minor-point update, but a major one – 3.4 to 3.6 – I felt not only ready to tackle the job but to make real improvements to my process.

Making a plan

Since this was the biggest Moodle project I had yet taken on I knew it was critical to come into the upgrade with a solid plan. So first I pored over the official Moodle Docs guide to updating. I made sure my server environment would continue to meet all recommendations, and painstakingly went through all extra plugins to check for compatability and available upgrades.

For the main piece of the update – the Moodle code – I initially researched updating in much the same way I had done the smaller point updates. Since I use Git for version control with submodules for plugins, that would simply entail changing the tracked branch from the official Moodle Git repo and doing a git merge. Unfortunately, this was not as easy as it first seemed, and I found that others in the community struggled with this approach. Unlike minor point updates, merging a major update requires resolving many conflicts or differences between your existing code base and the updated code. I quickly decided instead to use a clean copy of the 3.6 code.

So, I simply cloned the MOODLE_36_STABLE branch from the Moodle GitHub page into a new project directory. All I migrated over from my old folder was my config.php file and my .gitmodules file.

That introduced two new challenges:

1. How to decouple any existing code customization that was tracked in my main Moodle Git repo (and therefore was not present in a fresh version of Moodle core)
2. How to update plugins and efficiently migrate them as Git submodules to my new project

Decoupling customization

This was pretty straight-forward if somewhat time-consuming. Since Moodle treats basically everything as a plugin, I simply went to the Site Admin -> Plugins -> Plugins overview -> Additional plugins screen and compared that with my .gitmodules file (the file in git that keeps track of all of your project’s submodules). This let me identify any custom items that were being tracked in my main repository.

Last year I got UP signed up with a GitHub for Education plan, so I had unlimited free private repositories. So I just created new repos for the few items that were either homegrown (a custom HTML block) or that vendors for whatever reason do not make available through a public GitHub. The most annoying was Kaltura’s video plugins, which they put on GitHub as a whole repo with multiple plugins in various directories in a way that won’t work as a submodule, meaning I had to make my own repo for each individual included plugin. If anyone has an idea of how to include something like that wholesale as a submodule, I’d love to hear from you in the comments!

Anyhow, once I had moved everything into it’s own Git repo, I just added those to my .gitmodules file in the new project.

Migrating submodules

To make sure I was getting updated versions of my plugins that were compatible with 3.6, I just updated my .gitmodules file as needed. Honestly, most Moodle developers do a great job of providing branches for multiple Moodle versions and use the same naming convention so doing a find an replace for “MOODLE_34_STABLE” to “MOODLE_36_STABLE” took care of 75% of the work. But the remainder of plugins that needed updating either used non-standard branch naming conventions or had changed remote URLs or maintainers, so there was a fair bit of drudgy manual work to get everything pointed to the right place.

The problem of migrating submodules was a bit less straight-forward. You can’t just copy a .gitmodules file into a new repo, and as far as I can tell there are no easy commands to run that will look through your list of modules and actually add them to your project – maybe I’m wrong… That said: scripts to the rescue. I genuinely wish I had kept a bookmark so I could give proper credit but I found a script posted online somewhere and modified it to suit my needs. It took a bit of experimentation to get it right, but I was able to just put the script below in my 3.6 project directory (with a .gitmodules file copied from my 3.4 project) and run it to add all of my submodules to the project. If this looks useful to you, note that it requires all of the entries in .gitmodules to include the optional branch = branchname line, which you may not have if you are tracking master. So you may need to manually add a line like branch = master to your .gitmodules entries.

 #!/bin/sh

 ## script that looks in a .gitmodules file and re-adds all of the submodules to the git repo
 ## for future improvement - currently fails if branch not listed in gitmodules. should roll back to default master

 set -e

 git config -f .gitmodules --get-regexp '^submodule\..*\.path$' |
     while read path_key path
     do
         url_key=$(echo $path_key | sed 's/\.path/.url/')
         url=$(git config -f .gitmodules --get "$url_key")
         branch_key=$(echo $path_key | sed 's/\.path/.branch/')
         branch=$(git config -f .gitmodules --get "$branch_key")
         git submodule add -f -b $branch $url $path

     done

With that work done, I had a fresh copy of the 3.6 code with ALL local modifications exiting as submodules that pointed to a 3.6 compatible version of a plugin. I ran the update script and….boom! Everything went through and installed fine on the first try.

Photo by Joshua Chun on Unsplash