Automating the day away: An annoying Moodle plugin and bash scripts

I’ve spent a lot of time working with Git submodules as a part of my journey in systems admin. They have been a key tool for me in improving my upgrade and deployment strategies for Moodle by making the process faster, less prone to error, and more secure. Meaning, there is less downtime and less risk when I do upgrades. All it takes to update all my site plugins is:

sudo git submodule update –remote

Anyway, the above is my justification for working with submodules. But there are struggles sometimes. They work great with 98% of Moodle plugins, as most plugins are published in publicly accessible Github repositories. However, there are those cases where a plugin is not published publicly — usually a premium plugin that is only available to download directly from a vendor website. When I need to use these, I just make a git repository in the plugin folder on my local computer and push to a private repository I can add to my project as a submodule. When a new version comes out, I swap in the new code in the folder, commit and push changes, and am good to go.

So the thorn in my side are the Kaltura plugins. UP uses Kaltura for video and they have a very extensive and robust set of Moodle plugins. My problem is, they host all 10(!) plugins in a single Github repository. The plugins are all in subdirectories as if they were installed in the root directory of a Moodle site, like mod/kalvidassign. That makes it impossible to track them as submodules. I’ve opened a Github issue to see if they would change this but I doubt they will, so I need to rely on the manual method of creating private repos for each one. When an update comes out, that means an annoying, repetitive process of copying new files into folders, and then adding, committing, and pushing ten git repos. No thank you!

I thought this would be a good chance to improve my bash scripting skills. I know the basic programming/scripting techniques but am not very experienced writing in bash. So I did a little research and came up with the script below. Now my process to update the Kaltura plugins in my repo is:

  • Download and unzip the new version (a single folder that contains all the updated code)
  • Move that folder into the root of my existing local Kaltura plugins repository
  • Run my gitcomitter script

The script does the following:

  • asks for the name of the folder containing the updated code
  • asks for a commit message
  • iterates through each plugin directory and copies over the new code, adds and commits changes, and pushes to the remote repo.
#!/bin/bash
#
# this helper script will go into each plugin directory and add/commit/push any changes
# (assuming you have initialized git repos and set up remotes in them)
# use from the root of the Kaltura plugins folder
# copy in the updated code folder and run this script

#get the name of the folder with the updated plugin code
read -p "Directory of of updated code: " updatedir
#get a commit message
read -p "Commit description: " desc

#create array of plugin paths
MODPATHS[0]='blocks/kalturamediagallery'
MODPATHS[1]='filter/kaltura'
MODPATHS[2]='lib/editor/atto/plugins/kalturamedia'
MODPATHS[3]='lib/editor/tinymce/plugins/kalturamedia'
MODPATHS[4]='local/kaltura'
MODPATHS[5]='local/kalturamediagallery'
MODPATHS[6]='local/mymedia'
MODPATHS[7]='mod/kalvidassign'
MODPATHS[8]='mod/kalvidpres'
MODPATHS[9]='mod/kalvidres'

#loop through array, copy over new files, and do git stuff
for i in ${MODPATHS[@]}

do

   cp -r $updatedir/$i/* $i
   ( cd $i && git add . && git add -u && git commit -m "$desc" && git push )

done

#delete the update folder, we don't need it anymore
rm -r updatedir

Photo by Markus Spiske on Unsplash

Gushing Over Git

When I started working in sysadmin/ops land, I knew that version control would be important, and knew a little bit about it. I had used Git in a very basic way while playing around with some amateur coding projects. I knew how to initialize a repo, see which files were tracked, and stage and commit changes.  But I didn’t quite appreciate how crucial a thorough understanding of Git would be to really be effective in maintaining and deploying code in a systems context – with thousands of users depending on availability of a server that needs to be patched, updated, etc safely, securely, and on-schedule. It didn’t take long before I was dedicating as much time as possible to mastering Git. I even created a “Git” section on my OneNote notebook, which is a big deal if you know me.

I knew I needed to learn a lot in a hurry. Thankfully, not only is Git Open Source software with thorough documentation – there is a real open community around git. A great example of this is the Pro Git textbook. This is a fantastic OER (open education resource). The book can be viewed online or downloaded in an ebook format for free. In my case, I downloaded the .mobi Kindle format and used the “Send to Kindle” option to add the ebook to my Kindle library. This allowed me to read the book with variable fonts and text sizes, and to highlight and make notes. There are also tons of good forum spaces for discussing git and asking questions (or more likely finding an answer that has already been given), from Stack Overflow forums specifically for GitHub.

Another virtue of git is that by its nature, it allows you to experiment and play. I cloned a repo of my institution’s production Moodle code into my own safe practice space, knowing that I could try things out and was perfectly safe as long as I didn’t push any commits back. And even if I did somehow end up with bad code, you can always reset back to a working commit. This allowed me to work on git skills and have a Moodle directory to explore and mess-up in a consequence free environment.

The git skills I’ve built have already paid off – aside from being crucial to any sane workflow for updating Moodle or WordPress sites, git has already saved me from crashing a site at least once. I got a bit sloppy on a dev WordPress site and did something to the wp-config PHP file that crashed the site. Yikes! It could have been anything from a missing semi-colon to a misspelled word. Hard to say, but the site was totally dead. Instead of spending a lot of valuable time poring over the file to find the typo, I executed a simple command:

sudo git checkout wp-config.php

The one area of consternation I’ve had with Git has been with submodules. The previous Moodle admin at my school set-up a submodule system to manage all of the various plugins we run (50 or so at last count). In theory this should make managing a suite of plugins easier, but in practice I’ve struggled with submodules, at least in terms of using them at all efficiently, as they make the git process a bit more complicated conceptually. However I am starting to get the hang of submodules now and and getting clarity on how they function in relation to the “superproject” in which they reside. This article from Catalyst, The Git submodule: misunderstood beast or remorseless slavering monster?  was especially helpful for me.

Photo by Fatos Bytyqi on Unsplash