Monday got off to a good start, with a message from my customer in EE while I was eating my breakfast letting me know that the /var partition on the production server was full, and the app was only partially working. It turned out that the logs for the past few days had been filling up with error messages about a failure to connect to the relay server to send emails. We initially thought it was a connectivity issue, but it also looked like the Bamboo email library (or almost certainly my use of it) was raising exceptions because of a failure to gracefully handle the inability to send emails. Normally this kind of exception would have triggered an email letting me know something was up, but those emails never arrived because … well, you know.
After clearing some disk space to allow everything (apart from emails, but fortunately they aren’t a huge part of the application – a few people will have failed to get alerts, and I had to send a couple of password reset links manually) to start working again, I decided to treat it as an opportunity to switch from Bamboo to Swoosh. I didn’t have any reason to think it would better, but the Phoenix framework switched to Swoosh as its default for new apps a while ago, and presumably they had their reasons. The migration was pretty painless – both libraries use gen_smtp under the covers, and the APIs are very similar – so it was mostly some changes to how tests worked, plus a bonus improvement of extracting the email bodies to separate template files using phoenix_swoosh. With that change deployed, everything started working again. I’m still not 100% sure what the root cause was, but I think it started failing when I upgraded to Erlang 26 and Elixir 1.15, so I suspect some incompatibility there (the tests, of course, don’t use the smtp adapter, allowing this kind of thing to fall through the cracks – I guess we should probably test actual emails as part of a post-deploy checklist).
Embarrassingly, it was several hours into the day before I twigged that the reason there were so many logs was that each exception triggered a notification email, which then failed, caused an exception, and triggered another notification email …
On Thursday stage five of the Tour of Britain started and finished in Felixstowe, coming within half a mile of my house en route, and I felt like a day off anyway so thought I’d go and watch them zip past. I decided to go slightly further afield, so incorporated my spectating in a trail run out to a narrow country road three or four miles away.
After getting home, showering and having some lunch, Robin let me know that a few people were cycling out to Felixstowe to watch the finish and have a couple of beers, so I ended up riding out to meet them there and watch the riders come flying down Sea Road at a ridiculous speed. Then after some refreshment in the Felsto Arms (and a quick stop at the Walton Half Moon on the way home) I tried – and mostly failed – to keep up with everyone at what felt like an only slightly less ridiculous speed back to Ipswich. Hot off the press, it turns out that the three second gap that we watched Wout van Aert open up with his sprint finish was the deciding factor in his victory in the overall event (about which I just accidentally sent a spoiler to Robin, not realising he was waiting to watch the highlights. Oops).
Saturday was Ipswich parkrun’s 500th event, which I ran very slowly because …
Sunday was FRR’s big race of the year, the Felixstowe Coastal 10. It seems that the weather always adds some kind of extra challenge to this one, whether it’s rain, wind or (this year) unseasonably roasting temperatures. Even before race day this made life a bit harder by attracting hordes of people to the seaside when we were trying to drive round putting up signage on the Saturday afternoon, but also meant the race itself was horribly tough (not sure whether it was warmer than the Newmarket 10k back in June, but it felt like it). My time was five minutes slower than last year, but at least there was some convenient sea to get in at the finish to cool off (possibly the only time I’ve swum at Felixstowe when it wasn’t new year’s eve!)
I did at least manage to pretend I was enjoying it when I spotted Liz with her camera, before reverting to the grimace that I suspect accompanied the rest of the ten miles.
