A few lessons learned from tracking The Queue

Just over a week ago, I got a call from a friend at DCMS asking for ideas on how they might help people to find the end of the queue for Her Majesty the Queen’s Lying-in-State. They had an interesting plan to livestream information to YouTube, and wanted to include some kind of live map alongside some dynamic public guidance. But how to get the data back and plot it realtime? Google (because if you’re DCMS, you just talk straight to Google…) didn’t have a ready solution since Google Maps doesn’t really have realtime pin movements. DCMS had an ingenious plan to use My Maps and shared location, with someone beaming back their location from the ground. But that would risk breaking if mobile signal dropped, or the phone battery died… and would need someone with the special phone at the back of the queue around the clock.

I thought I’d have a fiddle around in Google Maps’ Static API and see if I could come up with something more robust and less onerous that might work within their free tier.

It didn’t seem like realtime information was literally needed here – a marshall in the queue would be reporting locations every hour or so maybe, and there’s only so fast a queue will really move. So the challenge was to show the current position of the queue, styled up clearly in a way that would show up in a two-pane livestream, and refresh it whenever a new location was reported.

From proof of concept to launch was about 3 days. The map itself was built as a single script PHP application using a flatfile database to store the lat/lon co-ordinates of the end of the queue, and a KML file of the planned maximum queue path. A simple management app with a big purple button designed to run on a phone would be used by the marshall at the end of the queue to use the HTML geolocation API built into modern browsers to get their position, preview it on a map to check, and then update the map.

Google Maps Static API meant the map itself was pretty simple – literally just a square image – with couple of custom marker pins, and a chunky line drawn to plot the queue, drawing it out following the KML file up to the point closest to the last reported queue location. The map page would refresh every 3 minutes to show the latest queue path and end point.

DCMS’ plan was then to stream this map via the open source OBS software, alongside a text slide which they’d keep updated with public information messages. The beauty of their plan was that YouTube became the content delivery network handling the load and reducing a lot of the security risk to the app – and they just needed a single machine in the office broadcasting the map and accompanying text.

Some users understandably questioned why the information was being put out in this relatively data-intense way, but it made a lot of sense: not least because users are so familiar and comfortable with YouTube and media (not to mention GOV.UK) will happily embed YouTube videos when embedding maps or linking to external sites is trickier. Plus, it emerged from social media that people found watching the livestream of the queue tracker – a bit like the livestream of the lying-in-state itself – intrinsically settling, and a way to participate during a shared period of national mourning.

I cannot stress how much a part of the whole experience this was for those of us who couldn’t even conceive flying over to join the Queue. It added pizzaz and quirkyness, and allowed me to point to my Spanish colleagues about how seriously we are taking all of this. Thank you!

— Moof! Now mainly on Mastodon. (@Moof) September 20, 2022

Lesson 1: Automate whatever you can

The map system automated the process of updating a map, but in the first few hours social media critics started to point out human errors in the text updates being made to the livestreamed data. Behind every minor error is usually a small team doing its best under huge pressure, but in this case we figured we could automate a bit more of the process, using the what3words API (which is free and unlimited for lat/lon -> w3w lookups) to help reduce transcription errors.

A day or so into the livestream, we switched up the two panel system so the little web app was actually driving all the graphics, refreshing the map alongside a panel showing automated what3words location, checking the position of the nearest London landmark to the end of the queue by lat/lon, and introducing a mini content management system to manage text updates within the frame. This meant the system needed less manual intervention from a sleep-deprived team.

Lesson 2: …but incorporate manual overrides

Reporting the end of the queue location from a marshall on the ground worked great in testing… on personal phones with location enabled. When reality bit in the first couple of hours and a marshall tried reporting location from an official device (which had geolocation disabled), things didn’t work as planned. Thankfully I’d incorporated enough manual slack in the little app that someone in the office could override the co-ordinates, or fix the queue end point to a known point on the planned queue path.

Lesson 3: Think a step ahead

If I’d thought a bit more about what the team might need as the queue predictably hit maximum length, things might have been a bit less of a scramble at times and deploying code changes to production with very light testing might have been less intimidating (deploying code being livestreamed to 12k viewers and the full glare of Twitter and global media is a special feeling of dread!). Essentially, we ended up constructing a mini-CMS on the fly – for instance, bits of data that started as a simple flatfile text database, had to be migrated into JSON when I realised we would need to store settings alongside strings.

Lesson 4: Keep logs

One good decision I made early on was to log updates to a human-readable log file. That helped us quickly sort out problems with updating locations, but also extract a nice timestamped dataset of how the queue location changed over time, plus an audit trail if needed showing how and when messages were updated. Having that logfile gave me important peace of mind when deploying such a fast-changing app.

In what became a bit of a whirlwind week, it was a privilege to be able to support the creative, ambitious and dedicated team  at DCMS delivering what turned out to be a fast-moving digital service with even bigger global reach than we’d imagined. Huge credit to Andrew Simpson, Stuart Livesey, Joe Folan, Lucy Foster and Andy Dangerfield for their vision and hard work pulling everything together, and navigating hurdles I can only imagine.

I didn’t get to The Queue myself, and as an ex-civil servant still working out what I do as a postbureaucrat, I think on a personal level working on the Queue Tracker has been my contribution to saying thank you and goodbye to the Queen.

In many ways, it was indeed the most bonkersly British bit of internet ever invented.