April Progress Update

It has been a long time since I updated this blog and there are a lot of changes to discuss, so I shall provide a brief summary and then you can read whatever interests you. Otherwise, I hope you are staying safe and well in these difficult times!

23rd March: RescindentArchives returned with the 0.4.0h release
30th March: Archive server drive replacement 1
6th April:

• Archive server drive replacement 2
• Broadcaster server CPU and RAM upgrade

13th April: Archive server drive replacement 3

Wait- What about 0.4.1?

0.4.1 is releasing on 20th April, with support for playing audio on select videos and numerous enhancements. Many extra changes were required to bring this version to fruition, more discussion on that next week.

RescindentArchives returns and 0.4.0h

It was with some great relief that my family and I found somewhere to settle down for a number of months, not a moment too soon given the situation worldwide. This gave me reason to bring the stream back online, but before that could happen I had to solve several problems:

1. Setup the stream on a much less powerful server
2. The new server I had with me would not run 0.4.0g
3. Streamlabs OBS was very unstable on the new server

The first problem was solved with time and the second did not go away so I had to abandon the 0.4.0g build I had and fix the issues with a fresh build, 0.4.0h. This would not have been possible without the code residing in a Version Control System (VCS), Git in my case. In the months since I last streamed, back in late January, it would seem Streamlabs OBS became more unstable and kept dropping every few hours. Instability of this sort is something I strive to avoid, so I looked to OBS Studio for an answer- with the WebSocket package it ran without incident and was easier to integrate with than Streamlabs.

So all fine, the stream was back, correct? Not quite. The new server was an old late 2015 Macbook Pro running Windows 10 and could handle the encoding for hours before thermal throttling kicked in and meant even 720p at 30 FPS would cripple it. I was fortunate to have brought my group of servers along and stored them, so I dug out the old server, wired it up, installed OBS Studio and ran 0.4.0h. RescindentArchives was back!

Archive Server drive replacement

The Archive Server is the machine holding my archived video, holding it on a ZFS volume with multiple mirrors of 2 disks. This was setup in April 2017 with some new Western Digital Black Hard Disk Drives (HDDs), so it has been three years since they began spinning 24/7. Three years is a long time in a HDDs lifetime, as the excellent reports by BackBlaze attest to, so I decided to swap out a drive in each mirror to reduce the chances of data loss in the near future. Replacing each drive takes around 8 hours, which is why I had to take the stream down each Monday to facilitate the migration.

There are other drive setups, RAIDZ2 and RAIDZ3, that can reduce the capacity penalties of mirroring, but they can only handle 2 or 3 drive failures respectively. There are some excellent blogs out there endlessly debating the merits of different array setups and I opted for mirroring since the worst case is I lose some data rather than all of it.

Broadcast Server CPU and RAM upgrade

Upgrading the Broadcast Server was an unscheduled change, spurred on by the discovery of an old AM3+ motherboard with 16GB RAM and an FX-8350. All I needed was a new CPU cooler, which arrived a day early (thanks Amazon!).