Avatar

Starbeamrainbowlabs Tjovik

About

A computer science student who loves to explore and learn new things.

Blog posts

PhD Aside: Reading a file descriptor line-by-line from multiple Node.js processes

Phew, that's a bit of a mouthful. We're taking a short break from the cluster series of posts (though those will be back next week I hope), because I've just run into a fascinating problem, the solution to which I thought I'd share here - since I didn't find a solution elsewhere on the web. For my PhD, I've got a big old lump of data, and it all needs preprocessing before I train an AI model (or a… Read more →


Pure CSS spoilers with the CSS :target selector

For 1 reason or another, I've been working on some parser improvements for Pepperminty Wiki recently. Pepperminty Wiki uses Markdown for the page content syntax - specifically Parsedown. Markdown has a number of variations and extensions, some of which are more widely accepted than others. For Pepperminty Wiki, I try to stick as closely to existing Markdown conventions as possible (such as the Com… Read more →


Cluster, Part 7: Wrangling... boxes? | Expanding the Hashicorp stack with Docker and Nomad

Welcome back to part 7 of my cluster configuration series. Sorry this one's a bit late - the last one was a big undertaking, and I needed a bit of a rest :P Anyway, I'm back at it with another part to my cluster series. For reference, here are all the posts in this series so far: Cluster, Part 1: Answers only lead to more questions Cluster, Part 2: Grand Designs Cluster, Part 3: Laying groundwork… Read more →


Cluster, Part 6: Superglue Service Discovery | Setting up Consul

Hey, welcome back to another weekly installment of cluster configuration for colossal computing control. Apparently I'm managing to keep this up as a weekly series every Wednesday. [Last week]() (TODO LINK HERE), we sorted out managing updates to the host machines in our cluster to keep them fully patched. We achieved this by firstly setting up an apt caching server with apt-cacher-ng. Then we con… Read more →


Pipes, /dev/shm, or a TCP socket: Which is faster?

I've been busy patching HAIL-CAESAR (a simplified 2D flood simulation program designed for HPC supercomputers) to make it more suitable for the scale of my PhD project, and as part of this I'm trying to use the standard input & output where possible to speed up data transfer for the pre and post-processing steps, since I need to convert the data to and from different formats. As part of this, it c… Read more →


Cluster, Part 5: Staying current | Automating apt updates and using apt-cacher-ng

Hey there! Welcome to another cluster blog post. In the last post, we looked at setting up a WireGuard mesh VPN as a trusted private network for management interfaces and inter-cluster communication. As a refresher, here's a list of all the parts in this series so far: Cluster, Part 1: Answers only lead to more questions Cluster, Part 2: Grand Designs Cluster, Part 3: Laying groundwork with Unboun… Read more →


Cluster, Part 4: Weaving Wormholes | Peer-to-Peer VPN with WireGuard

(Above: The WireGuard and wesher logos. Background photo: from Unsplash by Clint Adair.) Hey - welcome back! Last week, we set Unbound up as our primary DNS server for our network. We also configured cluster member devices to use it for DNS resolution. As a recap, here are links to the all the posts in this series so far: Cluster, Part 1: Answers only lead to more questions Cluster, Part 2: Grand… Read more →


Document your network infrastructure with a wiki

Recently, I've realised that I actually manage quite an extensive network of interconnected devices. Some of them depend on each other in a non-obvious fashion, so after reading a comment on Reddit I decided to begin the process of documenting it all on a wiki. If you've followed this blog for a while, then you'll probably know that I'm the author of a wiki engine called Pepperminty Wiki. If you'r… Read more →


Cluster, part 3: Laying groundwork with Unbound as a DNS server

(Above: A picture from my wallpapers folder. If you took this, comment below and I'll credit you) Welcome to another blog post about my cluster! Although I had to replace the ATX PSU I was planning on using to power the thing with a USB power supply instead, I've got all the Pis powered up and networked together now - so it's time to finally start on the really interesting bit! In this post, I'm g… Read more →


Measuring maximum RAM usage with Bash on Linux

While running a simulation on my University's Viper HPC, I found that I needed to measure the maximum RAM usage of a simulation that I was running. Since the solution wasn't particularly easy to find, I thought I'd quickly blog about it here. Doing this isn't actually as easy as you might think. In the end, I used this: /usr/bin/time --format 'Max RAM working set size: %Mk' command_here --foo bar… Read more →