Talk - Clustered, Distributed File and Volume Storage with GlusterFS

Using GlusterFS to provide volume storage to Kubernetes as a replacement for our existing file and static content hosting. This talk was given at Infracoders on Tuesday 14th November 2017. NOTE: Below link to slides currently broken - will fix soon! (03/08/2019) Click below to view slides (PDF version): Direct download link

November 14, 2017 · 1 min · 52 words · Sam McLeod

Applying syntax in Sublime based on the first file line

In vim, you can add a comment at the top of files to set the syntax, e.g.: # vim: syntax=ruby In SublimeText there are many ways to detect syntax, one interesting approach I’ve recently found useful is to match on the top line in the file. For example, with Puppet there is a file called Puppetfile, it has no extension but it’s really Ruby syntax, so it’s useful to add linting incase you miss something simple like a , and break deployments. ...

October 25, 2017 · 1 min · 162 words · Sam McLeod

Broadcom, Or How I Learned To Start Worrying And Drop The Packet

Earlier this week we started the process to upgrade one of our hypervisor compute clusters when we encountered a rather painful bug with HP’s Broadcom NIC chipsets. We were part way through a routine rolling pool upgrade of our hypervisor (XenServer) cluster when we observed unexpected and intermittent loss of connectivity between several VMs, then entire XenServer hosts. The problems appeared to impact hosts that hadn’t yet upgraded to XenServer 7.2. We now attribute this to a symptom of extreme packet loss between the hosts in the pool and thanks to buggy firmware from Broadcom and HP. ...

October 13, 2017 · 6 min · 1240 words · Sam McLeod

GlusterFS

We’re in the process of shifting from using our custom ‘glue’ for orchestrating Docker deployments to Kubernetes, When we first deployed Docker to replace LXC and our legacy Puppet-heavy application configuration and deployment systems there really wasn’t any existing tool to manage this, thus we rolled our own, mainly a few Ruby scripts combined with a Puppet / Hiera / Mcollective driven workflow. The main objective is to replace our legacy NFS file servers used to host uploads / attachments and static files for our web applications, while NFS(v4) performance is adequate, it is a clear single point of failure and of course, there are the age old stale mount problems should network interruptions occur. ...

September 25, 2017 · 6 min · 1106 words · Sam McLeod

Return Of The RSS

Of all the tools for reading news and subscribing to software releases, I still find RSS the most useful. I use Feedly to manage my rss subscriptions and keep all my devices in sync, but instead of using the Feedly’s own client, I use an app called Reeder as the client / reader itself. Link: My Feedly RSS Feed Feedly RSS feed subscription management Features: Keyword alerts. Browser plugins to subscribe to (current) url. Notation and highlighting support (a bit like Evernote). Search and filtering across large numbers of feeds / content. IFTTT, Zapier, Buffer and Hootsuite integration. Built in save / share functionality (that I only use when I’m on the website). Backup feeds to Dropbox. Very fast, regardless of the fact that I’m in Australia - which often impacts the performance of apps / sites that tend to be hosted on AWS in the US as the latency is so high. Article de-duplication is currently being developed I believe, so I’m looking forward to that! Easy manual import, export and backup (no vendor lock-in is important to me). Public sharing of your Feedly feeds (we’re getting very meta here!). Reeder A (really) beautiful and fast iOS / macOS client ...

September 22, 2017 · 2 min · 283 words · Sam McLeod

The State of Android in 2016 & The OnePlus 3 Phone

I wanted to try Android for a couple of weeks, I like staying on top of technology, gadgets and making sure I never become a blind ‘zealot’ for any platform or brand. The OnePlus 3 I did a lot of research and decided to try the “Oneplus 3” as it was good bang-for-buck, ran the latest software had plenty of grunt with the latest 8 core, high clock speed Qualcomm processor coupled with 6GB of DDR4 - the specs really are very impressive, especially for a $400USD phone. ...

July 11, 2016 · 4 min · 721 words · Sam McLeod

Update Delayed Serial STONITH Design

note: This is a follow up post from 2015-07-21-rcd-stonith A Linux Cluster Base STONITH provider for use with modern Pacemaker clusters This has since been accepted and merged into Fedora’s code base and as such will make it’s way to RHEL. Source Code: Github Diptrace CAD Design: Github I have open sourced the CAD circuit design and made this available within this repo under CAD Design and Schematics Related RedHat Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1240868 v1 vs v2/v3 versions of the rcd_serial STONITH system The v2/v3 cables include the following improvements: ...

July 4, 2016 · 2 min · 217 words · Sam McLeod

Monitoring SystemD Units With Nagios

Ever forgotten to add a critical service to monitoring? Want to know if a service or process fails without explicitly monitoring every service on a host? …Then why not use SystemD’s existing knowledge of all the enabled services? Thanks to ‘Kbyte’ who made a simple Nagios plugin to do just this! Requirements Python3 (For RHEL/CentOS 7 yum install python34) python-nagiosplugin My pre-built RPMs or pip3 install nagiosplugin PyNagSystemD

May 23, 2016 · 1 min · 68 words · Sam McLeod

Online Conversion from SQL_ASCII to UTF8 in PostgreSQL

Scripts and source available here: sql_ascii_to_utf8 The Goal To be able to take a Postgres Database which is in SQL_ASCII encoding, and import it into a UTF8 encoded database. Requirements: Python3 (For RHEL/CentOS 7 yum install python34) python-nagiosplugin My pre-built RPMs or pip3 install nagiosplugin PyNagSystemD The Problem PostreSQL will generate errors like this if it encounters any non-UTF8 byte-sequences during a database restore: # pg_dump -Fc test_badchar | pg_restore -d test_badchar_utf8 pg_restore: [archiver (db)] Error while PROCESSING TOC: pg_restore: [archiver (db)] Error from TOC entry 2839; 0 26852 TABLE DATA table101 postgres pg_restore: [archiver (db)] COPY failed for table "table101": ERROR: invalid byte sequence for encoding "UTF8": 0x91 CONTEXT: COPY table101, line 1 WARNING: errors ignored on restore: 1 And the corresponding data will be omitted from the database (in this case, the whole table, even the rows which did not have a problem): ...

May 23, 2016 · 5 min · 902 words · Sam McLeod

Speeding Up rsync

The most common way to use rsync is probably as such: rsync -avr user@<source>:<source_dir> <dest_dir> Resulting in 30-35MB/s depending on file sizes This can be improved by using a more efficient, less secure encryption algorithm, disabling compression and telling the SSH client to disable some unneeded features that slow things down. With the settings below I have achieved 100MB/s (at work between VMs) and over 300MB/s at home between SSD drives. ...

May 3, 2016 · 2 min · 224 words · Sam McLeod