smcleod.net 👋

NVApi is a small application I’ve written for monitoring and presenting utilisation metrics from Nvidia GPUs. This can be used to monitor GPU memory, temperature, power usage, and utilisation of GPUs in a system and can easily be integrated into tools such as HomeAssistant or Prometheus. The package uses the Nvidia Management Library (NVML) and provides a simple API for monitoring Nvidia GPUs along with a basic GUI client. NVApi on Github ...

The Enterprise “Agile” Software Development Manifesto We are pretending to uncover better ways of developing software by talking about it and making others believe we’re doing it. Through this facade, we have come to value: Processes and tools over individuals and interactions We believe in the almighty power of bureaucracy and red tape. Who needs skilled individuals when you have a convoluted process that nobody understands? Interactions are overrated; just follow the process blindly, and everything will magically work out....

SuperPrompter is a Python-based application that utilises the SuperPrompt-v1 model to generate optimised text prompts for AI/LLM image generation (for use with Stable Diffusion etc…) from user prompts. See Brian Fitzgerald’s Blog for a detailed explanation of the SuperPrompt-v1 model and its capabilities / limitations. Features Utilises the SuperPrompt-v1 model for text generation. A basic (aka ugly) graphical user interface built with tkinter. Customisable generation parameters (max new tokens, repetition penalty, temperature, top p, top k, seed)....

Two of my most commonly used LLM tools are Ollama and LM Studio. Unfortunately they store their models in different locations and filenames. Manually copying or linking files was a pain, so I wrote a simple command-line tool to automate the process. This is why I created Llamalink. Ollama is a cross-platform model server that allows you to run LLMs and manage their models in a similar way to Docker containers and images, while LM Studio is a macOS app that provides a user-friendly interface for running LLMs....

This is a high level intro to LLMs that I’m writing for a few friends that are new to the concept. It is far from complete, definitely contains some errors and is a work in progress. This is a work in progress and a living document. Language models, or LLMs, are a type of artificial intelligence that can generate text based on a given prompt. They work by learning patterns in large amounts of text data and using those patterns to generate new text....

Offline AI / LLM Assistant More info on this soon but the basic idea was to use Willow, Home Assistant and local LLM models to create a locally hosted, offline, AI powered Siri replacement and interface it with ESP32 S3 Box 3 devices. ...

...

“MBA Washing” refers to the phenomenon where individuals, often with a strong theoretical or academic background but limited recent practical experience, adopt and reinterpret industry-specific terminology and cultural movements. – This reinterpretation is typically influenced by their academic learning, recent literature, and biases towards larger enterprise perspectives. As a result, the original intent and practical effectiveness of these concepts, such as DevOps or Platform Engineering for example, may be diluted or misrepresented. ...

A talk I gave to some peers on creating your own SDXL LoRA models from my tinkering around over the last few weeks. ...

Recently, after replacing my home server I noticed that the CPU (Ryzen 7600) was only scaling between 3000MHz and 3800MHz, which is the base and the first level boost clock of the CPU. I was expecting it to scale down to as low as 400Mhz when idle, and up to 5.17Ghz on boost. ...

Before adding this to my shell config, I would manually add –depth=1 to all my git clones. ...

I’m tired of getting emails from companies asking me to rate their products and services It feels like you can’t even buy a coffee these days without being asked if you would recommend the coffee shop to a friend or colleague. ...

Many organisations - while branding themselves as “Agile” - continue to make the mistake of estimating project velocity based on time than effort. ...

...

Deployment / Delivery Frequency I often end up needing to advocate for more frequent delivery/deployments with clients. There’s the usual benefits commonly discussed such as improved feedback, reduced risk, well understood processes, maintainable dependencies etc… however what’s often missed entirely is how it relates to the health and sustainability of the team. ...

...

Github Actions is a “BYOBE” (Bring Your Own Bloody Everything) offering that provides basic CI with surprisingly convoluted configuration to Github. The product as a whole is an exercise in frustration, one of the worst parts is the lack of reusability and the complexity required to achieve it. Github’s concept of reusable workflows on Github Actions is clearly a cobbled together afterthought. ...

The YAML Spec defined a simple way to reuse values in YAML documents by means of Anchors and Aliases. ...

As of 2022, I’ve been using zsh as my primary shell for 14 years. Over that time I’ve experimented with and accumulated a lot of configuration, scripts, hacks, plugins and themes. I’ve settled on a configuration that I’m mostly happy with, over this series I’m going to share my current setup. ...

Contributing to Open Source is important to the quality and maintainability of the software and engineering communities we rely on every day - so why is it that so many developers/engineers never participate? ...

We create more value by having conversations in public instead of behind closed doors. ...

Near the end of each year I note down a summary of the best apps I’ve enjoyed using throughout the year, here’s 2022. ...

We’re (still) not deploying enough It’s 2022 and not deploying frequently enough is still one of the most common causes of software failure. ...

Over the past month I’ve been playing with OpenAI’s DALL·E 2, below are some of the interesting images I’ve generated. “DALL·E 2 is a new AI system that can create realistic images and art from a description in natural language.” “A renaissance oil painting of two developers arguing over which javascript framework is the worst” “A Ralph Steadman painting of 7 cats sitting on a fence, and one raven” “A poster of Bruce Lee posing in front of a wall painted with traditional Japanese dragon artwork” “An Art Nouveau painting of Batman outside a church” “Old man yells at cloud” Variations on “A romantic oil painting of a chicken wearing beats by dre and smoking a cigarette” “A Romantic oil painting of a chicken wearing ripped jeans, aviator sunglasses, smoking a cigarette leaning up against a wall” “A romantic painting of snoop dog and a cool chicken wearing ripped jeans and aviators standing in a recording booth rapping into a microphone” “A dutch masters style oil painting of the rapper snoop dogg rapping into a microphone along side a cool looking animal wearing ripped jeans” “An oil painting of the rapper snoop dogg as a marvel super hero whose super power is smoking” “A romantic style oil painting of the rapper snoop dogg as a marvel super hero whose super power is smoking” “A detailed psychedelic painting of a person listening to the band TOOL for the first time and having their third eye opened” “A white Romantic Sculpture of the dude from the film the big lebowski” “A dutch masters oil painting of a robot cat” “A red Tesla model 3 crashed on the side of the road due to it’s poorly designed suspension” “A kiwi walking over hot coals” “A Ralph Steadman style ink painting of cats sitting on a fence smoking” “Two chickens playing chess while smoking” “Cats get dressed up to go to the theatre” “A grumpy cat drinking black coffee and smoking a cigarette” “A female artist with brown and grey hair hangs white deer antlers in an art gallery” “A black and white cat and a boarder collie dog in the style of Grayson Perry” “A lonely cat by a window in a Ralph Steadman style” “A post-apocalyptic city at night ruled by cats” “A border collie dog and black and white cat in the style of Gustav Klimt” “DevOps hipsters deploy to the cloud” “A psychedelic monkey smoking” “Cats smoking cigars and playing poker” “An elite hacker breaks into the computer mainframe, his eyes are red, his beard is long, he is in a dark room at his desk” “Cats captaining space ships with lasers” “Two lamas duelling with lightsabers at a disco” “A dark, detailed oil painting of a post-apocalyptic city a night, two terminator cyborgs are patrolling the street” “A photo of a beautiful British field in the summer, there are butterflys, birds and a small pond....

Beginner Harmonica Practice Exercises Way back in around 2008 I wrote up a doc with a list of beginner harp exercises and practice riffs, at the time it was uploaded to a few forums and a shared Google doc - both have since disappeared. To this day I still get regular emails asking for a copy of it so I thought I’d copy it to a post here. Note: I haven’t updated these since around 2011....

Firefox Addons - 2022 Edition My list of must-have Firefox addons - 2022 edition Updated: 2022-11-07 Privacy and Security Firstly - you should have Firefox’s Enhanced Tracking Protection enabled. Don’t Track Me Google LocalCDN UTM Tracking Token Stripper Note: You can accomplish some of what this does by setting up the removeparam uBlock origin rules I’ve listed below. Multi-Account Containers Useful for setting sites such as Amazon, eBay, Twitter, LinkedIn, Banking etc… each to always open in their own isolated container....

Until recently I used Evernote for my notes, web clippings and document drafts, I was a subscriber going on 14 years and had over 3000 notes. However in recent years Evernote has gone downhill to the point it could be considered hostile to it’s users. The native macOS app was replaced with an Electron Javascript webframe and like almost all Electron apps I’ve tried it was plagued by poor performance, memory leaks and aggressive idle CPU utilisation and above all else it now lacks the low latency feeling the native application had....

Weekend upgrades to my Ranchilio Silvia v4 espresso machine. Added a digital PID kit for improved temperature control Replaced the boiler element Thermal insulation Basic “smart” power control with Siri integration Upgrade the standard Ranchilio basket to a VST 18g Ridged Rebuild Photos Note the damage to the boiler elements terminals, this was shorting and causing the houses RCD to trip Wiring the PID controller Replacing the boiler thermostat with one connected to the PID controller...

A template team charter for software and platform engineering teams. Research1 shows there is great value in people embracing a shared vision and ideas that are bigger than themselves. When it comes to a team charter, it’s important to remember that it’s not a static document. It’s a living document that should be updated as the team evolves and grows. Keep in mind that One size doesn’t fit all. Consider your (teams) mission, cultural background and values....

Last weekend I had a chance encounter with a Ferrari F12 Berlinetta and the owner kindly let me jump in and take a few photos. Specs 6.3L v12 544 KW (740 Horsepower) 690 Nm at 6000RPM 1525 kg 0-100 in 3.1s 0-200 in 8.5s Maximum speed: ‘over 340 km/h’ Was ‘The fastest road car Ferrari ever built’ Photos

Near the end of every year I note down a summary of the best apps, hardware & podcasts I’ve enjoyed throughout the year (and often for some time before). This post has been superseded. You can find the latest version of this post here. Software and Services Plex Things Pixelmator Pro Reeder NextMeeting Bumpr Fastmail 1Password Calibre Backblaze iTerm Gitlab PFsense Stay Onyx Evernote Handbrake Amphetamine Firefox Software - Mobile Plex Things Reeder Overcast Evernote MiniHack Alien Blue Singal Prompt Browser Addons uBlock Origin Dencentraleyes Hardware LG C9 OLED TV Das Ultimate 4 Logitech MX Master 3 16’’ Macbook Pro iPhone 12 Pro AppleTV 4K Audioquest DragonFly Red USB DAC Ultrasone Signature Pro Headphones Kindle Paperwhite All Day Socks Bellroy Wallets Podcasts The Skeptics Guide To The Universion (SGU) You Are Not So Smart Risky....

You magnificent girl, you brought me 14 years of love and happiness Photos

Stop trying to hire with titles like ‘DevOps Engineer’ or ‘Cloud Engineer’ “DevOps is … not a job title” “It’s more of a cultural practice, like innovation, and it makes just as little sense hiring “innovation engineers” and expecting your organisation to be innovative, without also creating the culture to foster innovation.” (Joel Shea when responding to this posted on LinkedIn) In most cases it’s clear the organisation doesn’t truly know what they want or need and likely don’t understand the nuances of the aspects of engineering....

As stated in my announcement post - after 7~ years, I resigned from Infoxchange several months ago. I planned on taking a holiday, relaxing, decompressing, dipping my toes in some off-the-shelf tech I haven’t spent a lot of time with and performing some personal growth through reading a number of books that have been sitting on my ’to read’ shelf. This is exactly what I’ve done. I’m now open to new opportunities I’d rather find a workplace that I can add great value too and be happy to work with than rush into a new role quickly, obviously if something great turns up right away - I’ll pursue it....

After 7~ years, I have resigned from Infoxchange. In 2012 I was hired to work with Infoxchange to “shake things up” (in the Systems Operations team) and stabilise the hosting environment. As Team Lead of Systems Operations I was tasked with melding the team and then for us to work to provide a stable, robust, modern and scalable platform for Infoxchange developed application hosting and product delivery and in a short time, with a small budget - we did just that....

In 2018 I set out to replace our XenSever 7.2 based virtualisation after Citrix essentially screwed over free / open source users. This project was to directly replace XenServer 7.2 with something supported and manageable for our traditional virtualisation needs. High Level Selection Considerations I evaluated a number of options, with the primary candidates below. Key criteria (at a high level) I was evaluating: Ease of moving from our existing XenServer 7....

In late November of 2018 I was lucky enough to get my hands on the upcoming 2019 Sport spec Mustang GT v8 to be its first driver. I’m not usually an American Muscle enthusiast, but the new 2019 GT really is quite different from its slightly older counterparts, with a new specially developed 10 speed gearbox (yes, 10 gears - because more is better right?), tuned up suspension and so seemed like it would make for a pretty fun trip....

I wanted an alias or function to use youtube-dl in the background. Looking around the web lots of people seemed to want this and most of them were banging their heads against a wall due to: A) bash quoting B) backgrounding dying when their terminal closed C) passing the argument (url in my case) to the function Here’s a simple function I whipped up that seems to ‘just works’™: function yt { nohup youtube-dl "$1" --no-progress 2>&1 > youtube-dl-"$(date +%Y%m%d-%H%M%S)"....

This feature annoys me endlessly, I end up zoomed in and out of websites all over the internet. … But the fix is easy and there’s no addons required. Navigate to about:config (in Firefox’s URL bar) Change the value of the following two properties to 0: mousewheel.with_control.action mousewheel.with_meta.action If you use Firefox sync and want these settings to sync between your machines, also add the following properties: Create two new properties both of type boolean and set them to true:...

Last weekend I headed to New Zealand and marshalled at the Highlands Festival of Speed. The Highlands Festival of Speed is a weekend packed with some of the best modern classic racing you will see in New Zealand. Featuring the Pre 65 Racing, Mainland Muscle, Formula Libre, South Island Porsche, OSCA, Highlands Modern Classics/Nostalgic Classics and the Highlands Sprint Series. This is a weekend of true blue Kiwi racing starring some of New Zealand’s most well known racing legends....

It was almost 3 years ago that my open source storage project went into production. In that time it’s been running 24/7 serving as highly available solid state storage for hundreds of VMs and several virtualisation clusters across our two main sites. I’m happy to report that the clusters have been operating very successfully since their conception. Since moving away from proprietary ‘black box’ vendor SANs, we haven’t had a single SAN issue, storage outage....

“A chain is no stronger than its weakest link” Any improvement made anywhere besides the bottleneck is an illusion. Any improvement made after the bottleneck is useless because it will always remain starved waiting for work from the bottleneck. Any improvement made before the bottleneck merely results in more ‘work’ piling up at the bottleneck. Identify the system’s constraint(s) (that which prevents the organisation from obtaining more of the goal in a unit of time) Decide how to exploit the system’s constraint(s) (how to get the most out of the constraint) Subordinate everything else to the above decision (align the whole system or organization to support the decision made above) Elevate the system’s constraint(s) (make other major changes needed to increase the constraint’s capacity) Warning!...

Here are my top album pics for 2017 (in no particular order) Note: This is by no means an exhaustive list, it’s just the top albums that really stood out to me and in all fairness, I’ve thought of several others since so there may be a follow up post (or two). Father John Misty - Pure Comedy As a bonus, a short film was released with the album: From Wikipedia:...

My good friend Joel Shea received a most unlikely gift this Christmas - A vintage HP 4951 Protocol Analyser. According to the HP Computer Museum: Original Price: $3595 The 4951B was replaced by the 4951C and 4952A in 1986. Both new models handled Async, BSC, SDLC, HDLC, X.25 and SNA protocols. The 4951C also handled DDCMP, while the 4952A did not. The 4952A handled X.21 while the 4951C did not. Both new analysers used a floppy dive (618 KB) for removable media....

Using GlusterFS to provide volume storage to Kubernetes as a replacement for our existing file and static content hosting. This talk was given at Infracoders on Tuesday 14th November 2017. NOTE: Below link to slides currently broken - will fix soon! (03/08/2019) Click below to view slides (PDF version): Direct download link

In vim, you can add a comment at the top of files to set the syntax, e.g.: # vim: syntax=ruby In SublimeText there are many ways to detect syntax, one interesting approach I’ve recently found useful is to match on the top line in the file. For example, with Puppet there is a file called Puppetfile, it has no extension but it’s really Ruby syntax, so it’s useful to add linting incase you miss something simple like a , and break deployments....

Earlier this week we started the process to upgrade one of our hypervisor compute clusters when we encountered a rather painful bug with HP’s Broadcom NIC chipsets. We were part way through a routine rolling pool upgrade of our hypervisor (XenServer) cluster when we observed unexpected and intermittent loss of connectivity between several VMs, then entire XenServer hosts. The problems appeared to impact hosts that hadn’t yet upgraded to XenServer 7....

We’re in the process of shifting from using our custom ‘glue’ for orchestrating Docker deployments to Kubernetes, When we first deployed Docker to replace LXC and our legacy Puppet-heavy application configuration and deployment systems there really wasn’t any existing tool to manage this, thus we rolled our own, mainly a few Ruby scripts combined with a Puppet / Hiera / Mcollective driven workflow. The main objective is to replace our legacy NFS file servers used to host uploads / attachments and static files for our web applications, while NFS(v4) performance is adequate, it is a clear single point of failure and of course, there are the age old stale mount problems should network interruptions occur....

Of all the tools for reading news and subscribing to software releases, I still find RSS the most useful. I use Feedly to manage my rss subscriptions and keep all my devices in sync, but instead of using the Feedly’s own client, I use an app called Reeder as the client / reader itself. Link: My Feedly RSS Feed Feedly RSS feed subscription management Features: Keyword alerts. Browser plugins to subscribe to (current) url....

MH-Z19 CO2 sensor reader, logger and visualiser Reads data from UART(serial)-connected MH-Z19 (or MH-Z14) sensor using python 3. If you dare to install nodejs you can visualise the logged data (using html and plotly.js library). Repository: sammcj/CO2-Logger Usage Note this post is from 2016, in 2021 I replaced my custom Co2 loggers with an Aranet4. While very expensive, is and excellent off-the-shelf solution, with many features. Connection Sensor can be queried using 3....

I wanted to try Android for a couple of weeks, I like staying on top of technology, gadgets and making sure I never become a blind ‘zealot’ for any platform or brand. The OnePlus 3 I did a lot of research and decided to try the “Oneplus 3” as it was good bang-for-buck, ran the latest software had plenty of grunt with the latest 8 core, high clock speed Qualcomm processor coupled with 6GB of DDR4 - the specs really are very impressive, especially for a $400USD phone....

note: This is a follow up post from 2015-07-21-rcd-stonith A Linux Cluster Base STONITH provider for use with modern Pacemaker clusters This has since been accepted and merged into Fedora’s code base and as such will make it’s way to RHEL. Source Code: Github Diptrace CAD Design: Github I have open sourced the CAD circuit design and made this available within this repo under CAD Design and Schematics Related RedHat Bug: https://bugzilla....

Ever forgotten to add a critical service to monitoring? Want to know if a service or process fails without explicitly monitoring every service on a host? …Then why not use SystemD’s existing knowledge of all the enabled services? Thanks to ‘Kbyte’ who made a simple Nagios plugin to do just this! Requirements Python3 (For RHEL/CentOS 7 yum install python34) python-nagiosplugin My pre-built RPMs or pip3 install nagiosplugin PyNagSystemD

Scripts and source available here: sql_ascii_to_utf8 The Goal To be able to take a Postgres Database which is in SQL_ASCII encoding, and import it into a UTF8 encoded database. Requirements: Python3 (For RHEL/CentOS 7 yum install python34) python-nagiosplugin My pre-built RPMs or pip3 install nagiosplugin PyNagSystemD The Problem PostreSQL will generate errors like this if it encounters any non-UTF8 byte-sequences during a database restore: # pg_dump -Fc test_badchar | pg_restore -d test_badchar_utf8 pg_restore: [archiver (db)] Error while PROCESSING TOC: pg_restore: [archiver (db)] Error from TOC entry 2839; 0 26852 TABLE DATA table101 postgres pg_restore: [archiver (db)] COPY failed for table "table101": ERROR: invalid byte sequence for encoding "UTF8": 0x91 CONTEXT: COPY table101, line 1 WARNING: errors ignored on restore: 1 And the corresponding data will be omitted from the database (in this case, the whole table, even the rows which did not have a problem):...

The most common way to use rsync is probably as such: rsync -avr user@<source>:<source_dir> <dest_dir> Resulting in 30-35MB/s depending on file sizes This can be improved by using a more efficient, less secure encryption algorithm, disabling compression and telling the SSH client to disable some unneeded features that slow things down. With the settings below I have achieved 100MB/s (at work between VMs) and over 300MB/s at home between SSD drives....

This is a quick tldr there are many other situations and options you could consider FIO man page IOP/s = Input or Output operations per second Throughput = How many MB/s can you read/write continuously Variables worth tuning based on your situation --iodepth The iodepth is very dependant on your hardware. Rotational drives without much cache and high latency (i.e. desktop SATA drives) will not benefit from a large iodepth, Values between 16 to 64 could be sensible....

Let’s pretend you have a project on Gitlab called ask-izzy and you want to mirror it up to Gitlab which is located at https://github.com/ask-izzy/ask-izzy Assuming you’re running Gitlab as the default user of git and that your repositories are stored in /mnt/repositories you can following something similar to the following instructions: Grant write access to Github Get your Gitlab install’s pubkey from the git user cat /home/git/.ssh/id_rsa.pub On Github add this pubkey as deploy key on the repo, make sure you tick the option to allow write access....

Today we launched a mobile website for homeless people … and it was launched by one of Australia’s many recent Prime Ministers Today alone we served up over 87,000 requests As many of you know, I work with Infoxchange as the operations lead. When I first heard the idea of a website or app for people that have found or are worried about finding themselves homeless in Australia I really didn’t think it made sense - until I saw the stats showing how many homeless people in Australia have regular access to a smart phone and data either via a cellular provider or free WiFi....

If a disk / VDI is orphaned or only partially deleted you’ll notice that under the SR it’s not assigned to any VM. This can cause issues that look like metadata corruption resulting in the inability to migrate VMs or edit storage. For example: [root@xenserver-host ~]# xe vdi-destroy uuid=6c2cd848-ac0e-441c-9cd6-9865fca7fe8b Error code: SR_BACKEND_FAILURE_181 Error parameters: , Error in Metadata volume operation for SR. [opterr=VDI delete operation failed for parameters: /dev/VG_XenStorage-3ae1df17-06ee-7202-eb92-72c266134e16/MGT, 6c2cd848-ac0e-441c-9cd6-9865fca7fe8b. Error: Failed to write file with params [3, 0, 512, 512]....

“Having a SCSI ID is a f*cking idiotic thing to do.” - Linus Torvalds …and after the amount of time I’ve wasted getting XenServer to play nicely with LIO iSCSI failover I tend to agree. The Problem One oddity of Xen / XenServer’s storage subsystem is that it identifies iSCSI storage repositories via a calculated SCSI ID rather than the iSCSI Serial - which would be the sane thing to do....

Slides Failover Demo

In an attempt to avoid using the Juniper Pulse (Now Pulse Secure) VPN client we tried OpenConnect but found that DNS did not work correctly when connected to the VPN. This bug has now been resolved recently but has not made it’s way into a new build, in fact there have been no releases for 6 months. Luckily the OpenConnect was not too difficult to build from source. Build OpenConnect on OSX Remove old openconnect and install deps brew remove openconnect brew install libxml2 lzlib openssl libtool libevent Build openconnect wget git....

Over the last two months I’ve been running selected IO intensive servers off the the SSD storage cluster, these hosts include (among others) our: Primary Puppetmaster Gitlab server Redmine app and database servers Nagios servers Several Docker database host servers Reliability We haven’t had any software or hardware failures since commissioning the storage units. During this time we have had 3 disk failures on our HP StoreVirtual SANs that have required us to call the supporting vendor and replace failed disks....

Set update channel to receive developer beta update sudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11seed-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz Set update channel to receive public beta update sudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz List available updates sudo softwareupdate --list Set update channel to receive default, stable updates sudo softwareupdate --clear-catalog Show current settings defaults read /Library/Preferences/com.apple.SoftwareUpdate.plist Write setting manually defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz

The following are benchmarks from our testings of our iSCSI SSD storage. 67,300 read IOP/s on a VM on iSCSI (Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM) Per VM and scales to 1,000,000 IOP/s total root@dev-samm:/mnt/pmt1 128 # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=128 --size=2G --readwrite=read test: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128 2.0.8 Starting 1 process bs: 1 (f=1): [R] [55....

A modified version of John Sutton’s rcd_serial cable coupled with our Supermicro reset switch hijacker: This works with the rcd_serial fence agent plugin. Reasons rcd_serial makes for a very good STONITH mechanism: It has no dependency on power state. It has no dependency on network state. It has no dependency on node operational state. It has no dependency on external hardware. It costs less that $5 + time to build. It is incredibly simple and reliable....

First some background… One of the many lessons I’ve learnt from my Linux HA / Storage clustering project is that the Debian HA ecosystem is essentially broken, We reached the point where packages were too old, too buggy or in Debian 8’s case - outright missing. In the past I was very disappointed with RHEL/CentOS 5 / 6 and (until now) have been quite satisfied with Debian as a stable server distribution with historicity more modern packages and kernels....

Due to several recent events beyond my control I’m a bit behind on the project - hence the lack of updates which I apologise for. The goods news is that I’m back working to finish off the clusters and I’m happy to report that all is going to plan. Here is the final digram of the two-node cluster design: Plain text version available here This was generated from the LCMC tool (beware - it’s java!...

A brief demonstration of the failover and recovery process on the storage clusters I’ve been building.

A high level talk from Infracoders Melbourne on 12/04/2015. There’s also a low quality recording available here: Related posts: Building a high performance SSD SAN - Part 1

Linux Kernel CI for Debian Github: sammcj/kernel-ci Those of us using technologies such as Docker and BTRFS or simply trying to gain a performance edge on the competition have a lot to gain from the features and performance of recent Kernel updates (especially from 3.18 onwards). ‘Enterprise’ Linux distributions such as RHEL & variants are concerningly out of date when comes to the Kernel. Many people seem to have forgotten what Linux is… Linux IS the Kernel....

Docker config to setup XO which is a web interface to visualize and administrate your XenServer (or XAPI enabled) hosts Github: sammcj/docker-xen-orchestra Running the app Updates are pushed to the Docker Hub’s automated build service: https://registry.hub.docker.com/u/sammcj/docker-xen-orchestra From Docker Hub docker pull sammcj/docker-xen-orchestra docker run -d -p 8000:80 sammcj/docker-xen-orchestra Building git clone https://github.com/sammcj/docker-xen-orchestra.git cd docker-xen-orchestra # Edit whatever config you want to change docker build -t xen-orchestra . See https://xen-orchestra.com for information on Xen Orchestra

Inspired by http://zitseng.com/archives/7489 Source (Github) WARNINGS Do not run unless you understand what this is doing The CA system is broken by design - This is not a fix for that This is merely a band-aid for those interested or concerned about these root CAs Usage chmod +x delete_gov_roots.sh ./delete_gov_roots.sh You’ll be prompted for your password as root access is required to delete system-wide root certs. See Also http://convergence.io https://addons.mozilla.org/en-US/firefox/addon/certificate-patrol/ https://github....

Over the coming month I will be architecting, building and testing a modular, high performance SSD-only storage solution. I’ll be documenting my progress / findings along the way and open sourcing all the information as a public guide. With recent price drops and durability improvements in solid state storage now is better time than any to ditch those old magnets. Modular server manufacturers such as SuperMicro have spent large on R&D thanks to the ever growing requirements from cloud vendors that utilise their hardware....

Further to my earlier post on XenServer storage performance with regards to directly attaching storage from the host, I have been analysing the performance of various SSD storage options. I have attached a HP DS2220sb storage blade to an existing server blade and compared performance with 4 and 6 SSD RAID-10 to our existing iSCSI SANs. While the P420i RAID controller in the DS2220sb is clearly saturated and unable to provide throughput much over 1,100MB/s - the IOP/s available to PostgreSQL are still a very considerably performance improvement over our P4530 SAN - in fact, 6 SSD’s result in a 39....

“Search - A Journey of Delivery on a Budget” Presented a Melbourne Search - July 2014 - “Search - A Journey of Delivery on a Budget” (Click for slides)

The way we work at Infoxchange has changed greatly. A retrospective journey into transforming Infoxchange’s technology and culture over the past 24 months - presented a Melbourne DevOps - December 2014

At the end of every year I note down a summary of the best applications, hardware & websites I’ve enjoyed & depended on throughout the year (and often for some time before). This post has long since been superseded. You can find the latest version of this post here. Software / General Use Fastmail - https://www.fastmail.com Evernote - https://evernote.com Reeder - http://reederapp.com Keynote - https://www.apple.com/au/mac/keynote Lastpass - https://lastpass.com Plex - https://plex....

Intro At Infoxchange we use XenServer as our Virtualisation of choice. There are many reasons for this including: Open Source. Offers greater performance than VMware. Affordability (it’s free unless you purchase support). Proven backend Xen is very reliable. Reliable cross-host migrations of VMs. The XenCentre client, (although having to run in a Windows VM) is quick and simple to use. Upgrades and patches have proven to be more reliable than VMware....

Reprogram BMW after doing Automatic to Manual Conversion Based on my E36 328is experience, written in 2010. Assumptions This guide assumes the following: You have programmed a BMW ECU/DME before using DIS or similar. You are taking all the necessary safety precautions. (Fully charged battery etc…) You have DIS working (We used EasyDIS 1.0, Base 44) You have Ediabas (INPA, NCS Expert, IFH Serve) installed and working. You’re able to read between the lines of our crappy document, some steps may differ slightly and our wording may be somewhat imprecise....