smcleod.net 👋

The two tools I use with AI / LLMs for generating diagrams are Mermaid and Excalidraw. Mermaid (or MermaidJS) is a popular diagramming library and format supported by many tools and is often rendered inside markdown (e.g. in a readme.md). Excalidraw is an excellent, free and open source diagramming and visualisation tool. I also often make use of a third party Obsidian plugin for Excalidraw. Excalidraw It has a ‘generate diagram with AI’ feature which if you’re using the excalidraw.com online editor offers a few free generations each day (I think this uses a low-end OpenAI model). If you’re running Excalidraw locally or by using the brilliant Obsidian plugin - you can provide any OpenAI compatible API endpoint and model for AI generations. Behind the scenes Excalidraw AI generates and then renders MermaidJS. ...

Ingest is a tool I’ve written to make my life easier when preparing content for LLMs. It parses directories of plain text files, such as source code, documentation etc… into a single markdown file suitable for ingestion by AI/LLMs. Ingest can also estimate vRAM requirements for a given model, quantisation and context length: Features Traverse directory structures and generate a tree view Include/exclude files based on glob patterns Estimate vRAM requirements and check model compatibility using another package I’ve created called quantest Parse output directly to LLMs such as Ollama or any OpenAI compatible API for processing Generate and include git diffs and logs Count approximate tokens for LLM compatibility Customisable output templates Copy output to clipboard (when available) Export to file or print to console Optional JSON output Ingest Intro (“Podcast” Episode): ...

Here’s a fun little tool I’ve been hacking on to explore the effects of different inference parameters on LLMs. You can find the code and instructions for running it locally on GitHub. It started as a fork of rooben-me’s tone-changer-open, which itself was a “fork” of Figma’s tone generator, I’ve made quite a few changes to make it more focused on local LLMs and advanced parameter exploration.

Code, Chaos, and Copilots is a talk I gave in July 2024 as an intro to how I use AI/LLMs to augment my capabilities every day. What I use AI/LLMs for Prompting tips Codegen workflow Picking the right models Model formats Context windows Quantisation Model servers Inference parameters Clients & tools Getting started cheat-sheets Download Slide Deck Disclaimer I’m not a ML Engineer or data scientist, As such, the information presented here is based on my understanding of the subject and may not be 100% accurate or complete. ...

AI models (“LLMs” in this case) have inherently large sizes and computational requirements that often pose challenges for deployment and use. ...

I spend a lot of time working with AI tools and often get asked for recommendations. The following is a list of some of the more notable tools I’ve tried and rated them based on my experience. Just because something has a low rating doesn’t mean it’s bad - it just means it didn’t work well for me and I wouldn’t personally recommend it. I plan on keeping this updated as I try new tools (we’ll see how that goes). ...

Gollama on Github Gollama is a client for Ollama for managing models. It provides a TUI for listing, filtering, sorting, selecting, inspecting (coming soon!) and deleting models and can link Ollama models to LM-Studio. The project started off as a rewrite of my llamalink project, but I decided to expand it to include more features and make it more user-friendly. ...

Confuddlement on Github I was tired of manually downloading Confluence pages and converting them to Markdown, so I wrote a small command-line tool designed to simplify this process. Confuddlement is a Go-based tool that uses the Confluence REST API to fetch page content and convert it to Markdown files. It can fetch pages from multiple spaces, skip pages that have already been fetched, and summarise the content of fetched pages using the Ollama API. $ go run ./main.go Confuddlement 0.3.0 Spaces: [COOLTEAM, MANAGEMENT] Fetching content from space COOLTEAM COOLTEAM (Totally Cool Team Homepage) Retrospectives Decision log Development Onboarding Saved page COOLTEAM - Feature List to ./confluence_dump/COOLTEAM - Feature List.md Skipping page 7. Support, less than 300 characters MANAGEMENT (Department of Overhead and Bureaucracy) Painful Change Management Illogical Diagrams Saved page ./confluence_dump/Painful Change Management.md Saved page Illogical Diagrams to ./confluence_dump/Ilogical Diagrams.md Done! $ go run ./main.go summarise Select a file to summarise: 0: + COOLTEAM - Feature List 1: + Painful Change Management 2: + Illogical Diagrams Enter the number of the file to summarise: 1 Summarising Painful Change Management... "Change management in the enterprise is painful and slow. It involves many forms and approvals." go run main.go -q 'who is the CEO?' -s 'management' -r 2 Querying the LLM with the prompt 'who is the CEO?'... "The CEO of the company is Peewee Herman." Usage Running the Program Copy .env.template to .env and update the environment variables. Run the program using the command go run main.go or build the program using the command go build and run the resulting executable. The program will fetch Confluence pages and save them as Markdown files in the specified directory. Querying the documents with AI You can summarise the content of a fetched page using the Ollama API by running the program with the summarise argument: go run main.go summarise To perform a custom query, you can use the query argument: -q: The query to to provide to the LLM. -s: The search term to match documents against. -r: The number of lines before and after the search term to include in the context to the LLM. go run main.go -q 'who is the CEO?' -s 'management' -r 2 Querying the LLM with the prompt 'who is the CEO?'... "The CEO of the company is Peewee Herman." ...

NVApi is a small application I’ve written for monitoring and presenting utilisation metrics from Nvidia GPUs. This can be used to monitor GPU memory, temperature, power usage, and utilisation of GPUs in a system and can easily be integrated into tools such as HomeAssistant or Prometheus. The package uses the Nvidia Management Library (NVML) and provides a simple API for monitoring Nvidia GPUs along with a basic GUI client. NVApi on Github ...

SuperPrompter is a Python-based application that utilises the SuperPrompt-v1 model to generate optimised text prompts for AI/LLM image generation (for use with Stable Diffusion etc…) from user prompts. See Brian Fitzgerald’s Blog for a detailed explanation of the SuperPrompt-v1 model and its capabilities / limitations. Features Utilises the SuperPrompt-v1 model for text generation. A basic (aka ugly) graphical user interface built with tkinter. Customisable generation parameters (max new tokens, repetition penalty, temperature, top p, top k, seed). Optional logging of input parameters and generated outputs. Bundling options to include or exclude pre-downloaded model files. Unloads the models when the application is idle to free up memory. Prebuilt Binaries Check releases page to see if there are any prebuilt binaries available for your platform. ...

Two of my most commonly used LLM tools are Ollama and LM Studio. Unfortunately they store their models in different locations and filenames. Manually copying or linking files was a pain, so I wrote a simple command-line tool to automate the process. This is why I created Llamalink. Ollama is a cross-platform model server that allows you to run LLMs and manage their models in a similar way to Docker containers and images, while LM Studio is a macOS app that provides a user-friendly interface for running LLMs. ...

Offline AI / LLM Assistant More info on this soon but the basic idea was to use Willow, Home Assistant and local LLM models to create a locally hosted, offline, AI powered Siri replacement and interface it with ESP32 S3 Box 3 devices. ...

“MBA Washing” refers to the phenomenon where individuals, often with a strong theoretical or academic background but limited recent practical experience, adopt and reinterpret industry-specific terminology and cultural movements. – This reinterpretation is typically influenced by their academic learning, recent literature, and biases towards larger enterprise perspectives. As a result, the original intent and practical effectiveness of these concepts, such as DevOps or Platform Engineering for example, may be diluted or misrepresented. ...

A talk I gave to some peers on creating your own SDXL LoRA models from my tinkering around over the last few weeks. ...

Recently, after replacing my home server I noticed that the CPU (Ryzen 7600) was only scaling between 3000MHz and 3800MHz, which is the base and the first level boost clock of the CPU. I was expecting it to scale down to as low as 400Mhz when idle, and up to 5.17Ghz on boost. ...

Before adding this to my shell config, I would manually add –depth=1 to all my git clones. ...

I’m tired of getting emails from companies asking me to rate their products and services It feels like you can’t even buy a coffee these days without being asked if you would recommend the coffee shop to a friend or colleague. ...

Many organisations - while branding themselves as “Agile” - continue to make the mistake of estimating project velocity based on time than effort. ...

This is a high level intro to LLMs that I’m writing for a few friends that are new to the concept. It is far from complete, definitely contains some errors and is a work in progress. This is a work in progress and a living document. Language models, or LLMs, are a type of artificial intelligence that can generate text based on a given prompt. They work by learning patterns in large amounts of text data and using those patterns to generate new text. LLMs can be used for a variety of tasks, such as generating chatbots, answering questions, and creating art. ...

Deployment / Delivery Frequency I often end up needing to advocate for more frequent delivery/deployments with clients. There’s the usual benefits commonly discussed such as improved feedback, reduced risk, well understood processes, maintainable dependencies etc… however what’s often missed entirely is how it relates to the health and sustainability of the team. ...

Github Actions is a “BYOBE” (Bring Your Own Bloody Everything) offering that provides basic CI with surprisingly convoluted configuration to Github. The product as a whole is an exercise in frustration, one of the worst parts is the lack of reusability and the complexity required to achieve it. Github’s concept of reusable workflows on Github Actions is clearly a cobbled together afterthought. ...

The YAML Spec defined a simple way to reuse values in YAML documents by means of Anchors and Aliases. ...

As of 2022, I’ve been using zsh as my primary shell for 14 years. Over that time I’ve experimented with and accumulated a lot of configuration, scripts, hacks, plugins and themes. I’ve settled on a configuration that I’m mostly happy with, over this series I’m going to share my current setup. ...

Contributing to Open Source is important to the quality and maintainability of the software and engineering communities we rely on every day - so why is it that so many developers/engineers never participate? ...

We create more value by having conversations in public instead of behind closed doors. ...

Near the end of each year I note down a summary of the best apps I’ve enjoyed using throughout the year, here’s 2022. ...

We’re (still) not deploying enough It’s 2022 and not deploying frequently enough is still one of the most common causes of software failure. ...

Over the past month I’ve been playing with OpenAI’s DALL·E 2, below are some of the interesting images I’ve generated. “DALL·E 2 is a new AI system that can create realistic images and art from a description in natural language.” “A renaissance oil painting of two developers arguing over which javascript framework is the worst” “A Ralph Steadman painting of 7 cats sitting on a fence, and one raven” ...

Beginner Harmonica Practice Exercises Way back in around 2008 I wrote up a doc with a list of beginner harp exercises and practice riffs, at the time it was uploaded to a few forums and a shared Google doc - both have since disappeared. To this day I still get regular emails asking for a copy of it so I thought I’d copy it to a post here. Note: I haven’t updated these since around 2011. ...

Firefox Addons - 2022 Edition My list of must-have Firefox addons - 2022 edition Updated: 2022-11-07 Privacy and Security Firstly - you should have Firefox’s Enhanced Tracking Protection enabled. Don’t Track Me Google LocalCDN UTM Tracking Token Stripper Note: You can accomplish some of what this does by setting up the removeparam uBlock origin rules I’ve listed below. Multi-Account Containers Useful for setting sites such as Amazon, eBay, Twitter, LinkedIn, Banking etc… each to always open in their own isolated container. The official addon for whatever Password Manager you use. Firefox Translations uBlock Origin Probably the single most important addon. ...

Until recently I used Evernote for my notes, web clippings and document drafts, I was a subscriber going on 14 years and had over 3000 notes. However in recent years Evernote has gone downhill to the point it could be considered hostile to it’s users. The native macOS app was replaced with an Electron Javascript webframe and like almost all Electron apps I’ve tried it was plagued by poor performance, memory leaks and aggressive idle CPU utilisation and above all else it now lacks the low latency feeling the native application had. ...

Weekend upgrades to my Ranchilio Silvia v4 espresso machine. Added a digital PID kit for improved temperature control Replaced the boiler element Thermal insulation Basic “smart” power control with Siri integration Upgrade the standard Ranchilio basket to a VST 18g Ridged Rebuild Photos Note the damage to the boiler elements terminals, this was shorting and causing the houses RCD to trip Wiring the PID controller ...

A template team charter for software and platform engineering teams. Research1 shows there is great value in people embracing a shared vision and ideas that are bigger than themselves. When it comes to a team charter, it’s important to remember that it’s not a static document. It’s a living document that should be updated as the team evolves and grows. Keep in mind that One size doesn’t fit all. Consider your (teams) mission, cultural background and values. While not exhaustive - this is on the longer side, you may wish to distill as is practical, but I implore you not to only pick the easier items - that’s missing the point. These can give you talking points when performing postmortems, retros and during times of conflict. See also: ‘Using Old Ways of Thinking to Apply New Ways of Working’ and ‘BVSSHJ Principles’. ...

Last weekend I had a chance encounter with a Ferrari F12 Berlinetta and the owner kindly let me jump in and take a few photos. Specs 6.3L v12 544 KW (740 Horsepower) 690 Nm at 6000RPM 1525 kg 0-100 in 3.1s 0-200 in 8.5s Maximum speed: ‘over 340 km/h’ Was ‘The fastest road car Ferrari ever built’ Photos ...

Near the end of every year I note down a summary of the best apps, hardware & podcasts I’ve enjoyed throughout the year (and often for some time before). This post has been superseded. You can find the latest version of this post here. Software and Services Plex Things Pixelmator Pro Reeder NextMeeting Bumpr Fastmail 1Password Calibre Backblaze iTerm Gitlab PFsense Stay Onyx Evernote Handbrake Amphetamine Firefox Software - Mobile Plex Things Reeder Overcast Evernote MiniHack Alien Blue Singal Prompt Browser Addons uBlock Origin Dencentraleyes Hardware LG C9 OLED TV Das Ultimate 4 Logitech MX Master 3 16’’ Macbook Pro iPhone 12 Pro AppleTV 4K Audioquest DragonFly Red USB DAC Ultrasone Signature Pro Headphones Kindle Paperwhite All Day Socks Bellroy Wallets Podcasts The Skeptics Guide To The Universion (SGU) You Are Not So Smart Risky.Biz RHLSTP Louis Theroux - Grounded Hiphop Saved My Life Geological 99% Invisible BBC Global News YouTube Channels Techmoan Jim Jefferies - Well I Don’t Know About That! Salvatore Ganacci Warped Perception Smarter Every Day DTM First We Feast (HotOnes) NPR Tiny Desk Concerts Logic The Absolute Worst Software I’ve Experienced 💩 Microsoft Office365 💩 💩 Microsoft Teams 💩 💩 Atlassian Jira 💩 💩 Atlassian Bamboo 💩 💩 Atlassian Confluence 💩 💩 McAffee Endpoint Security 💩 💩 Facebook 💩

You magnificent girl, you brought me 14 years of love and happiness Photos

Stop trying to hire with titles like ‘DevOps Engineer’ or ‘Cloud Engineer’ “DevOps is … not a job title” “It’s more of a cultural practice, like innovation, and it makes just as little sense hiring “innovation engineers” and expecting your organisation to be innovative, without also creating the culture to foster innovation.” (Joel Shea when responding to this posted on LinkedIn) In most cases it’s clear the organisation doesn’t truly know what they want or need and likely don’t understand the nuances of the aspects of engineering. ...

As stated in my announcement post - after 7~ years, I resigned from Infoxchange several months ago. I planned on taking a holiday, relaxing, decompressing, dipping my toes in some off-the-shelf tech I haven’t spent a lot of time with and performing some personal growth through reading a number of books that have been sitting on my ’to read’ shelf. This is exactly what I’ve done. I’m now open to new opportunities I’d rather find a workplace that I can add great value too and be happy to work with than rush into a new role quickly, obviously if something great turns up right away - I’ll pursue it. ...

After 7~ years, I have resigned from Infoxchange. In 2012 I was hired to work with Infoxchange to “shake things up” (in the Systems Operations team) and stabilise the hosting environment. As Team Lead of Systems Operations I was tasked with melding the team and then for us to work to provide a stable, robust, modern and scalable platform for Infoxchange developed application hosting and product delivery and in a short time, with a small budget - we did just that. ...

In 2018 I set out to replace our XenSever 7.2 based virtualisation after Citrix essentially screwed over free / open source users. This project was to directly replace XenServer 7.2 with something supported and manageable for our traditional virtualisation needs. High Level Selection Considerations I evaluated a number of options, with the primary candidates below. Key criteria (at a high level) I was evaluating: Ease of moving from our existing XenServer 7.2 based hypervisor clusters. Security (architecture, hardening, monitoring, logging). Cost (both licensing if any and self-support / management costs). VM Performance (Storage IOPs and throughput, Network latency and throughput, Processing latency, steal from over-provisioned workloads). Management UI/UX and performance (for BAU activities). Management / Cluster SPOFs, fail-over and redundancy. Installation and upgrade process. Update and security patching frequency. Networking design and complexity. Community (size, engagement, acceptance of suggestions / MRs). Reliable live VM migrations. Ease of management for a small team (Part of a low TCO). Risk of vendor and technology lock-in. Risk of survival (will it still be well maintained over the next 1-3 years). Stability and reliability above practically all else. XenServer 7.6 (w/ Paid License) For Potentially priority support from Citrix for issues. Easy upgrade from 7.2. Fresh installs and upgrades are simple, painless and easy to pxeboot, licensing can be a pain after install however. Against Slow moving development. Incredibly expensive, it would cost us something like $140,800 AUD per year (shelf price) for our 32 hosts (each with 2 sockets). Features heavily restricted by licensing model. Mostly older-fashioned enterprise users. No web management interface (although you can use Xen Orchestra from the folks behind XCP-ng). Diminishing community since XenServer licensing changes in 7.3. Poor storage performance compared to KVM based solutions. Many outdated packages, while kind of based on CentOS 7, there are a great deal of packages from older releases or completely custom rebuilt. Uses old technologies like EXT3 and doesn’t support SSD/Flash TRIM/DISCARD functions. SELinux not enforcing or supported. OpenvSwitch still uses a lot of Dom0 CPU (same as XenServer). Dom0 often ends up limiting VM storage operations (tapdisk maxing out Dom0 CPU). Applying updates can be painful or at least slow, requiring binary ISO files to be downloaded from Citrix. Uses Jira for bug tracking (I just can’t stand the thing, it’s painfully laggy, give my GitLab or Github over it any day). XCP-ng XCP-ng is a relatively recent fork from XenServer after it was open sourced, tracking upstream but clearly prioritising modernisation and community. ...

In late November of 2018 I was lucky enough to get my hands on the upcoming 2019 Sport spec Mustang GT v8 to be its first driver. I’m not usually an American Muscle enthusiast, but the new 2019 GT really is quite different from its slightly older counterparts, with a new specially developed 10 speed gearbox (yes, 10 gears - because more is better right?), tuned up suspension and so seemed like it would make for a pretty fun trip. ...

I wanted an alias or function to use youtube-dl in the background. Looking around the web lots of people seemed to want this and most of them were banging their heads against a wall due to: A) bash quoting B) backgrounding dying when their terminal closed C) passing the argument (url in my case) to the function Here’s a simple function I whipped up that seems to ‘just works’™: function yt { nohup youtube-dl "$1" --no-progress 2>&1 > youtube-dl-"$(date +%Y%m%d-%H%M%S)".log & } And if you don’t want logs, simply send the output to /dev/null: ...

This feature annoys me endlessly, I end up zoomed in and out of websites all over the internet. … But the fix is easy and there’s no addons required. Navigate to about:config (in Firefox’s URL bar) Change the value of the following two properties to 0: mousewheel.with_control.action mousewheel.with_meta.action If you use Firefox sync and want these settings to sync between your machines, also add the following properties: Create two new properties both of type boolean and set them to true: ...

Last weekend I headed to New Zealand and marshalled at the Highlands Festival of Speed. The Highlands Festival of Speed is a weekend packed with some of the best modern classic racing you will see in New Zealand. Featuring the Pre 65 Racing, Mainland Muscle, Formula Libre, South Island Porsche, OSCA, Highlands Modern Classics/Nostalgic Classics and the Highlands Sprint Series. This is a weekend of true blue Kiwi racing starring some of New Zealand’s most well known racing legends. ...

It was almost 3 years ago that my open source storage project went into production. In that time it’s been running 24/7 serving as highly available solid state storage for hundreds of VMs and several virtualisation clusters across our two main sites. I’m happy to report that the clusters have been operating very successfully since their conception. Since moving away from proprietary ‘black box’ vendor SANs, we haven’t had a single SAN issue, storage outage. ...

“A chain is no stronger than its weakest link” Any improvement made anywhere besides the bottleneck is an illusion. Any improvement made after the bottleneck is useless because it will always remain starved waiting for work from the bottleneck. Any improvement made before the bottleneck merely results in more ‘work’ piling up at the bottleneck. Identify the system’s constraint(s) (that which prevents the organisation from obtaining more of the goal in a unit of time) Decide how to exploit the system’s constraint(s) (how to get the most out of the constraint) Subordinate everything else to the above decision (align the whole system or organization to support the decision made above) Elevate the system’s constraint(s) (make other major changes needed to increase the constraint’s capacity) Warning! If in the previous steps a constraint has been broken, go back to step 1, but do not allow inertia to cause a system’s constraint. References The Phoenix Project Wikipedia - Theory of Constraints

Here are my top album pics for 2017 (in no particular order) Note: This is by no means an exhaustive list, it’s just the top albums that really stood out to me and in all fairness, I’ve thought of several others since so there may be a follow up post (or two). Father John Misty - Pure Comedy As a bonus, a short film was released with the album: ...

My good friend Joel Shea received a most unlikely gift this Christmas - A vintage HP 4951 Protocol Analyser. According to the HP Computer Museum: Original Price: $3595 The 4951B was replaced by the 4951C and 4952A in 1986. Both new models handled Async, BSC, SDLC, HDLC, X.25 and SNA protocols. The 4951C also handled DDCMP, while the 4952A did not. The 4952A handled X.21 while the 4951C did not. Both new analysers used a floppy dive (618 KB) for removable media. ...

Using GlusterFS to provide volume storage to Kubernetes as a replacement for our existing file and static content hosting. This talk was given at Infracoders on Tuesday 14th November 2017. NOTE: Below link to slides currently broken - will fix soon! (03/08/2019) Click below to view slides (PDF version): Direct download link

In vim, you can add a comment at the top of files to set the syntax, e.g.: # vim: syntax=ruby In SublimeText there are many ways to detect syntax, one interesting approach I’ve recently found useful is to match on the top line in the file. For example, with Puppet there is a file called Puppetfile, it has no extension but it’s really Ruby syntax, so it’s useful to add linting incase you miss something simple like a , and break deployments. ...

Earlier this week we started the process to upgrade one of our hypervisor compute clusters when we encountered a rather painful bug with HP’s Broadcom NIC chipsets. We were part way through a routine rolling pool upgrade of our hypervisor (XenServer) cluster when we observed unexpected and intermittent loss of connectivity between several VMs, then entire XenServer hosts. The problems appeared to impact hosts that hadn’t yet upgraded to XenServer 7.2. We now attribute this to a symptom of extreme packet loss between the hosts in the pool and thanks to buggy firmware from Broadcom and HP. ...

We’re in the process of shifting from using our custom ‘glue’ for orchestrating Docker deployments to Kubernetes, When we first deployed Docker to replace LXC and our legacy Puppet-heavy application configuration and deployment systems there really wasn’t any existing tool to manage this, thus we rolled our own, mainly a few Ruby scripts combined with a Puppet / Hiera / Mcollective driven workflow. The main objective is to replace our legacy NFS file servers used to host uploads / attachments and static files for our web applications, while NFS(v4) performance is adequate, it is a clear single point of failure and of course, there are the age old stale mount problems should network interruptions occur. ...

Of all the tools for reading news and subscribing to software releases, I still find RSS the most useful. I use Feedly to manage my rss subscriptions and keep all my devices in sync, but instead of using the Feedly’s own client, I use an app called Reeder as the client / reader itself. Link: My Feedly RSS Feed Feedly RSS feed subscription management Features: Keyword alerts. Browser plugins to subscribe to (current) url. Notation and highlighting support (a bit like Evernote). Search and filtering across large numbers of feeds / content. IFTTT, Zapier, Buffer and Hootsuite integration. Built in save / share functionality (that I only use when I’m on the website). Backup feeds to Dropbox. Very fast, regardless of the fact that I’m in Australia - which often impacts the performance of apps / sites that tend to be hosted on AWS in the US as the latency is so high. Article de-duplication is currently being developed I believe, so I’m looking forward to that! Easy manual import, export and backup (no vendor lock-in is important to me). Public sharing of your Feedly feeds (we’re getting very meta here!). Reeder A (really) beautiful and fast iOS / macOS client ...

MH-Z19 CO2 sensor reader, logger and visualiser Reads data from UART(serial)-connected MH-Z19 (or MH-Z14) sensor using python 3. If you dare to install nodejs you can visualise the logged data (using html and plotly.js library). Repository: sammcj/CO2-Logger Usage Note this post is from 2016, in 2021 I replaced my custom Co2 loggers with an Aranet4. While very expensive, is and excellent off-the-shelf solution, with many features. ...

I wanted to try Android for a couple of weeks, I like staying on top of technology, gadgets and making sure I never become a blind ‘zealot’ for any platform or brand. The OnePlus 3 I did a lot of research and decided to try the “Oneplus 3” as it was good bang-for-buck, ran the latest software had plenty of grunt with the latest 8 core, high clock speed Qualcomm processor coupled with 6GB of DDR4 - the specs really are very impressive, especially for a $400USD phone. ...

note: This is a follow up post from 2015-07-21-rcd-stonith A Linux Cluster Base STONITH provider for use with modern Pacemaker clusters This has since been accepted and merged into Fedora’s code base and as such will make it’s way to RHEL. Source Code: Github Diptrace CAD Design: Github I have open sourced the CAD circuit design and made this available within this repo under CAD Design and Schematics Related RedHat Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1240868 v1 vs v2/v3 versions of the rcd_serial STONITH system The v2/v3 cables include the following improvements: ...

Ever forgotten to add a critical service to monitoring? Want to know if a service or process fails without explicitly monitoring every service on a host? …Then why not use SystemD’s existing knowledge of all the enabled services? Thanks to ‘Kbyte’ who made a simple Nagios plugin to do just this! Requirements Python3 (For RHEL/CentOS 7 yum install python34) python-nagiosplugin My pre-built RPMs or pip3 install nagiosplugin PyNagSystemD

Scripts and source available here: sql_ascii_to_utf8 The Goal To be able to take a Postgres Database which is in SQL_ASCII encoding, and import it into a UTF8 encoded database. Requirements: Python3 (For RHEL/CentOS 7 yum install python34) python-nagiosplugin My pre-built RPMs or pip3 install nagiosplugin PyNagSystemD The Problem PostreSQL will generate errors like this if it encounters any non-UTF8 byte-sequences during a database restore: # pg_dump -Fc test_badchar | pg_restore -d test_badchar_utf8 pg_restore: [archiver (db)] Error while PROCESSING TOC: pg_restore: [archiver (db)] Error from TOC entry 2839; 0 26852 TABLE DATA table101 postgres pg_restore: [archiver (db)] COPY failed for table "table101": ERROR: invalid byte sequence for encoding "UTF8": 0x91 CONTEXT: COPY table101, line 1 WARNING: errors ignored on restore: 1 And the corresponding data will be omitted from the database (in this case, the whole table, even the rows which did not have a problem): ...

The most common way to use rsync is probably as such: rsync -avr user@<source>:<source_dir> <dest_dir> Resulting in 30-35MB/s depending on file sizes This can be improved by using a more efficient, less secure encryption algorithm, disabling compression and telling the SSH client to disable some unneeded features that slow things down. With the settings below I have achieved 100MB/s (at work between VMs) and over 300MB/s at home between SSD drives. ...

This is a quick tldr there are many other situations and options you could consider FIO man page IOP/s = Input or Output operations per second Throughput = How many MB/s can you read/write continuously Variables worth tuning based on your situation --iodepth The iodepth is very dependant on your hardware. Rotational drives without much cache and high latency (i.e. desktop SATA drives) will not benefit from a large iodepth, Values between 16 to 64 could be sensible. ...

Let’s pretend you have a project on Gitlab called ask-izzy and you want to mirror it up to Gitlab which is located at https://github.com/ask-izzy/ask-izzy Assuming you’re running Gitlab as the default user of git and that your repositories are stored in /mnt/repositories you can following something similar to the following instructions: Grant write access to Github Get your Gitlab install’s pubkey from the git user cat /home/git/.ssh/id_rsa.pub On Github add this pubkey as deploy key on the repo, make sure you tick the option to allow write access. ...

Today we launched a mobile website for homeless people … and it was launched by one of Australia’s many recent Prime Ministers Today alone we served up over 87,000 requests As many of you know, I work with Infoxchange as the operations lead. When I first heard the idea of a website or app for people that have found or are worried about finding themselves homeless in Australia I really didn’t think it made sense - until I saw the stats showing how many homeless people in Australia have regular access to a smart phone and data either via a cellular provider or free WiFi. ...

If a disk / VDI is orphaned or only partially deleted you’ll notice that under the SR it’s not assigned to any VM. This can cause issues that look like metadata corruption resulting in the inability to migrate VMs or edit storage. For example: [root@xenserver-host ~]# xe vdi-destroy uuid=6c2cd848-ac0e-441c-9cd6-9865fca7fe8b Error code: SR_BACKEND_FAILURE_181 Error parameters: , Error in Metadata volume operation for SR. [opterr=VDI delete operation failed for parameters: /dev/VG_XenStorage-3ae1df17-06ee-7202-eb92-72c266134e16/MGT, 6c2cd848-ac0e-441c-9cd6-9865fca7fe8b. Error: Failed to write file with params [3, 0, 512, 512]. Error: 5], Removing stale VDIs To fix this, you need to remove those VDIs from the SR after first deleting the logical volume: ...

“Having a SCSI ID is a f*cking idiotic thing to do.” - Linus Torvalds …and after the amount of time I’ve wasted getting XenServer to play nicely with LIO iSCSI failover I tend to agree. The Problem One oddity of Xen / XenServer’s storage subsystem is that it identifies iSCSI storage repositories via a calculated SCSI ID rather than the iSCSI Serial - which would be the sane thing to do. ...

Slides Failover Demo

In an attempt to avoid using the Juniper Pulse (Now Pulse Secure) VPN client we tried OpenConnect but found that DNS did not work correctly when connected to the VPN. This bug has now been resolved recently but has not made it’s way into a new build, in fact there have been no releases for 6 months. Luckily the OpenConnect was not too difficult to build from source. Build OpenConnect on OSX Remove old openconnect and install deps brew remove openconnect brew install libxml2 lzlib openssl libtool libevent Build openconnect wget git.infradead.org/users/dwmw2/openconnect.git/snapshot/0f1ec30d17aa674142552e275bf3fac30d891b39.tar.gz tar zxvf 0f1ec30d17aa674142552e275bf3fac30d891b39.tar.gz cd openconnect-0f1ec30 LIBTOOLIZE=glibtoolize ./autogen.sh PATH=/usr/local/opt/gettext/bin:$PATH ./configure make make install To connect sudo openconnect --juniper -u myusername www.myserver.com If you’re comfortable with allowing admin users to run openconnect without entering a sudo password, add the following using sudo visudo: ...

Over the last two months I’ve been running selected IO intensive servers off the the SSD storage cluster, these hosts include (among others) our: Primary Puppetmaster Gitlab server Redmine app and database servers Nagios servers Several Docker database host servers Reliability We haven’t had any software or hardware failures since commissioning the storage units. During this time we have had 3 disk failures on our HP StoreVirtual SANs that have required us to call the supporting vendor and replace failed disks. ...

Set update channel to receive developer beta update sudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11seed-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz Set update channel to receive public beta update sudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz List available updates sudo softwareupdate --list Set update channel to receive default, stable updates sudo softwareupdate --clear-catalog Show current settings defaults read /Library/Preferences/com.apple.SoftwareUpdate.plist Write setting manually defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz

The following are benchmarks from our testings of our iSCSI SSD storage. 67,300 read IOP/s on a VM on iSCSI (Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM) Per VM and scales to 1,000,000 IOP/s total root@dev-samm:/mnt/pmt1 128 # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=128 --size=2G --readwrite=read test: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128 2.0.8 Starting 1 process bs: 1 (f=1): [R] [55.6% done] [262.1M/0K /s] [67.3K/0 iops] [eta 00m:04s] 38,500 random 4k write IOP/s on a VM on iSCSI (Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM) Per VM and scales to 700,000 IOP/s total root@dev-samm:/mnt/pmt1 # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=128 --size=2G --readwrite=randwrite test: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128 2.0.8 Starting 1 process bs: 1 (f=1): [w] [26.3% done] [0K/150.2M /s] [0 /38.5K iops] [eta 00m:14s] Raw device latency on storage units Intel DC3600 1.2T PCIe NVMe root@s1-san6:/proc # ioping /dev/nvme0n1p1 4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=1 time=104 us 4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=2 time=83 us 4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=3 time=51 us 4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=4 time=71 us SanDisk SDSSDXPS960G SATA root@pm-san5:/proc # ioping /dev/sdc 4.0 KiB from /dev/sdc (device 894.3 GiB): request=1 time=4.2 ms 4.0 KiB from /dev/sdc (device 894.3 GiB): request=2 time=4.1 ms 4.0 KiB from /dev/sdc (device 894.3 GiB): request=3 time=4.1 ms 4.0 KiB from /dev/sdc (device 894.3 GiB): request=4 time=4.1 ms Micron_M600_MTFDDAK1T0MBF SATA root@pm-san5:/proc # ioping /dev/sdf 4.0 KiB from /dev/sdf (device 953.9 GiB): request=1 time=157 us 4.0 KiB from /dev/sdf (device 953.9 GiB): request=2 time=190 us 4.0 KiB from /dev/sdf (device 953.9 GiB): request=3 time=65 us 4.0 KiB from /dev/sdf (device 953.9 GiB): request=4 time=181 us ```shell ## Latency on the a VM - (Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM) ```shell root@dev-samm:/mnt 127 # ioping pmt1/ 4096 bytes from pmt1/ (ext4 /dev/xvdb1): request=1 time=0.6 ms 4096 bytes from pmt1/ (ext4 /dev/xvdb1): request=2 time=0.7 ms 4096 bytes from pmt1/ (ext4 /dev/xvdb1): request=3 time=0.7 ms --- pmt1/ (ext4 /dev/xvdb1) ioping statistics --- 3 requests completed in 2159.1 ms, 1508 iops, 5.9 mb/s min/avg/max/mdev = 0.6/0.7/0.7/0.1 ms root@dev-samm:/mnt # ioping pmt2/ 4096 bytes from pmt2/ (ext4 /dev/xvdc1): request=1 time=0.6 ms 4096 bytes from pmt2/ (ext4 /dev/xvdc1): request=2 time=0.8 ms --- pmt2/ (ext4 /dev/xvdc1) ioping statistics --- 2 requests completed in 1658.4 ms, 1470 iops, 5.7 mb/s min/avg/max/mdev = 0.6/0.7/0.8/0.1 ms root@dev-samm:/mnt # ioping pmt3/ 4096 bytes from pmt3/ (ext4 /dev/xvde1): request=1 time=0.6 ms 4096 bytes from pmt3/ (ext4 /dev/xvde1): request=2 time=0.9 ms 4096 bytes from pmt3/ (ext4 /dev/xvde1): request=3 time=0.9 ms ...

A modified version of John Sutton’s rcd_serial cable coupled with our Supermicro reset switch hijacker: This works with the rcd_serial fence agent plugin. Reasons rcd_serial makes for a very good STONITH mechanism: It has no dependency on power state. It has no dependency on network state. It has no dependency on node operational state. It has no dependency on external hardware. It costs less that $5 + time to build. It is incredibly simple and reliable. Essentially the most common STONITH agent type in use is probably those that control UPS / PDUs, while this sounds like a good idea in theory there are a number of issues with relying on a UPS / PDU: ...

First some background… One of the many lessons I’ve learnt from my Linux HA / Storage clustering project is that the Debian HA ecosystem is essentially broken, We reached the point where packages were too old, too buggy or in Debian 8’s case - outright missing. In the past I was very disappointed with RHEL/CentOS 5 / 6 and (until now) have been quite satisfied with Debian as a stable server distribution with historicity more modern packages and kernels. ...

Due to several recent events beyond my control I’m a bit behind on the project - hence the lack of updates which I apologise for. The goods news is that I’m back working to finish off the clusters and I’m happy to report that all is going to plan. Here is the final digram of the two-node cluster design: Plain text version available here This was generated from the LCMC tool (beware - it’s java!). ...

A brief demonstration of the failover and recovery process on the storage clusters I’ve been building.

A high level talk from Infracoders Melbourne on 12/04/2015. There’s also a low quality recording available here: Related posts: Building a high performance SSD SAN - Part 1

Linux Kernel CI for Debian Github: sammcj/kernel-ci Those of us using technologies such as Docker and BTRFS or simply trying to gain a performance edge on the competition have a lot to gain from the features and performance of recent Kernel updates (especially from 3.18 onwards). ‘Enterprise’ Linux distributions such as RHEL & variants are concerningly out of date when comes to the Kernel. Many people seem to have forgotten what Linux is… Linux IS the Kernel. ...

Docker config to setup XO which is a web interface to visualize and administrate your XenServer (or XAPI enabled) hosts Github: sammcj/docker-xen-orchestra Running the app Updates are pushed to the Docker Hub’s automated build service: https://registry.hub.docker.com/u/sammcj/docker-xen-orchestra From Docker Hub docker pull sammcj/docker-xen-orchestra docker run -d -p 8000:80 sammcj/docker-xen-orchestra Building git clone https://github.com/sammcj/docker-xen-orchestra.git cd docker-xen-orchestra # Edit whatever config you want to change docker build -t xen-orchestra . See https://xen-orchestra.com for information on Xen Orchestra ...

Inspired by http://zitseng.com/archives/7489 Source (Github) WARNINGS Do not run unless you understand what this is doing The CA system is broken by design - This is not a fix for that This is merely a band-aid for those interested or concerned about these root CAs Usage chmod +x delete_gov_roots.sh ./delete_gov_roots.sh You’ll be prompted for your password as root access is required to delete system-wide root certs. See Also http://convergence.io https://addons.mozilla.org/en-US/firefox/addon/certificate-patrol/ https://github.com/kirei/catt https://www.eff.org/observatory https://bugzilla.mozilla.org/show_bug.cgi?id=478418 http://support.apple.com/en-us/HT202858 https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning

Over the coming month I will be architecting, building and testing a modular, high performance SSD-only storage solution. I’ll be documenting my progress / findings along the way and open sourcing all the information as a public guide. With recent price drops and durability improvements in solid state storage now is better time than any to ditch those old magnets. Modular server manufacturers such as SuperMicro have spent large on R&D thanks to the ever growing requirements from cloud vendors that utilise their hardware. ...

Further to my earlier post on XenServer storage performance with regards to directly attaching storage from the host, I have been analysing the performance of various SSD storage options. I have attached a HP DS2220sb storage blade to an existing server blade and compared performance with 4 and 6 SSD RAID-10 to our existing iSCSI SANs. While the P420i RAID controller in the DS2220sb is clearly saturated and unable to provide throughput much over 1,100MB/s - the IOP/s available to PostgreSQL are still a very considerably performance improvement over our P4530 SAN - in fact, 6 SSD’s result in a 39.9x performance increase! ...

“Search - A Journey of Delivery on a Budget” Presented a Melbourne Search - July 2014 - “Search - A Journey of Delivery on a Budget” (Click for slides)

The way we work at Infoxchange has changed greatly. A retrospective journey into transforming Infoxchange’s technology and culture over the past 24 months - presented a Melbourne DevOps - December 2014

At the end of every year I note down a summary of the best applications, hardware & websites I’ve enjoyed & depended on throughout the year (and often for some time before). This post has long since been superseded. You can find the latest version of this post here. Software / General Use Fastmail - https://www.fastmail.com Evernote - https://evernote.com Reeder - http://reederapp.com Keynote - https://www.apple.com/au/mac/keynote Lastpass - https://lastpass.com Plex - https://plex.tv Calibre - http://calibre-ebook.com Software / Geek Use Sublime Text - http://www.sublimetext.com/3 Homebrew - http://brew.sh DropSync - http://mudflatsoftware.com Beets - http://beets.radbox.org Textual - http://www.codeux.com/textual XLD - http://tmkk.undo.jp/xld/index_e.html Code Academy - http://www.codecademy.com Exercism.io - http://exercism.io Sickbeard + Headphones + Couchpotato + Sabnzbd - http://www.totalhtpc.com/ultimate-usenet-guide.html Software / Mobile Tweetbot - http://tapbots.com/software/tweetbot Pushover - http://pushover.net Lastpass - https://lastpass.com Keynote - https://www.apple.com/au/ios/keynote Reeder - http://reederapp.com/ios Evernote- https://evernote.com Plex - https://plex.tv Backblaze - https://www.backblaze.com WTF Podcast - http://www.wtfpod.com/app MiniHack - https://itunes.apple.com/au/app/minihack-for-hacker-news/id631108846?mt=8 Uber - https://itunes.apple.com/au/app/uber/id368677368?mt=8 Goodreads - https://itunes.apple.com/au/app/goodreads-book-recommendations/id355833469?mt=8 Notify4M - https://itunes.apple.com/au/app/notify4m/id499161979?mt=8 Bandcamp - https://itunes.apple.com/au/app/bandcamp/id706408639?mt=8 Hype Machine - https://itunes.apple.com/au/app/hype-machine/id414315986?mt=8 Nuzzel (Only got onto this today) - https://itunes.apple.com/au/app/nuzzel-news-from-your-friends/id692285770?mt=8 Alien Blue - http://www.reddit.com/r/alienblue Software / SysAdmin or DevOps Specific Gitlab - http://gitlab.org Gitlab-CI - https://about.gitlab.com/gitlab-ci Dash - http://kapeli.com/dash PostgreSQL (Makes my list every year & just keeps getting better) - http://www.postgresql.org PGBadger - http://dalibo.github.io/pgbadger Docker - https://www.docker.com Consul - https://consul.io Puppet (I couldn’t do my job as well without it) - http://puppetlabs.com iTerm - http://iterm2.com Nginx - http://nginx.org Htop - http://hisham.hm/htop Bonnie++ - http://linux.die.net/man/8/bonnie++ Openfire - https://www.igniterealtime.org/projects/openfire Hiera-Eyaml - https://github.com/TomPoulton/hiera-eyaml Rubinius - http://rubini.us Puma - http://puma.io XenServer - http://xenserver.org ElasticSearch - www.elasticsearch.org Logstash - http://logstash.net FPM - https://github.com/jordansissel/fpm PFsense - https://www.pfsense.org Debian Jessie (Not quite released but the next great version of the best linux Distro IMO) - https://www.debian.org/releases/jessie Check_MK Multisite - https://mathias-kettner.de/checkmk_multisite.html PWSafe - https://itunes.apple.com/au/app/pwsafe-password-safe-compatible/id520993579?mt=12 Supervisord - http://supervisord.org Websites Last.fm - http://www.last.fm/user/sammcj2000 Feedly - https://feedly.com HackerNews - https://news.ycombinator.com/news Lucidchart - http://lucidchart.com/ MondoTunes (Might be a little biased here!) - http://mondotunes.org Hardware Das Ultimate 4- http://www.daskeyboard.com Logitech Performance MX - http://www.logitech.com/en-au/product/performance-mouse-mx 15’’ Macbook Pro Retina - http://www.apple.com/au/macbook-pro iPhone 6+ (because its bigger than bigger, or something) - https://www.apple.com/iphone-6 CuBox - http://www.solid-run.com/product/cubox-i4pro Parani SD1000 Bluetooth Serial Adapter - http://www.senaindustrial.com/products/industrial_bluetooth/sd1000.php SanDisk Extreme Pro 480GB SSD - http://www.newegg.com/Product/Product.aspx?Item=N82E16820171999 Other / Non-tech All Day Socks - http://alldaysocks.com Sennheiser Amperior On-Ear Headphones - http://www.head-fi.org/products/sennheiser-amperior-on-ear-headphones O2 + ODAC - http://www.jdslabs.com/products/48/o2-odac-combo Benchmark DAC1 HDR - http://benchmarkmedia.com/products/benchmark-dac1-hdr-digital-to-analog-converter Bellroy Wallets - http://bellroy.com Ink Shoes - http://www.inkshoes.it Kindle Paperwhite - http://www.amazon.com.au/gp/feature.html?docId=3077740006 Nerf Jolt - http://nerf.wikia.com/wiki/Jolt_EX-1 Books The Phoenix Project (Reread this year for the 3rd time) - http://itrevolution.com/books/phoenix-project-DevOps-book Surely You’re Joking Mr Feynman! - https://www.goodreads.com/book/show/5544.Surely_You_re_Joking_Mr_Feynman_ The Dark Tower Series - https://www.goodreads.com/book/show/43615.The_Gunslinger Snow Crash - https://www.goodreads.com/book/show/830.Snow_Crash

Intro At Infoxchange we use XenServer as our Virtualisation of choice. There are many reasons for this including: Open Source. Offers greater performance than VMware. Affordability (it’s free unless you purchase support). Proven backend Xen is very reliable. Reliable cross-host migrations of VMs. The XenCentre client, (although having to run in a Windows VM) is quick and simple to use. Upgrades and patches have proven to be more reliable than VMware. OpenStack while interesting, is not yet reliable or streamlined enough for our small team of 4 to implement and manage. XenServer Storage & Filesystems Unfortunately the downside to XenServer is that it’s underlying OS is quite old. The latest version (6.5) about to be released is still based on Centos 5 and still lacks any form of EXT4 and BTRFS support, direct disk access is not available… without some tweaking and has no real support for TRIM unless you have direct disk access and are happy with EXT3. ...

Reprogram BMW after doing Automatic to Manual Conversion Based on my E36 328is experience, written in 2010. Assumptions This guide assumes the following: You have programmed a BMW ECU/DME before using DIS or similar. You are taking all the necessary safety precautions. (Fully charged battery etc…) You have DIS working (We used EasyDIS 1.0, Base 44) You have Ediabas (INPA, NCS Expert, IFH Serve) installed and working. You’re able to read between the lines of our crappy document, some steps may differ slightly and our wording may be somewhat imprecise. Data Gathering Make notes of the following: ...