Over the last two months I’ve been running selected IO intensive servers off the the SSD storage cluster, these hosts include (among others) our: Primary Puppetmaster Gitlab server Redmine app and database servers Nagios servers Several Docker database host servers ReliabilityWe haven’t had any software or hardware failures since commissioning the storage units.
Set update channel to receive developer beta updatesudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11seed-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz Set update channel to receive public beta updatesudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz List available updatessudo softwareupdate --list Set update channel to receive default, stable updatessudo softwareupdate --clear-catalog Show current settingsdefaults read /Library/Preferences/com.apple.SoftwareUpdate.plist Write setting manuallydefaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz
The following are benchmarks from our testings of our iSCSI SSD storage. 67,300 read IOP/s on a VM on iSCSI (Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM) Per VM and scales to 1,000,000 IOP/s total root@dev-samm:/mnt/pmt1 128 # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=128 --size=2G --readwrite=read test: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128 2.
A modified version of John Sutton’s rcd_serial cable coupled with our Supermicro reset switch hijacker: This works with the rcd_serial fence agent plugin. Reasons rcd_serial makes for a very good STONITH mechanism: It has no dependency on power state. It has no dependency on network state. It has no dependency on node operational state.
First some background… One of the many lessons I’ve learnt from my Linux HA / Storage clustering project is that the Debian HA ecosystem is essentially broken, We reached the point where packages were too old, too buggy or in Debian 8’s case - outright missing. In the past I was very disappointed with RHEL/CentOS 5 / 6 and (until now) have been quite satisfied with Debian as a stable server distribution with historicity more modern packages and kernels.
Due to several recent events beyond my control I’m a bit behind on the project - hence the lack of updates which I apologise for. The goods news is that I’m back working to finish off the clusters and I’m happy to report that all is going to plan. Here is the final digram of the two-node cluster design:
A brief demonstration of the failover and recovery process on the storage clusters I’ve been building.
A high level talk from Infracoders Melbourne on 12/04/2015. There’s also a low quality recording available here: Related posts: Building a high performance SSD SAN - Part 1
Linux Kernel CI for Debian Github: sammcj/kernel-ci Those of us using technologies such as Docker and BTRFS or simply trying to gain a performance edge on the competition have a lot to gain from the features and performance of recent Kernel updates (especially from 3.18 onwards). ‘Enterprise’ Linux distributions such as RHEL & variants are concerningly out of date when comes to the Kernel.
Docker config to setup XO which is a web interface to visualize and administrate your XenServer (or XAPI enabled) hosts Github: sammcj/docker-xen-orchestra Running the appUpdates are pushed to the Docker Hub’s automated build service: https://registry.hub.docker.com/u/sammcj/docker-xen-orchestra From Docker Hubdocker pull sammcj/docker-xen-orchestra docker run -d -p 8000:80 sammcj/docker-xen-orchestra Buildinggit clone https://github.com/sammcj/docker-xen-orchestra.git cd docker-xen-orchestra # Edit whatever config you want to change docker build -t xen-orchestra .
Inspired by http://zitseng.com/archives/7489 Source (Github) WARNINGS Do not run unless you understand what this is doing The CA system is broken by design - This is not a fix for that This is merely a band-aid for those interested or concerned about these root CAs Usagechmod +x delete_gov_roots.sh ./delete_gov_roots.sh You’ll be prompted for your password as root access is required to delete system-wide root certs.