Generating Diagrams with with AI / LLMs

The two tools I use with AI / LLMs for generating diagrams are Mermaid and Excalidraw. Mermaid (or MermaidJS) is a popular diagramming library and format supported by many tools and is often rendered inside markdown (e.g. in a readme.md). Excalidraw is an excellent, free and open source diagramming and visualisation tool. I also often make use of a third party Obsidian plugin for Excalidraw. Excalidraw It has a ‘generate diagram with AI’ feature which if you’re using the excalidraw.com online editor offers a few free generations each day (I think this uses a low-end OpenAI model). If you’re running Excalidraw locally or by using the brilliant Obsidian plugin - you can provide any OpenAI compatible API endpoint and model for AI generations. Behind the scenes Excalidraw AI generates and then renders MermaidJS. ...

October 8, 2024 · Sam McLeod

Ingest: Streamlining Content Preparation for LLMs

Ingest is a tool I’ve written to make my life easier when preparing content for LLMs. It parses directories of plain text files, such as source code, documentation etc… into a single markdown file suitable for ingestion by AI/LLMs. Ingest can also estimate vRAM requirements for a given model, quantisation and context length: Features Traverse directory structures and generate a tree view Include/exclude files based on glob patterns Estimate vRAM requirements and check model compatibility using another package I’ve created called quantest Parse output directly to LLMs such as Ollama or any OpenAI compatible API for processing Generate and include git diffs and logs Count approximate tokens for LLM compatibility Customisable output templates Copy output to clipboard (when available) Export to file or print to console Optional JSON output Ingest Intro (“Podcast” Episode): ...

July 29, 2024 · Sam McLeod

LLM Parameter Playground

Here’s a fun little tool I’ve been hacking on to explore the effects of different inference parameters on LLMs. You can find the code and instructions for running it locally on GitHub. It started as a fork of rooben-me’s tone-changer-open, which itself was a “fork” of Figma’s tone generator, I’ve made quite a few changes to make it more focused on local LLMs and advanced parameter exploration.

July 20, 2024 · Sam McLeod

Code, Chaos, and Copilots (AI/LLM Talk July 2024)

Code, Chaos, and Copilots is a talk I gave in July 2024 as an intro to how I use AI/LLMs to augment my capabilities every day. What I use AI/LLMs for Prompting tips Codegen workflow Picking the right models Model formats Context windows Quantisation Model servers Inference parameters Clients & tools Getting started cheat-sheets Download Slide Deck Disclaimer I’m not a ML Engineer or data scientist, As such, the information presented here is based on my understanding of the subject and may not be 100% accurate or complete. ...

July 18, 2024 · Sam McLeod

Understanding AI/LLM Quantisation Through Interactive Visualisations

AI models (“LLMs” in this case) have inherently large sizes and computational requirements that often pose challenges for deployment and use. ...

July 17, 2024 · Sam McLeod

Rating AI Tools

I spend a lot of time working with AI tools and often get asked for recommendations. The following is a list of some of the more notable tools I’ve tried and rated them based on my experience. Just because something has a low rating doesn’t mean it’s bad - it just means it didn’t work well for me and I wouldn’t personally recommend it. I plan on keeping this updated as I try new tools (we’ll see how that goes). ...

July 3, 2024 · 1 min · 101 words · Sam McLeod