Two Weeks Inside a Local AI Chatbot

Roughly two weeks of building and experimenting with my local AI chatbot using Electron.

The conversation uses three main models: Qwen3-1.7B in Q8_0 format for replies, Whisper Tiny English (onnx-community/whisper-tiny.en) for turning speech into text, and Kokoro-82M v1.0 (onnx-community/Kokoro-82M-v1.0-ONNX) for turning replies into speech. Spoken input becomes text, goes through the same conversation system as typed input, and the reply can be read aloud. There’s also a separate Xenova/all-MiniLM-L6-v2 model that helps find relevant memories by meaning.

Running locally means those models work on my computer after their files are installed. The language model supports GPU acceleration and CPU fallback; speech recognition prefers WebGPU and falls back to CPU processing through WebAssembly. Keeping the interface responsive while all of that happens is its own puzzle. Local doesn't mean free of waiting, RAM usage, or troubleshooting.

Then there's the little animated character. I built it with Three.js and React Three Fiber, with dark rounded shapes, bright eyes, a glowing core, and orbiting rings. It blinks, hovers, glances around, and has desktop movement and interaction reactions. MediaPipe handles local face tracking and hand gestures, with detected cues feeding gaze and animation. Upright pointing is still experimental, and real-person gesture reliability needs more checking. Apparently waving at my own computer now counts as development.

Memory has been especially interesting. Remembering words isn't the same as knowing how to use them, and I don't want a remembered sentence treated as a new instruction. I separated long-term memory from conversation history, added conservative extraction and relevance scoring, and explicitly marked retrieved memories as context rather than commands. Messages, memories, and their search vectors use encrypted local SQLite storage. It still misses some ordinary wording, and successful retrieval doesn't prove the next answer will use it sensibly.

I've built in automated regression checks, launch smoke tests, restart checks, and packaged Windows testing. The latest recorded suite has 355 passing tests. Packaged speech-recognition fixtures and local speech-generation checks also have recorded passes. Manual checks still matter for gesture reliability, audible playback, and memory during a real conversation.

That's been the interesting part for me: following a problem through several systems, finding the small thing that breaks the flow, and figuring out why. Sometimes the chatbot answers. Sometimes I spend the evening learning why it didn't.