AI Portfolio Assistant
Intelligent voice-enabled chatbot with RAG, speech recognition, and natural language understanding
Media & Demos
Voice capture state with particle field
Call-to-action panel that opens the assistant
Role
Creator / Full-Stack Engineer
Period
2024 – Present
Category
ai
Overview
An AI-powered portfolio assistant that lets visitors explore my work through natural conversations. Features voice interaction, retrieval-augmented generation (RAG), and real-time responses powered by Ollama and FAISS vector search.
Key Highlights
- Voice interaction with Whisper STT and Piper TTS (sub-second latency)
- RAG pipeline using FAISS for semantic search across portfolio content
- Local LLM inference with Ollama (Qwen 2.5 3B) - zero API costs
- Graceful degradation with TTS availability checks
- Docker-based deployment on DigitalOcean with SSL
- Dependency-hardened audio stack after swapping XTTS for Piper + SoX to avoid GPU driver drift
Tech Stack
Summary
On-page overviewThis is a concise summary of the challenges, solution, and outcome for this project. Use the Case Study button above for the full deep dive.
The Problem
Visitors needed an engaging way to explore my work, yet the original XTTS/whisper stack was brittle on small CPUs and constantly broke whenever dependencies updated.
The Solution
Rebuilt the assistant with a Piper + Faster-Whisper toolchain, pinned torch/sox dependencies inside Docker, and wrapped everything with RAG + Ollama so the conversational layer stays responsive and locally hosted.
The Outcome
Live at ayushv.dev with ~500 ms text responses, <1 s audio playback, zero API costs, and a hardened audio stack that survives rebuilds without CUDA headaches.
Team & Role
Solo project – architected the FastAPI backend, dependency-managed Docker images, crafted the motion-heavy Next.js UI, and tuned deployment for DigitalOcean.
What I Learned
This project deepened my understanding of FastAPI 0.115 and Ollama (Qwen 2.5 3B) and reinforced best practices in system design and scalability. I gained valuable insights into production-grade development and performance optimization.