In the world of AI, the barrier between an idea and a working application is shrinking every day. In my latest project, I built a full-stack, face-to-face AI interview coach—complete with a high-fidelity, lip-synced avatar—in under 15 minutes.
The craziest part? I didn’t write a single line of code. I used Claude Code to architect the entire system from the command line. Here is how you can do it too.
The Tech Stack: The "Brain" and the "Body"
To get this running with zero lag, we use a specialized pipeline of real-time tools:
Claude Code: The "brain" behind the build. It's an agentic CLI that can read documentation, write code, and run tests.
LiveKit: The backbone infrastructure that handles the real-time audio and video streams.
Deepgram: Transcribes your speech into text for the LLM.
OpenAI (GPT-4o mini): Processes the text and generates the interview coach's response.
Cartesia: Converts the AI’s text response back into lifelike spoken audio.
Simli: Generates the realistic, lip-synced AI face that responds to you in real time.
Step 1: Setting Up Your Environment
Before prompting the agent, you need your workspace ready:
Install VS Code: This is where you’ll interact with the Claude Code extension.
Claude Pro Account: You’ll need at least a Pro plan to access the Claude Code features within VS Code.
LiveKit & Simli Accounts: Sign up for both to grab your API keys.
Step 2: Making Claude an Expert
To ensure Claude knows exactly how to build with these specific tools, we use the Model Context Protocol (MCP). By running a simple command in your terminal, you can provide Claude with the entire LiveKit documentation. Instead of searching the web, Claude queries this "MCP Server" to become an instant expert on building voice agents.
Step 3: Architecting the App
With the environment set up, I simply spoke to Claude: "Set up a LiveKit voice agent using their Python quickstart with separate folders for the agent and the front end".
Claude then:
Created the folder structure.
Handled the Python dependencies using the UV package manager.
Authenticated with LiveKit to pull project credentials automatically.
Step 4: Customizing the Coach
A generic voice agent is cool, but an Interview Coach is useful. I prompted Claude to rewrite the system prompt to act as a technical recruiter for software engineering roles.
Claude modified the backend to cover:
Data Structures & Algorithms.
System Design.
Behavioral Questions (like "Tell me about a time you faced a challenge").
Step 5: Adding the Face (The Simli Integration)
The final piece of the puzzle is the visual avatar. Using the Simli integration, Claude added a real-time video face to the pipeline with just a few lines of code. After grabbing a Face ID from Simli's library—like the "Charlotte" preset—the agent was no longer just a voice; it was a person you could look at while practicing your pitch.
Want to watch the full tutorial? Check it out here: https://youtu.be/zFIjUDRi6Zg