Basic Chatbot (Jarvis-style)

Abstract

A “Jarvis”-style voice assistant is one of the most rewarding beginner-to-intermediate Python projects. In ~100 lines you wire together a microphone, speech-to-text, a command dispatcher, web automation, and a text-to-speech engine — and end up with something that genuinely responds when you speak. In this tutorial you build that classic assistant (open YouTube, search Wikipedia, tell the time, send mail), then refactor the if/elif chain into a clean intent registry, add fuzzy matching, and finish with a sketch of an LLM-powered upgrade using the OpenAI / Anthropic / local-LLM APIs.

You will learn:

• How speech_recognition captures and transcribes microphone input.
• How pyttsx3 produces speech offline.
• The intent-dispatch pattern (vs. growing if/elif chains).
• How to integrate Wikipedia, webbrowser, and SMTP cleanly.
• How to design assistants safely (mic privacy, command validation).

Prerequisites

• Python 3.6 or above.
• Microphone and speakers.
• A code editor or IDE.
• Internet connection (for Google’s speech recognition and Wikipedia).

Install Dependencies

install

pip install pyttsx3 SpeechRecognition wikipedia pyaudio

If pyaudio fails on Windows:

windows

pip install pipwin
pipwin install pyaudio

On macOS:

macos

brew install portaudio
pip install pyaudio

On Linux:

linux

sudo apt install portaudio19-dev python3-pyaudio
pip install pyaudio

Getting Started

Create the project

1. Create folder basicchatbot.
2. Inside, create basicchatbot.py.

Write the code

Basic Chatbot

pch.viewSource

Basic Chatbot

# Basic Chat Bot
# Credits: https://www.youtube.com/watch?v=Lp9Ftuq2sVI&t=3008s
# Author: Code With Harry

import pyttsx3 #pip install pyttsx3
import speech_recognition as sr #pip install speechRecognition
import datetime
import wikipedia #pip install wikipedia
import webbrowser
import os
import smtplib

engine = pyttsx3.init('sapi5')
voices = engine.getProperty('voices')
engine.setProperty('voice', voices[0].id)


def speak(audio):
    engine.say(audio)
    engine.runAndWait()


def wishMe():
    hour = int(datetime.datetime.now().hour)
    if hour>=0 and hour<12:
        speak("Good Morning!")

    elif hour>=12 and hour<18:
        speak("Good Afternoon!")   

    else:
        speak("Good Evening!")  

    speak("I am Jarvis Sir. Please tell me how may I help you")       

def takeCommand():
    #It takes microphone input from the user and returns string output

    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Listening...")
        r.pause_threshold = 1
        audio = r.listen(source)

    try:
        print("Recognizing...")    
        query = r.recognize_google(audio, language='en-in')
        print(f"User said: {query}\n")

    except Exception as e:
        # print(e)    
        print("Say that again please...")  
        return "None"
    return query

def sendEmail(to, content):
    server = smtplib.SMTP('smtp.gmail.com', 587)
    server.ehlo()
    server.starttls()
    server.login('youremail@gmail.com', 'your-password') # Enter your email and password at youremail@gmail.com and your-password
    server.sendmail('youremail@gmail.com', to, content) # Enter your email at youremail@gmail
    server.close()

if __name__ == "__main__":
    wishMe()
    while True:
    # if 1:
        query = takeCommand().lower()

        # Logic for executing tasks based on query
        if 'wikipedia' in query:
            speak('Searching Wikipedia...')
            query = query.replace("wikipedia", "")
            results = wikipedia.summary(query, sentences=2)
            speak("According to Wikipedia")
            print(results)
            speak(results)

        elif 'open youtube' in query:
            webbrowser.open("youtube.com")

        elif 'open google' in query:
            webbrowser.open("google.com")

        elif 'open stackoverflow' in query:
            webbrowser.open("stackoverflow.com")   


        elif 'play music' in query:
            music_dir = 'Your Music Directory' # Enter your music directory here like C:\\Users\\Ravi\\Music
            songs = os.listdir(music_dir)
            print(songs)    
            os.startfile(os.path.join(music_dir, songs[0]))

        elif 'the time' in query:
            strTime = datetime.datetime.now().strftime("%H:%M:%S")    
            speak(f"Sir, the time is {strTime}")

        elif 'open code' in query:
            codePath = "C:\\Users\\Ravi\\AppData\\Local\\Programs\\Microsoft VS Code\\Code.exe"
            os.startfile(codePath)

        elif 'email to ravi' in query:
            try:
                speak("What should I say?")
                content = takeCommand()
                to = "ravi@gmail.com"    
                sendEmail(to, content)
                speak("Email has been sent!")
            except Exception as e:
                print(e)
                speak("Sorry. I am not able to send this email")

Run it

command

C:\Users\Your Name\basicchatbot> python basicchatbot.py
Good Evening!
I am Jarvis Sir. Please tell me how may I help you
Listening...
Recognizing...
User said: open YouTube

How it fits together

Read from the top: this is what runs when you execute the file, and which function calls which. It is generated from the code, so it cannot drift from it.

diagram Diagram mermaid

  flowchart TD
  RUN(["python basicchatbot.py"])
  speak("speak")
  wishMe("wishMe")
  takeCommand("takeCommand")
  sendEmail("sendEmail")
  RUN --> wishMe
  RUN --> speak
  RUN --> takeCommand
  RUN --> sendEmail
  wishMe --> speak

Step-by-Step Explanation

1. Initialize TTS

basicchatbot.py

import pyttsx3
engine = pyttsx3.init("sapi5" if os.name == "nt" else None)
voices = engine.getProperty("voices")
engine.setProperty("voice", voices[0].id)
 
def speak(text: str):
    print(f"Jarvis: {text}")
    engine.say(text)
    engine.runAndWait()

• sapi5 is the Windows speech API; on macOS use nsss, on Linux espeak.
• voices[0] is usually male; voices[1] female on Windows.
• runAndWait() blocks until speech finishes — important because the assistant is otherwise a fire-and-forget design.

2. Wish based on time of day

basicchatbot.py

import datetime
def wish_me():
    hour = datetime.datetime.now().hour
    if   hour < 12:  speak("Good Morning!")
    elif hour < 18:  speak("Good Afternoon!")
    else:            speak("Good Evening!")
    speak("I am Jarvis. How may I help you?")

3. Capture and recognize speech

basicchatbot.py

import speech_recognition as sr
def take_command() -> str:
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Listening...")
        r.adjust_for_ambient_noise(source, duration=0.5)
        r.pause_threshold = 1
        try:
            audio = r.listen(source, timeout=5, phrase_time_limit=8)
        except sr.WaitTimeoutError:
            return ""
    try:
        print("Recognizing...")
        query = r.recognize_google(audio, language="en-in")
        print(f"User said: {query}")
        return query.lower()
    except sr.UnknownValueError:
        speak("Sorry, I did not catch that.")
        return ""
    except sr.RequestError as e:
        speak("Speech service unavailable.")
        return ""

Important upgrades over the naïve version:

• adjust_for_ambient_noise calibrates for background hum.
• timeout and phrase_time_limit prevent the call from hanging forever.
• Specific exception handling distinguishes “could not hear you” from “Google is offline.”

4. The if/elif command chain (start here)

basicchatbot.py

import webbrowser, wikipedia, os
while True:
    query = take_command()
    if not query: continue
 
    if "wikipedia" in query:
        speak("Searching Wikipedia.")
        topic = query.replace("wikipedia", "").strip()
        try:
            summary = wikipedia.summary(topic, sentences=2)
            speak(summary)
        except wikipedia.exceptions.PageError:
            speak("I could not find that topic.")
    elif "open youtube" in query:
        webbrowser.open("https://youtube.com")
    elif "open google" in query:
        webbrowser.open("https://google.com")
    elif "the time" in query:
        speak(datetime.datetime.now().strftime("It is %H %M."))
    elif "quit" in query or "exit" in query:
        speak("Goodbye."); break

This works — but the chain becomes unmanageable past ten commands. The next section fixes that.

Refactor: Intent Registry

Replace the if/elif chain with a list of (matcher, handler) pairs:

intents.py

INTENTS = []
 
def intent(*keywords):
    def deco(fn):
        INTENTS.append((keywords, fn))
        return fn
    return deco
 
@intent("wikipedia")
def do_wiki(query):
    topic = query.replace("wikipedia", "").strip()
    speak(wikipedia.summary(topic, sentences=2))
 
@intent("open youtube", "play youtube")
def do_youtube(query):
    webbrowser.open("https://youtube.com")
 
@intent("the time", "what time")
def do_time(query):
    speak(datetime.datetime.now().strftime("It is %H %M."))
 
def dispatch(query):
    for kws, fn in INTENTS:
        if any(k in query for k in kws):
            return fn(query)
    speak("I don't know how to do that yet.")

Adding a new command is now one decorator, no menu re-shuffling. Each handler is independently testable.

Fuzzy Matching

Exact substring matching breaks on typos (“youtuve” instead of “youtube”). Use difflib:

fuzzy.py

from difflib import get_close_matches
def best_intent(query):
    all_words = [k for kws, _ in INTENTS for k in kws]
    return get_close_matches(query, all_words, n=1, cutoff=0.6)

For real-world intent matching, consider rapidfuzz (faster) or train a small classifier with scikit-learn over labeled examples.

Sending Email Safely

The original tutorial included an email handler with hardcoded credentials. Replace with environment variables and an App Password:

email.py

import smtplib, os
from email.mime.text import MIMEText
 
def send_email(to_addr: str, body: str):
    sender = os.environ["GMAIL_USER"]
    password = os.environ["GMAIL_APP_PASSWORD"]
    msg = MIMEText(body)
    msg["Subject"] = "From Jarvis"
    msg["From"] = sender
    msg["To"] = to_addr
    with smtplib.SMTP("smtp.gmail.com", 587) as s:
        s.starttls()
        s.login(sender, password)
        s.send_message(msg)

See Basic Email Sender for the full breakdown.

Common Mistakes

Problem	Cause	Fix
OSError: No Default Input Device	No mic, or system muted	Check OS sound settings
pyaudio install fails	Missing PortAudio	Use OS-specific install steps above
Recognizer hangs	No timeout set	Pass timeout=5 to r.listen()
TTS sounds robotic	Default voice	Try voices[1] or install a better engine
Mic always hears its own output	TTS and STT both running	Use headphones, or pause listening during runAndWait
400 Bad Request from Google	No internet	Catch sr.RequestError, offer fallback
Same word matches two intents	Overlapping keywords	Order intents most-specific first

Privacy & Ethics

Voice assistants raise real concerns:

• The mic is always live during listen() — pause when not in use.
• Audio is sent to Google’s servers by default. Use an offline recognizer (Vosk, Whisper) for privacy-sensitive deployments.
• Never log raw audio or transcripts without explicit user consent.
• Hardcoded passwords in source are a constant temptation in tutorials — use env vars or keyring.

If you ship the assistant to friends/family:

• Display a clear “listening” indicator.
• Provide a one-key mute.
• Document what data leaves the device.

Variations to Try

1. Wake word

Only respond after hearing “hey jarvis”. Libraries: pvporcupine (best, free for personal), snowboy (deprecated but widely used).

2. Offline speech recognition

Replace recognize_google with Vosk (small, ~50 MB models) or OpenAI whisper running locally. No internet, no third-party logging.

3. LLM-powered fallback

When no built-in intent matches, send the query to an LLM:

llm.py

from anthropic import Anthropic
client = Anthropic()
def llm_answer(query):
    r = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=500,
        messages=[{"role": "user", "content": query}])
    return r.content[0].text

Now your assistant can chat about anything — but it costs per-call and needs an API key.

4. Smart-home control

Integrate with Home Assistant or Hue APIs: “turn off the lights”.

5. Music player

Replace os.startfile with Basic Music Player — actual play/pause/next controls.

6. Calendar / weather / news

Pull from APIs: Google Calendar, OpenWeatherMap, NewsAPI.

7. Cross-platform launcher

platform.system() to pick the right subprocess command for opening VS Code on Windows/macOS/Linux.

8. Conversation memory

Keep a short rolling history so “open it” works after “find me Python docs”.

9. Multi-language

recognize_google accepts language="hi-IN" (Hindi), "fr-FR", etc. Pair with a TTS voice that matches.

10. Web GUI

Run the assistant in a Flask app with a “press to talk” button. Streams the response back as text and synthesized audio.

Architecture for the Real Version

text

microphone
   ↓
[Wake-word detector]  ← optional
   ↓
[Speech-to-text]  (Vosk / Whisper / Google)
   ↓
[Intent classifier]  ← if/elif → registry → ML model
   ↓
[Handler]  ← Wikipedia / webbrowser / smarthome / LLM fallback
   ↓
[Text-to-speech]  (pyttsx3 / ElevenLabs / Azure)
   ↓
speaker

Each stage is swappable. Start with the simplest version of each, then upgrade where it matters.

Real-World Applications

• Personal automation — launch apps, run scripts.
• Accessibility — voice control for users with limited mobility.
• Smart home — Alexa/Google Home clones using your own voice.
• Customer-service bots — but rate-limit and validate inputs aggressively.
• Hands-free workflows — cooking, garage, workshop.

Educational Value

• Microphone I/O — capturing audio in Python.
• External APIs — Wikipedia, Google STT.
• Pattern matching — keyword vs. fuzzy vs. classifier.
• Intent-dispatch — design pattern used by every voice assistant.
• Cross-platform concerns — TTS engines differ per OS.
• Privacy mindset — every audio capture is a trust boundary.

Next Steps

• Refactor to the intent-registry pattern.
• Move secrets to environment variables.
• Add wake-word detection so the mic is not always processing.
• Switch to offline STT (Vosk / Whisper) for privacy.
• Wire an LLM fallback for the long tail of queries.
• Cross-link with Basic Email Sender and Basic Music Player.

Conclusion

You built a working voice assistant — mic in, action out, voice back — with off-the-shelf libraries. The same architecture (STT → intent → handler → TTS) underlies every commercial assistant from Alexa to Siri. The difference is the quality of each stage, not the shape of the pipeline. Full source on GitHub. Find more automation projects on Python Central Hub.