Skip to content
Python Central Hub

Basic Chatbot (Jarvis-style)

View on GitHub

Abstract

A “Jarvis”-style voice assistant is one of the most rewarding beginner-to-intermediate Python projects. In ~100 lines you wire together a microphone, speech-to-text, a command dispatcher, web automation, and a text-to-speech engine — and end up with something that genuinely responds when you speak. In this tutorial you build that classic assistant (open YouTube, search Wikipedia, tell the time, send mail), then refactor the if/elif chain into a clean intent registry, add fuzzy matching, and finish with a sketch of an LLM-powered upgrade using the OpenAI / Anthropic / local-LLM APIs.

You will learn:

  • How speech_recognitionspeech_recognition captures and transcribes microphone input.
  • How pyttsx3pyttsx3 produces speech offline.
  • The intent-dispatch pattern (vs. growing ifif/elifelif chains).
  • How to integrate Wikipedia, webbrowserwebbrowser, and SMTP cleanly.
  • How to design assistants safely (mic privacy, command validation).

Prerequisites

  • Python 3.6 or above.
  • Microphone and speakers.
  • A code editor or IDE.
  • Internet connection (for Google’s speech recognition and Wikipedia).

Install Dependencies

install
pip install pyttsx3 SpeechRecognition wikipedia pyaudio
install
pip install pyttsx3 SpeechRecognition wikipedia pyaudio

If pyaudiopyaudio fails on Windows:

windows
pip install pipwin
pipwin install pyaudio
windows
pip install pipwin
pipwin install pyaudio

On macOS:

macos
brew install portaudio
pip install pyaudio
macos
brew install portaudio
pip install pyaudio

On Linux:

linux
sudo apt install portaudio19-dev python3-pyaudio
pip install pyaudio
linux
sudo apt install portaudio19-dev python3-pyaudio
pip install pyaudio

Getting Started

Create the project

  1. Create folder basicchatbotbasicchatbot.
  2. Inside, create basicchatbot.pybasicchatbot.py.

Write the code

Basic ChatbotSource
Basic Chatbot
# Basic Chat Bot
# Credits: https://www.youtube.com/watch?v=Lp9Ftuq2sVI&t=3008s
# Author: Code With Harry
 
import pyttsx3 #pip install pyttsx3
import speech_recognition as sr #pip install speechRecognition
import datetime
import wikipedia #pip install wikipedia
import webbrowser
import os
import smtplib
 
engine = pyttsx3.init('sapi5')
voices = engine.getProperty('voices')
engine.setProperty('voice', voices[0].id)
 
 
def speak(audio):
    engine.say(audio)
    engine.runAndWait()
 
 
def wishMe():
    hour = int(datetime.datetime.now().hour)
    if hour>=0 and hour<12:
        speak("Good Morning!")
 
    elif hour>=12 and hour<18:
        speak("Good Afternoon!")   
 
    else:
        speak("Good Evening!")  
 
    speak("I am Jarvis Sir. Please tell me how may I help you")       
 
def takeCommand():
    #It takes microphone input from the user and returns string output
 
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Listening...")
        r.pause_threshold = 1
        audio = r.listen(source)
 
    try:
        print("Recognizing...")    
        query = r.recognize_google(audio, language='en-in')
        print(f"User said: {query}\n")
 
    except Exception as e:
        # print(e)    
        print("Say that again please...")  
        return "None"
    return query
 
def sendEmail(to, content):
    server = smtplib.SMTP('smtp.gmail.com', 587)
    server.ehlo()
    server.starttls()
    server.login('youremail@gmail.com', 'your-password') # Enter your email and password at youremail@gmail.com and your-password
    server.sendmail('youremail@gmail.com', to, content) # Enter your email at youremail@gmail
    server.close()
 
if __name__ == "__main__":
    wishMe()
    while True:
    # if 1:
        query = takeCommand().lower()
 
        # Logic for executing tasks based on query
        if 'wikipedia' in query:
            speak('Searching Wikipedia...')
            query = query.replace("wikipedia", "")
            results = wikipedia.summary(query, sentences=2)
            speak("According to Wikipedia")
            print(results)
            speak(results)
 
        elif 'open youtube' in query:
            webbrowser.open("youtube.com")
 
        elif 'open google' in query:
            webbrowser.open("google.com")
 
        elif 'open stackoverflow' in query:
            webbrowser.open("stackoverflow.com")   
 
 
        elif 'play music' in query:
            music_dir = 'Your Music Directory' # Enter your music directory here like C:\\Users\\Ravi\\Music
            songs = os.listdir(music_dir)
            print(songs)    
            os.startfile(os.path.join(music_dir, songs[0]))
 
        elif 'the time' in query:
            strTime = datetime.datetime.now().strftime("%H:%M:%S")    
            speak(f"Sir, the time is {strTime}")
 
        elif 'open code' in query:
            codePath = "C:\\Users\\Ravi\\AppData\\Local\\Programs\\Microsoft VS Code\\Code.exe"
            os.startfile(codePath)
 
        elif 'email to ravi' in query:
            try:
                speak("What should I say?")
                content = takeCommand()
                to = "ravi@gmail.com"    
                sendEmail(to, content)
                speak("Email has been sent!")
            except Exception as e:
                print(e)
                speak("Sorry. I am not able to send this email")
Basic Chatbot
# Basic Chat Bot
# Credits: https://www.youtube.com/watch?v=Lp9Ftuq2sVI&t=3008s
# Author: Code With Harry
 
import pyttsx3 #pip install pyttsx3
import speech_recognition as sr #pip install speechRecognition
import datetime
import wikipedia #pip install wikipedia
import webbrowser
import os
import smtplib
 
engine = pyttsx3.init('sapi5')
voices = engine.getProperty('voices')
engine.setProperty('voice', voices[0].id)
 
 
def speak(audio):
    engine.say(audio)
    engine.runAndWait()
 
 
def wishMe():
    hour = int(datetime.datetime.now().hour)
    if hour>=0 and hour<12:
        speak("Good Morning!")
 
    elif hour>=12 and hour<18:
        speak("Good Afternoon!")   
 
    else:
        speak("Good Evening!")  
 
    speak("I am Jarvis Sir. Please tell me how may I help you")       
 
def takeCommand():
    #It takes microphone input from the user and returns string output
 
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Listening...")
        r.pause_threshold = 1
        audio = r.listen(source)
 
    try:
        print("Recognizing...")    
        query = r.recognize_google(audio, language='en-in')
        print(f"User said: {query}\n")
 
    except Exception as e:
        # print(e)    
        print("Say that again please...")  
        return "None"
    return query
 
def sendEmail(to, content):
    server = smtplib.SMTP('smtp.gmail.com', 587)
    server.ehlo()
    server.starttls()
    server.login('youremail@gmail.com', 'your-password') # Enter your email and password at youremail@gmail.com and your-password
    server.sendmail('youremail@gmail.com', to, content) # Enter your email at youremail@gmail
    server.close()
 
if __name__ == "__main__":
    wishMe()
    while True:
    # if 1:
        query = takeCommand().lower()
 
        # Logic for executing tasks based on query
        if 'wikipedia' in query:
            speak('Searching Wikipedia...')
            query = query.replace("wikipedia", "")
            results = wikipedia.summary(query, sentences=2)
            speak("According to Wikipedia")
            print(results)
            speak(results)
 
        elif 'open youtube' in query:
            webbrowser.open("youtube.com")
 
        elif 'open google' in query:
            webbrowser.open("google.com")
 
        elif 'open stackoverflow' in query:
            webbrowser.open("stackoverflow.com")   
 
 
        elif 'play music' in query:
            music_dir = 'Your Music Directory' # Enter your music directory here like C:\\Users\\Ravi\\Music
            songs = os.listdir(music_dir)
            print(songs)    
            os.startfile(os.path.join(music_dir, songs[0]))
 
        elif 'the time' in query:
            strTime = datetime.datetime.now().strftime("%H:%M:%S")    
            speak(f"Sir, the time is {strTime}")
 
        elif 'open code' in query:
            codePath = "C:\\Users\\Ravi\\AppData\\Local\\Programs\\Microsoft VS Code\\Code.exe"
            os.startfile(codePath)
 
        elif 'email to ravi' in query:
            try:
                speak("What should I say?")
                content = takeCommand()
                to = "ravi@gmail.com"    
                sendEmail(to, content)
                speak("Email has been sent!")
            except Exception as e:
                print(e)
                speak("Sorry. I am not able to send this email")

Run it

command
C:\Users\Your Name\basicchatbot> python basicchatbot.py
Good Evening!
I am Jarvis Sir. Please tell me how may I help you
Listening...
Recognizing...
User said: open YouTube
command
C:\Users\Your Name\basicchatbot> python basicchatbot.py
Good Evening!
I am Jarvis Sir. Please tell me how may I help you
Listening...
Recognizing...
User said: open YouTube

Step-by-Step Explanation

1. Initialize TTS

basicchatbot.py
import pyttsx3
engine = pyttsx3.init("sapi5" if os.name == "nt" else None)
voices = engine.getProperty("voices")
engine.setProperty("voice", voices[0].id)
 
def speak(text: str):
    print(f"Jarvis: {text}")
    engine.say(text)
    engine.runAndWait()
basicchatbot.py
import pyttsx3
engine = pyttsx3.init("sapi5" if os.name == "nt" else None)
voices = engine.getProperty("voices")
engine.setProperty("voice", voices[0].id)
 
def speak(text: str):
    print(f"Jarvis: {text}")
    engine.say(text)
    engine.runAndWait()
  • sapi5sapi5 is the Windows speech API; on macOS use nsssnsss, on Linux espeakespeak.
  • voices[0]voices[0] is usually male; voices[1]voices[1] female on Windows.
  • runAndWait()runAndWait() blocks until speech finishes — important because the assistant is otherwise a fire-and-forget design.

2. Wish based on time of day

basicchatbot.py
import datetime
def wish_me():
    hour = datetime.datetime.now().hour
    if   hour < 12:  speak("Good Morning!")
    elif hour < 18:  speak("Good Afternoon!")
    else:            speak("Good Evening!")
    speak("I am Jarvis. How may I help you?")
basicchatbot.py
import datetime
def wish_me():
    hour = datetime.datetime.now().hour
    if   hour < 12:  speak("Good Morning!")
    elif hour < 18:  speak("Good Afternoon!")
    else:            speak("Good Evening!")
    speak("I am Jarvis. How may I help you?")

3. Capture and recognize speech

basicchatbot.py
import speech_recognition as sr
def take_command() -> str:
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Listening...")
        r.adjust_for_ambient_noise(source, duration=0.5)
        r.pause_threshold = 1
        try:
            audio = r.listen(source, timeout=5, phrase_time_limit=8)
        except sr.WaitTimeoutError:
            return ""
    try:
        print("Recognizing...")
        query = r.recognize_google(audio, language="en-in")
        print(f"User said: {query}")
        return query.lower()
    except sr.UnknownValueError:
        speak("Sorry, I did not catch that.")
        return ""
    except sr.RequestError as e:
        speak("Speech service unavailable.")
        return ""
basicchatbot.py
import speech_recognition as sr
def take_command() -> str:
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Listening...")
        r.adjust_for_ambient_noise(source, duration=0.5)
        r.pause_threshold = 1
        try:
            audio = r.listen(source, timeout=5, phrase_time_limit=8)
        except sr.WaitTimeoutError:
            return ""
    try:
        print("Recognizing...")
        query = r.recognize_google(audio, language="en-in")
        print(f"User said: {query}")
        return query.lower()
    except sr.UnknownValueError:
        speak("Sorry, I did not catch that.")
        return ""
    except sr.RequestError as e:
        speak("Speech service unavailable.")
        return ""

Important upgrades over the naïve version:

  • adjust_for_ambient_noiseadjust_for_ambient_noise calibrates for background hum.
  • timeouttimeout and phrase_time_limitphrase_time_limit prevent the call from hanging forever.
  • Specific exception handling distinguishes “could not hear you” from “Google is offline.”

4. The if/elif command chain (start here)

basicchatbot.py
import webbrowser, wikipedia, os
while True:
    query = take_command()
    if not query: continue
 
    if "wikipedia" in query:
        speak("Searching Wikipedia.")
        topic = query.replace("wikipedia", "").strip()
        try:
            summary = wikipedia.summary(topic, sentences=2)
            speak(summary)
        except wikipedia.exceptions.PageError:
            speak("I could not find that topic.")
    elif "open youtube" in query:
        webbrowser.open("https://youtube.com")
    elif "open google" in query:
        webbrowser.open("https://google.com")
    elif "the time" in query:
        speak(datetime.datetime.now().strftime("It is %H %M."))
    elif "quit" in query or "exit" in query:
        speak("Goodbye."); break
basicchatbot.py
import webbrowser, wikipedia, os
while True:
    query = take_command()
    if not query: continue
 
    if "wikipedia" in query:
        speak("Searching Wikipedia.")
        topic = query.replace("wikipedia", "").strip()
        try:
            summary = wikipedia.summary(topic, sentences=2)
            speak(summary)
        except wikipedia.exceptions.PageError:
            speak("I could not find that topic.")
    elif "open youtube" in query:
        webbrowser.open("https://youtube.com")
    elif "open google" in query:
        webbrowser.open("https://google.com")
    elif "the time" in query:
        speak(datetime.datetime.now().strftime("It is %H %M."))
    elif "quit" in query or "exit" in query:
        speak("Goodbye."); break

This works — but the chain becomes unmanageable past ten commands. The next section fixes that.

Refactor: Intent Registry

Replace the if/elif chain with a list of (matcher, handler) pairs:

intents.py
INTENTS = []
 
def intent(*keywords):
    def deco(fn):
        INTENTS.append((keywords, fn))
        return fn
    return deco
 
@intent("wikipedia")
def do_wiki(query):
    topic = query.replace("wikipedia", "").strip()
    speak(wikipedia.summary(topic, sentences=2))
 
@intent("open youtube", "play youtube")
def do_youtube(query):
    webbrowser.open("https://youtube.com")
 
@intent("the time", "what time")
def do_time(query):
    speak(datetime.datetime.now().strftime("It is %H %M."))
 
def dispatch(query):
    for kws, fn in INTENTS:
        if any(k in query for k in kws):
            return fn(query)
    speak("I don't know how to do that yet.")
intents.py
INTENTS = []
 
def intent(*keywords):
    def deco(fn):
        INTENTS.append((keywords, fn))
        return fn
    return deco
 
@intent("wikipedia")
def do_wiki(query):
    topic = query.replace("wikipedia", "").strip()
    speak(wikipedia.summary(topic, sentences=2))
 
@intent("open youtube", "play youtube")
def do_youtube(query):
    webbrowser.open("https://youtube.com")
 
@intent("the time", "what time")
def do_time(query):
    speak(datetime.datetime.now().strftime("It is %H %M."))
 
def dispatch(query):
    for kws, fn in INTENTS:
        if any(k in query for k in kws):
            return fn(query)
    speak("I don't know how to do that yet.")

Adding a new command is now one decorator, no menu re-shuffling. Each handler is independently testable.

Fuzzy Matching

Exact substring matching breaks on typos (“youtuve” instead of “youtube”). Use difflibdifflib:

fuzzy.py
from difflib import get_close_matches
def best_intent(query):
    all_words = [k for kws, _ in INTENTS for k in kws]
    return get_close_matches(query, all_words, n=1, cutoff=0.6)
fuzzy.py
from difflib import get_close_matches
def best_intent(query):
    all_words = [k for kws, _ in INTENTS for k in kws]
    return get_close_matches(query, all_words, n=1, cutoff=0.6)

For real-world intent matching, consider rapidfuzzrapidfuzz (faster) or train a small classifier with scikit-learn over labeled examples.

Sending Email Safely

The original tutorial included an email handler with hardcoded credentials. Replace with environment variables and an App Password:

email.py
import smtplib, os
from email.mime.text import MIMEText
 
def send_email(to_addr: str, body: str):
    sender = os.environ["GMAIL_USER"]
    password = os.environ["GMAIL_APP_PASSWORD"]
    msg = MIMEText(body)
    msg["Subject"] = "From Jarvis"
    msg["From"] = sender
    msg["To"] = to_addr
    with smtplib.SMTP("smtp.gmail.com", 587) as s:
        s.starttls()
        s.login(sender, password)
        s.send_message(msg)
email.py
import smtplib, os
from email.mime.text import MIMEText
 
def send_email(to_addr: str, body: str):
    sender = os.environ["GMAIL_USER"]
    password = os.environ["GMAIL_APP_PASSWORD"]
    msg = MIMEText(body)
    msg["Subject"] = "From Jarvis"
    msg["From"] = sender
    msg["To"] = to_addr
    with smtplib.SMTP("smtp.gmail.com", 587) as s:
        s.starttls()
        s.login(sender, password)
        s.send_message(msg)

See Basic Email Sender for the full breakdown.

Common Mistakes

ProblemCauseFix
OSError: No Default Input DeviceOSError: No Default Input DeviceNo mic, or system mutedCheck OS sound settings
pyaudiopyaudio install failsMissing PortAudioUse OS-specific install steps above
Recognizer hangsNo timeouttimeout setPass timeout=5timeout=5 to r.listen()r.listen()
TTS sounds roboticDefault voiceTry voices[1]voices[1] or install a better engine
Mic always hears its own outputTTS and STT both runningUse headphones, or pause listening during runAndWaitrunAndWait
400 Bad Request from GoogleNo internetCatch sr.RequestErrorsr.RequestError, offer fallback
Same word matches two intentsOverlapping keywordsOrder intents most-specific first

Privacy & Ethics

Voice assistants raise real concerns:

  • The mic is always live during listen()listen() — pause when not in use.
  • Audio is sent to Google’s servers by default. Use an offline recognizer (VoskVosk, WhisperWhisper) for privacy-sensitive deployments.
  • Never log raw audio or transcripts without explicit user consent.
  • Hardcoded passwords in source are a constant temptation in tutorials — use env vars or keyringkeyring.

If you ship the assistant to friends/family:

  • Display a clear “listening” indicator.
  • Provide a one-key mute.
  • Document what data leaves the device.

Variations to Try

1. Wake word

Only respond after hearing “hey jarvis”. Libraries: pvporcupinepvporcupine (best, free for personal), snowboysnowboy (deprecated but widely used).

2. Offline speech recognition

Replace recognize_googlerecognize_google with VoskVosk (small, ~50 MB models) or OpenAI whisperOpenAI whisper running locally. No internet, no third-party logging.

3. LLM-powered fallback

When no built-in intent matches, send the query to an LLM:

llm.py
from anthropic import Anthropic
client = Anthropic()
def llm_answer(query):
    r = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=500,
        messages=[{"role": "user", "content": query}])
    return r.content[0].text
llm.py
from anthropic import Anthropic
client = Anthropic()
def llm_answer(query):
    r = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=500,
        messages=[{"role": "user", "content": query}])
    return r.content[0].text

Now your assistant can chat about anything — but it costs per-call and needs an API key.

4. Smart-home control

Integrate with Home Assistant or Hue APIs: “turn off the lights”.

5. Music player

Replace os.startfileos.startfile with Basic Music Player — actual play/pause/next controls.

6. Calendar / weather / news

Pull from APIs: Google Calendar, OpenWeatherMap, NewsAPI.

7. Cross-platform launcher

platform.system()platform.system() to pick the right subprocesssubprocess command for opening VS Code on Windows/macOS/Linux.

8. Conversation memory

Keep a short rolling history so “open it” works after “find me Python docs”.

9. Multi-language

recognize_googlerecognize_google accepts language="hi-IN"language="hi-IN" (Hindi), "fr-FR""fr-FR", etc. Pair with a TTS voice that matches.

10. Web GUI

Run the assistant in a Flask app with a “press to talk” button. Streams the response back as text and synthesized audio.

Architecture for the Real Version

text
microphone
   ↓
[Wake-word detector]  ← optional
   ↓
[Speech-to-text]  (Vosk / Whisper / Google)
   ↓
[Intent classifier]  ← if/elif → registry → ML model
   ↓
[Handler]  ← Wikipedia / webbrowser / smarthome / LLM fallback
   ↓
[Text-to-speech]  (pyttsx3 / ElevenLabs / Azure)
   ↓
speaker
text
microphone
   ↓
[Wake-word detector]  ← optional
   ↓
[Speech-to-text]  (Vosk / Whisper / Google)
   ↓
[Intent classifier]  ← if/elif → registry → ML model
   ↓
[Handler]  ← Wikipedia / webbrowser / smarthome / LLM fallback
   ↓
[Text-to-speech]  (pyttsx3 / ElevenLabs / Azure)
   ↓
speaker

Each stage is swappable. Start with the simplest version of each, then upgrade where it matters.

Real-World Applications

  • Personal automation — launch apps, run scripts.
  • Accessibility — voice control for users with limited mobility.
  • Smart home — Alexa/Google Home clones using your own voice.
  • Customer-service bots — but rate-limit and validate inputs aggressively.
  • Hands-free workflows — cooking, garage, workshop.

Educational Value

  • Microphone I/O — capturing audio in Python.
  • External APIs — Wikipedia, Google STT.
  • Pattern matching — keyword vs. fuzzy vs. classifier.
  • Intent-dispatch — design pattern used by every voice assistant.
  • Cross-platform concerns — TTS engines differ per OS.
  • Privacy mindset — every audio capture is a trust boundary.

Next Steps

  • Refactor to the intent-registry pattern.
  • Move secrets to environment variables.
  • Add wake-word detection so the mic is not always processing.
  • Switch to offline STT (Vosk / Whisper) for privacy.
  • Wire an LLM fallback for the long tail of queries.
  • Cross-link with Basic Email Sender and Basic Music Player.

Conclusion

You built a working voice assistant — mic in, action out, voice back — with off-the-shelf libraries. The same architecture (STT → intent → handler → TTS) underlies every commercial assistant from Alexa to Siri. The difference is the quality of each stage, not the shape of the pipeline. Full source on GitHub. Find more automation projects on Python Central Hub.

If this helped you, consider buying me a coffee ☕

Buy me a coffee

Was this page helpful?

Let us know how we did