Advanced Spam Detection System

Abstract

Advanced Spam Detection System is a Python project that uses machine learning to classify messages as spam or not spam. The application features text preprocessing, model training, and a CLI interface, demonstrating best practices in NLP and classification.

Prerequisites

• Python 3.8 or above
• A code editor or IDE
• Basic understanding of NLP and machine learning
• Required libraries: scikit-learn, nltk, pandas

Before you Start

Install Python and the required libraries:

Install dependencies

pip install scikit-learn nltk pandas

Getting Started

Create a Project

1. Create a folder named advanced-spam-detection-system.
2. Open the folder in your code editor or IDE.
3. Create a file named advanced_spam_detection_system.py.
4. Copy the code below into your file.

Write the Code

Advanced Spam Detection System

pch.viewSource

Advanced Spam Detection System

"""Spam detection, scored the way a mail filter has to be scored.

The version this replaces trained a Naive Bayes classifier on a handful of
strings and printed a label for one message. It never reported accuracy,
never held anything out, and never faced the asymmetry that defines the
problem: a spam message in the inbox is an annoyance, and a real message in
the spam folder can be a missed job offer.

That asymmetry is the whole design. A filter is tuned for precision on the
spam class -- almost nothing legitimate should be caught -- and recall is
whatever is left over. This file measures both, at several thresholds, and
then measures what happens when the spammer adapts.

    python advanced_spam_detection_system.py
"""

import re

from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.metrics import precision_recall_fscore_support
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB

SPAM_TEMPLATES = [
    "WINNER! You have been selected for a {prize} prize. Claim now at {url}",
    "Congratulations, your account qualifies for a free {prize}. Reply YES",
    "URGENT: verify your account at {url} or it will be closed within 24 hours",
    "Cheap {product} online, no prescription needed, discreet delivery",
    "Make ${amount} a week working from home. No experience. Click {url}",
    "Your parcel is held at customs. Pay the {amount} fee at {url}",
    "Limited offer: {product} at 90% off today only. Buy now {url}",
    "You have unclaimed funds of ${amount}. Send your bank details to claim",
    # The hard cases: spam that reads like ordinary business mail. Without
    # these the classes share no vocabulary and every model scores 1.0000,
    # which measures the corpus rather than the classifier.
    "Hi {name}, please review the attached {document} and confirm the "
    "{amount} payment by Friday",
    "Reminder: your {product} subscription renews next week. Update details "
    "at {url}",
    "{name}, the invoice for last month is overdue. Settle {amount} here: "
    "{url}",
    "Following up on our {meeting} -- the {document} is ready for your "
    "signature at {url}",
]
HAM_TEMPLATES = [
    "Hi {name}, can we move the {meeting} to Thursday afternoon?",
    "Please find the {document} attached for review before Friday",
    "The build failed on main, looks like the {module} tests are flaky",
    "Thanks for lunch yesterday. I will send the {document} over tonight",
    "Reminder: {meeting} at 10am, room 4. Agenda attached",
    "Could you review my pull request when you get a chance, {name}?",
    "The invoice for last month is attached. Let me know if anything is off",
    "I am out on Friday, {name} is covering the {meeting}",
    # Legitimate mail that uses the vocabulary spam filters look for.
    "URGENT: production is down, joining the {meeting} now",
    "Your {product} licence renews on the 3rd, invoice for {amount} attached",
    "{name}, please confirm the {amount} refund went through today",
    "Congratulations on the promotion! Drinks after the {meeting}?",
    "Click {url} for the {document} -- sharepoint link, expires in 7 days",
]

FILLERS = {
    "prize": ["cash", "iPhone", "holiday", "gift card"],
    "url": ["http://bit.ly/x1", "www.claim-now.biz", "http://secure-verify.co"],
    "product": ["watches", "pills", "software", "handbags"],
    "amount": ["500", "1,200", "97", "3,000"],
    "name": ["Sam", "Priya", "Alex", "Jordan"],
    "meeting": ["standup", "retro", "planning call", "one to one"],
    "document": ["report", "spec", "invoice", "slides"],
    "module": ["auth", "billing", "search", "upload"],
}


def build_corpus(n_spam=400, n_ham=1600, seed=20260809):
    """Imbalanced on purpose: most mail is not spam."""
    import random
    rng = random.Random(seed)

    def fill(template):
        out = template
        for key, options in FILLERS.items():
            out = out.replace("{" + key + "}", rng.choice(options))
        return out

    texts = [fill(rng.choice(SPAM_TEMPLATES)) for _ in range(n_spam)]
    texts += [fill(rng.choice(HAM_TEMPLATES)) for _ in range(n_ham)]
    labels = [1] * n_spam + [0] * n_ham
    return texts, labels


def obfuscate(text):
    """What a spammer does the day after a filter starts working.

    Character substitutions defeat word-level features completely: `v1agra`
    and `viagra` share no token, so a model that learned the second has never
    seen the first.
    """
    swaps = {"a": "@", "i": "1", "o": "0", "e": "3", "s": "$"}
    return "".join(swaps.get(c, c) for c in text)


def score(model, vectorizer, texts, labels, threshold=0.5):
    probabilities = model.predict_proba(vectorizer.transform(texts))[:, 1]
    predicted = (probabilities >= threshold).astype(int)
    precision, recall, f1, _ = precision_recall_fscore_support(
        labels, predicted, average="binary", zero_division=0)
    caught_ham = int(((predicted == 1) & (labels == 0)).sum())
    return {"precision": precision, "recall": recall, "f1": f1,
            "ham_lost": caught_ham}


def main():
    print("Advanced Spam Detection System")
    import numpy as np

    texts, labels = build_corpus()
    labels = np.array(labels)
    print(f"  messages           : {len(texts):,} "
          f"({labels.sum()} spam, {labels.mean():.1%})")

    X_train, X_test, y_train, y_test = train_test_split(
        texts, labels, test_size=0.3, random_state=0, stratify=labels)
    print(f"  train / test       : {len(X_train):,} / {len(X_test):,}")

    print(f"\n{'features':>26} {'precision':>10} {'recall':>8} {'F1':>7} "
          f"{'real mail lost':>15}")
    print("  " + "-" * 70)
    fitted = {}
    for name, vectorizer in (
            ("word counts", CountVectorizer()),
            ("word tf-idf", TfidfVectorizer()),
            ("char 3-5 grams tf-idf",
             TfidfVectorizer(analyzer="char_wb", ngram_range=(3, 5)))):
        model = MultinomialNB()
        model.fit(vectorizer.fit_transform(X_train), y_train)
        fitted[name] = (model, vectorizer)
        result = score(model, vectorizer, X_test, y_test)
        print(f"  {name:>24} {result['precision']:>10.4f} "
              f"{result['recall']:>8.4f} {result['f1']:>7.4f} "
              f"{result['ham_lost']:>15}")

    model, vectorizer = fitted["word tf-idf"]
    print(f"\n  the same word tf-idf model at different thresholds:")
    print(f"    {'threshold':>10} {'precision':>10} {'recall':>8} "
          f"{'real mail lost':>15}")
    for threshold in (0.5, 0.7, 0.9, 0.99):
        result = score(model, vectorizer, X_test, y_test, threshold)
        print(f"    {threshold:>10.2f} {result['precision']:>10.4f} "
              f"{result['recall']:>8.4f} {result['ham_lost']:>15}")
    print("    A mail filter is tuned on this table, not on F1. Losing one")
    print("    real message is worse than passing a hundred spam, so the")
    print("    right-hand column is the constraint and recall is whatever")
    print("    remains once it is satisfied.")

    # And what happens when the spammer adapts.
    spam_texts = [t for t, l in zip(X_test, y_test) if l == 1]
    obfuscated = [obfuscate(t) for t in spam_texts]
    print(f"\n  the spammer substitutes characters (v1agra for viagra):")
    print(f"    {'features':>24} {'spam caught before':>19} "
          f"{'after':>8}")
    for name, (model, vectorizer) in fitted.items():
        before = model.predict(vectorizer.transform(spam_texts)).mean()
        after = model.predict(vectorizer.transform(obfuscated)).mean()
        print(f"    {name:>24} {before:>19.1%} {after:>8.1%}")
    print("    `claim` and `cl@1m` share no token, so every substituted word")
    print("    is a word the model has never seen. What survives for the word")
    print("    models is the untouched text -- capitals, numbers, punctuation")
    print("    -- which is why they degrade rather than collapse.")
    print("    The character model loses least: its features span the")
    print("    substitutions, so `cl@1m` still shares n-grams with `claim`.")

    print(f"\n  example messages and what the word model says:")
    for text in (spam_texts[0], obfuscate(spam_texts[0]),
                 [t for t, l in zip(X_test, y_test) if l == 0][0]):
        model, vectorizer = fitted["word tf-idf"]
        p = model.predict_proba(vectorizer.transform([text]))[0, 1]
        print(f"    P(spam)={p:.3f}  {text[:58]}")


if __name__ == "__main__":
    main()

Example Usage

Run the spam detector

python advanced_spam_detection_system.py

What it produces

Running the file exactly as it ships takes 3.4 s and prints:

python advanced_spam_detection_system.py

Advanced Spam Detection System
  messages           : 2,000 (400 spam, 20.0%)
  train / test       : 1,400 / 600
 
                  features  precision   recall      F1  real mail lost
  ----------------------------------------------------------------------
               word counts     1.0000   1.0000  1.0000               0
               word tf-idf     1.0000   1.0000  1.0000               0
     char 3-5 grams tf-idf     1.0000   0.9000  0.9474               0
 
  the same word tf-idf model at different thresholds:
     threshold  precision   recall  real mail lost
          0.50     1.0000   1.0000               0
          0.70     1.0000   0.9000               0
          0.90     1.0000   0.9000               0
          0.99     1.0000   0.7833               0
    A mail filter is tuned on this table, not on F1. Losing one
    real message is worse than passing a hundred spam, so the
    right-hand column is the constraint and recall is whatever
    remains once it is satisfied.
...

The first 20 of 37 lines are shown; the run continues past this point.

How it fits together

Read from the top: this is what runs when you execute the file, and which function calls which. It is generated from the code, so it cannot drift from it.

diagram Diagram mermaid

  flowchart TD
  RUN(["python advanced_spam_detection_system.py"])
  SpamDetector["SpamDetector
class"]
  CLI["CLI
class"]
  RUN --> SpamDetector
  CLI --> SpamDetector

Explanation

Key Features

• Text Preprocessing: Tokenization, stopword removal, and vectorization.
• Model Training: Uses Naive Bayes for classification.
• Prediction: Classifies new messages as spam or not spam.
• Error Handling: Validates inputs and manages exceptions.
• CLI Interface: Interactive command-line usage.

Code Breakdown

1. What it imports (lines 17–22)

advanced_spam_detection_system.py

import re
 
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.metrics import precision_recall_fscore_support
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB

2. build_corpus — the function (lines 74–88)

advanced_spam_detection_system.py

def build_corpus(n_spam=400, n_ham=1600, seed=20260809):
    """Imbalanced on purpose: most mail is not spam."""
    import random
    rng = random.Random(seed)
 
    def fill(template):
        out = template
        for key, options in FILLERS.items():
            out = out.replace("{" + key + "}", rng.choice(options))
        return out
 
    texts = [fill(rng.choice(SPAM_TEMPLATES)) for _ in range(n_spam)]
    texts += [fill(rng.choice(HAM_TEMPLATES)) for _ in range(n_ham)]
    labels = [1] * n_spam + [0] * n_ham
    return texts, labels

3. obfuscate — the function (lines 91–99)

advanced_spam_detection_system.py

def obfuscate(text):
    """What a spammer does the day after a filter starts working.
 
    Character substitutions defeat word-level features completely: `v1agra`
    and `viagra` share no token, so a model that learned the second has never
    seen the first.
    """
    swaps = {"a": "@", "i": "1", "o": "0", "e": "3", "s": "$"}
    return "".join(swaps.get(c, c) for c in text)

4. score — the function (lines 102–109)

advanced_spam_detection_system.py

def score(model, vectorizer, texts, labels, threshold=0.5):
    probabilities = model.predict_proba(vectorizer.transform(texts))[:, 1]
    predicted = (probabilities >= threshold).astype(int)
    precision, recall, f1, _ = precision_recall_fscore_support(
        labels, predicted, average="binary", zero_division=0)
    caught_ham = int(((predicted == 1) & (labels == 0)).sum())
    return {"precision": precision, "recall": recall, "f1": f1,
            "ham_lost": caught_ham}

5. main — the function (lines 112–177)

advanced_spam_detection_system.py

def main():
    print("Advanced Spam Detection System")
    import numpy as np
 
    texts, labels = build_corpus()
    labels = np.array(labels)
    print(f"  messages           : {len(texts):,} "
          f"({labels.sum()} spam, {labels.mean():.1%})")
 
    X_train, X_test, y_train, y_test = train_test_split(
        texts, labels, test_size=0.3, random_state=0, stratify=labels)
    print(f"  train / test       : {len(X_train):,} / {len(X_test):,}")
 
    print(f"\n{'features':>26} {'precision':>10} {'recall':>8} {'F1':>7} "
          f"{'real mail lost':>15}")
    print("  " + "-" * 70)
    fitted = {}
    for name, vectorizer in (
    # ... 42 more lines in the file ...
    print(f"\n  example messages and what the word model says:")
    for text in (spam_texts[0], obfuscate(spam_texts[0]),
                 [t for t, l in zip(X_test, y_test) if l == 0][0]):
        model, vectorizer = fitted["word tf-idf"]
        p = model.predict_proba(vectorizer.transform([text]))[0, 1]
        print(f"    P(spam)={p:.3f}  {text[:58]}")

The file defines 4 top-level symbols in all; the whole thing is above under Write the Code.

Features

• Machine Learning-Based Classification: High-accuracy spam detection
• Modular Design: Separate functions for preprocessing and prediction
• Error Handling: Manages invalid inputs and exceptions
• Production-Ready: Scalable and maintainable code

Next Steps

Enhance the project by:

• Integrating with real-world datasets
• Adding support for more languages
• Creating a GUI with Tkinter or a web app with Flask
• Supporting batch predictions
• Adding evaluation metrics (precision, recall)
• Unit testing for reliability

Educational Value

This project teaches:

• NLP Fundamentals: Text preprocessing and classification
• Software Design: Modular, maintainable code
• Error Handling: Writing robust Python code

Real-World Applications

• Email Filtering
• Messaging Apps
• Enterprise Security
• Educational Tools

Conclusion

Advanced Spam Detection System demonstrates how to build a scalable and accurate spam classifier using Python. With modular design and extensibility, this project can be adapted for real-world applications in email, messaging, and more. For more advanced projects, visit Python Central Hub.

Pitfalls

• The two mistakes do not cost the same. Spam in the inbox is an annoyance; a real message in the spam folder can be a missed job offer. Any metric that weights precision and recall equally — F1 among them — has quietly assumed they do.
• Tune on the threshold table, not on a single score. Measured on the word tf-idf model: recall 1.0000 at threshold 0.50 and 0.7833 at 0.99, with zero real mail lost at every setting tested. The right-hand column is the constraint; recall is whatever is left once it is satisfied.
• A separable corpus measures the corpus, not the classifier. The first version of this file scored 1.0000 everywhere because spam and ham shared no vocabulary. Adding spam that reads like business mail — invoices, renewals, “please confirm the payment” — and ham that uses spam vocabulary is what makes the numbers mean anything.
• The adversary adapts, and word features have no defence. Measured: substituting characters (cl@1m for claim) takes word tf-idf from 100.0% of spam caught to 68.3%. Character n-grams go from 90.0% to 87.5%, because their features span the substitution.
• Class imbalance is the normal case. 400 spam against 1,600 real messages here, and real inboxes are more skewed still. Accuracy on that split is not a useful number.

Recap

• Measured: 2,000 messages (20.0% spam), 1,400 train / 600 test.
• Word counts and word tf-idf both reach precision 1.0000 and recall 1.0000; character n-grams reach 1.0000 / 0.9000.
• Under character substitution the ranking inverts: character n-grams lose 2.5 points, word tf-idf loses 31.7.
• Naive Bayes is the classic choice here because it is fast, needs little data, and its independence assumption is wrong in a way that rarely hurts text classification.

pch.quizTag pch.quizDefaultTitle

1. Why is F1 a poor tuning target for a mail filter?

   A It is too slow to compute B It weights precision and recall equally, which asserts that losing a real message costs the same as letting one spam through — and for mail that is false C It cannot handle class imbalance D It is undefined when recall is zero
   pch.quizShowAnswer

   B — It weights precision and recall equally, which asserts that losing a real message costs the same as letting one spam through — and for mail that is false

2. Character substitution took word tf-idf from 100% of spam caught to 68.3%, and character n-grams from 90.0% to 87.5%. Why the difference?

   A Character n-grams were trained on more data B A substituted word is a token the word model has never seen, while character n-grams still overlap with the original spelling around the changed letters C tf-idf weights are more sensitive to noise D Naive Bayes handles characters better
   pch.quizShowAnswer

   B — A substituted word is a token the word model has never seen, while character n-grams still overlap with the original spelling around the changed letters

3. The first version of this corpus gave every model precision and recall of 1.0000. What was wrong?

   A The models were overfitted B Spam and ham shared no vocabulary, so the task was trivial — the score measured how the corpus was built rather than how well anything classified C The test set was too small D The labels were leaked into the features
   pch.quizShowAnswer

   B — Spam and ham shared no vocabulary, so the task was trivial — the score measured how the corpus was built rather than how well anything classified

Try it yourself

pch.exerciseTag pch.exerciseTitle