Twitter Sentiment Analysis

Goal

Given tweets and sentiment labels (or sentiment scores):

• Clean text
• Explore sentiment distribution
• Visualize sentiment over time (if timestamps exist)

Analysis pipeline

diagram Twitter sentiment analysis pipeline mermaid

  flowchart LR
  A["Raw tweets"] --> B["Clean text
(urls, mentions, hashtags)"]
  B --> C["Explore
(sentiment counts)"]
  C --> D["Visualize
(distribution / trend)"]
  D --> E["Conclude
(dominant sentiment)"]

From raw noisy tweets to a clean sentiment distribution.

From the book

McKinney faces the same problem parsing Bitly’s "a" (user agent) field: one raw string packs URLs, symbols, and noise together, so he strips it down to the first useful token before counting anything. Tweets need the same treatment — strip URLs, @mentions, and # symbols with regex before you count or model sentiment, or the noise will drown out the signal.

Step 1: Load

Load tweets

import pandas as pd
 
df = pd.read_csv("data/tweets.csv")
print(df.head())

Step 2: Basic text cleaning

Clean text

import re
 
def clean_tweet(s: str) -> str:
    s = re.sub(r"http\S+", "", s)       # remove URLs
    s = re.sub(r"@\w+", "", s)         # remove mentions
    s = re.sub(r"#", "", s)            # remove hashtag symbol
    s = re.sub(r"\s+", " ", s).strip() # normalize whitespace
    return s
 
if "text" in df.columns:
    df["text_clean"] = df["text"].astype(str).apply(clean_tweet)

Step 3: Sentiment distribution

Sentiment counts

import seaborn as sns
import matplotlib.pyplot as plt
 
plt.figure(figsize=(7, 4))
sns.countplot(data=df, x="sentiment")
plt.title("Sentiment distribution")
plt.tight_layout()
plt.show()

Visualize it

sketch Sentiment distribution p5.js Pause Restart

Bar height shows how many tweets fall into each sentiment bucket.

Deliverable

• Most common sentiment
• Examples of strongly positive/negative tweets
• Trend over time (optional)

🧪 Try It Yourself

Exercise 1 – Strip URLs from text with regex

pch.exerciseTag pch.exerciseTitle

Exercise 2 – Sentiment counts

pch.exerciseTag pch.exerciseTitle

Exercise 3 – Filter tweets mentioning a keyword

pch.exerciseTag pch.exerciseTitle