Goal
Given tweets and sentiment labels (or sentiment scores):
- Clean text
- Explore sentiment distribution
- Visualize sentiment over time (if timestamps exist)
Analysis pipeline
flowchart LR A["Raw tweets"] --> B["Clean text
(urls, mentions, hashtags)"] B --> C["Explore
(sentiment counts)"] C --> D["Visualize
(distribution / trend)"] D --> E["Conclude
(dominant sentiment)"]
Step 1: Load
import pandas as pd
df = pd.read_csv("data/tweets.csv")
print(df.head())Step 2: Basic text cleaning
import re
def clean_tweet(s: str) -> str:
s = re.sub(r"http\S+", "", s) # remove URLs
s = re.sub(r"@\w+", "", s) # remove mentions
s = re.sub(r"#", "", s) # remove hashtag symbol
s = re.sub(r"\s+", " ", s).strip() # normalize whitespace
return s
if "text" in df.columns:
df["text_clean"] = df["text"].astype(str).apply(clean_tweet)Step 3: Sentiment distribution
import seaborn as sns
import matplotlib.pyplot as plt
plt.figure(figsize=(7, 4))
sns.countplot(data=df, x="sentiment")
plt.title("Sentiment distribution")
plt.tight_layout()
plt.show()Visualize it
Deliverable
- Most common sentiment
- Examples of strongly positive/negative tweets
- Trend over time (optional)
Check your understanding
Why clean tweets with regex before counting or modeling sentiment?
The project strips noise first so later counts and models see the useful text.
Show answer
A — URLs, mentions and hashtag symbols are noise that can drown out the signal — The project strips noise first so later counts and models see the useful text.
Which pattern in clean_tweet removes URLs?
The first re.sub removes everything from http up to the next whitespace.
Show answer
A — The substitution for text starting with http — The first re.sub removes everything from http up to the next whitespace.
Why apply astype(str) before apply(clean_tweet)?
re.sub fails on NaN floats, so everything is converted to text first.
Show answer
A — Missing or non-string cells would otherwise break the regex functions — re.sub fails on NaN floats, so everything is converted to text first.
Which chart shows the sentiment distribution in the project?
Sentiment is categorical, so counting rows per label is the right view.
Show answer
A — sns.countplot on the sentiment column — Sentiment is categorical, so counting rows per label is the right view.
🧪 Try It Yourself
Exercise 1 – Strip URLs from text with regex
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 2 – Sentiment counts
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.Exercise 3 – Filter tweets mentioning a keyword
Tab indents. Press Escape, then Tab, to leave the editor. Control plus Enter runs the code; Control plus Shift plus Enter submits it.
Press Run to see what your code prints.