{ "cells": [ { "cell_type": "markdown", "id": "7ab0a3be", "metadata": {}, "source": [ "# Day 3 β€” The Pattern Finder πŸ”\n", "\n", "> **Mission briefing:** A crate just landed in the Lab with no label β€” thousands of measurements and *nobody* to tell us what they mean. No answer key, no teacher. Your job: find the hidden structure in the data on your own. This is **unsupervised learning**, and by tonight it'll power your Studio's newest module.\n", "\n", "**Previously in the Lab:** yesterday you trained the 🐧 **Prediction Machine** β€” a classifier that learned penguin species *from labeled examples*. Today, the labels are gone.\n", "\n", "**Today you will:**\n", "- See the difference between **supervised** (with an answer key) and **unsupervised** (without) learning\n", "- Run **k-means** to discover groups no one told the machine about\n", "- Watch the machine **rediscover the penguin species** with zero labels\n", "- Use the **elbow method** to guess how many groups exist\n", "- Build a **photo color-compressor** with k-means\n", "- Squash 4-D and 64-D data into 2-D pictures with **PCA** and **t-SNE**\n", "\n", "**Today you unlock:** πŸ”“ **Pattern Finder** β€” upload any photo and it finds the dominant colors, repainting the image with just a handful of them.\n", "\n", "
A precise reading

K-means minimizes squared distances to centroids and may settle at a local solution. Cluster numbers are arbitrary, and clusters need not equal real species. The elbow is a heuristic. PCA preserves variance, not all information; t-SNE emphasizes local neighborhoods, so distances between islands and island sizes are not reliable measures of the original data.

\n" ] }, { "cell_type": "code", "execution_count": null, "id": "ac339af7", "metadata": {}, "outputs": [], "source": [ "# ╔═══════════════════════════════════════════════════════════════════╗\n", "# β•‘ SETUP β€” run me first! (identical in every Lab notebook) β•‘\n", "# β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•\n", "import os, sys, pathlib\n", "\n", "IN_COLAB = \"google.colab\" in sys.modules\n", "SMOKE = os.environ.get(\"SMOKE_TEST\", \"0\") == \"1\" # tiny fast run for automated tests\n", "\n", "REPO_URL = \"https://github.com/A-Halimi/ai-studio-internship.git\" # instructor: set once\n", "\n", "if IN_COLAB:\n", " if not pathlib.Path(\"/content/ai-studio-internship\").exists():\n", " !git clone -q {REPO_URL} /content/ai-studio-internship\n", " %cd /content/ai-studio-internship/notebooks/day03\n", "\n", "HERE = pathlib.Path.cwd()\n", "REPO = HERE.parents[1] # .../notebooks/dayNN -> repo root\n", "DATA_DIR = pathlib.Path(os.environ.get(\"COURSE_DATA_DIR\", REPO / \"data\"))\n", "SAMPLES_DIR = REPO / \"data_samples\" # small datasets shipped with the repo\n", "MODELS_DIR = REPO / \"ai_studio\" / \"models\" # where your Studio modules unlock\n", "for p in (REPO / \"data\", MODELS_DIR):\n", " p.mkdir(parents=True, exist_ok=True)\n", "\n", "import random\n", "import numpy as np\n", "SEED = 42\n", "random.seed(SEED); np.random.seed(SEED)\n", "print(f\"βœ… Setup done | Colab: {IN_COLAB} | Smoke test: {SMOKE}\")\n", "print(f\" data: {DATA_DIR}\")" ] }, { "cell_type": "markdown", "id": "d86b5bc3", "metadata": {}, "source": [ "## 1 Β· Two ways to learn\n", "\n", "Yesterday every penguin came with its species written on the tag. The machine's job was easy to *grade*: guess the species, peek at the tag, correct itself. That's **supervised learning** β€” learning **with an answer key**.\n", "\n", "Today the tags are gone. We have measurements, but nothing to check our guesses against. So we change the question. Instead of *\"what species is this?\"* we ask:\n", "\n", "> **\"Which of these penguins resemble each other?\"**\n", "\n", "Finding groups and structure **without any labels** is **unsupervised learning**. There's no single \"right answer\" to copy β€” the machine has to notice patterns by itself.\n", "\n", "It's not a toy. Unsupervised learning is how real systems:\n", "- **group customers** into look-alike audiences no one defined in advance,\n", "- **flag anomalies** β€” a fraudulent transaction is the point that fits no group,\n", "- **compress images** by replacing millions of colors with a handful (you'll do exactly this today).\n", "\n", "Let's open the crate." ] }, { "cell_type": "code", "execution_count": null, "id": "6ad360f6", "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "# The crate: penguin measurements, with the species column still attached so\n", "# we can peek LATER β€” but the machine will never get to see it.\n", "penguins = pd.read_csv(SAMPLES_DIR / \"penguins.csv\").dropna().reset_index(drop=True)\n", "print(f\"{len(penguins)} penguins in the crate\")\n", "penguins.head()" ] }, { "cell_type": "markdown", "id": "1408600c", "metadata": {}, "source": [ "## 2 Β· How k-means thinks β€” the pizza-shop story\n", "\n", "Imagine three pizza shops opening in a new city. Nobody has a map of where customers live, so the shops play a simple game, over and over:\n", "\n", "1. **Assign:** every customer orders from the *nearest* shop.\n", "2. **Update:** each shop moves to the *center* of the customers it just served.\n", "\n", "Do that a few times and the shops drift until they sit right in the middle of three natural neighborhoods. Customers stop switching. Done.\n", "\n", "That's **k-means**. Swap \"pizza shop\" for **cluster center** (a *centroid*) and \"customer\" for **data point** and you have the whole algorithm. The `k` is how many shops (clusters) you decide to place.\n", "\n", "Let's watch it happen on two penguin measurements β€” **bill length** and **flipper length** β€” standardized so neither one dominates just because it's measured in bigger numbers." ] }, { "cell_type": "code", "execution_count": null, "id": "61170d0f", "metadata": {}, "outputs": [], "source": [ "# Two measurements, standardized (mean 0, spread 1) so both count equally.\n", "feat2 = penguins[[\"bill_length_mm\", \"flipper_length_mm\"]].to_numpy(dtype=float)\n", "X2 = (feat2 - feat2.mean(axis=0)) / feat2.std(axis=0)\n", "\n", "# This one is INTERACTIVE: press β–Ά play (or drag the slider one round at a\n", "# time) and watch the red βœ•'s drift to the middle of their groups.\n", "try:\n", " import plotly.graph_objects as go\n", "except ImportError: # not installed? grab it, then retry\n", " %pip install -q plotly\n", " import plotly.graph_objects as go\n", "\n", "# Start the 3 \"shops\" (centroids) at 3 random penguins.\n", "rng = np.random.default_rng(SEED)\n", "centroids = X2[rng.choice(len(X2), size=3, replace=False)].copy()\n", "\n", "# Play the assign/update game for 8 rounds, saving a snapshot after each ASSIGN.\n", "snapshots = []\n", "for step in range(8):\n", " # 1) ASSIGN: each point joins its nearest centroid\n", " dists = np.linalg.norm(X2[:, None, :] - centroids[None, :, :], axis=2) # (N, 3)\n", " labels = dists.argmin(axis=1)\n", " snapshots.append((labels.copy(), centroids.copy()))\n", " # 2) UPDATE: each centroid hops to the mean of its assigned points\n", " for j in range(3):\n", " if np.any(labels == j):\n", " centroids[j] = X2[labels == j].mean(axis=0)\n", "\n", "CLUSTER_COLORS = [\"#440154\", \"#21918c\", \"#fde725\"] # viridis-style\n", "\n", "\n", "def frame_traces(labels, cents):\n", " return [\n", " go.Scatter(x=X2[:, 0], y=X2[:, 1], mode=\"markers\",\n", " marker=dict(size=6, color=[CLUSTER_COLORS[l] for l in labels]),\n", " hoverinfo=\"skip\", name=\"penguins\"),\n", " go.Scatter(x=cents[:, 0], y=cents[:, 1], mode=\"markers\",\n", " marker=dict(size=18, color=\"red\", symbol=\"x\",\n", " line=dict(color=\"black\", width=2)),\n", " hoverinfo=\"skip\", name=\"centroids\"),\n", " ]\n", "\n", "\n", "frames = [go.Frame(data=frame_traces(l, c), name=str(i))\n", " for i, (l, c) in enumerate(snapshots)]\n", "\n", "fig = go.Figure(data=frame_traces(*snapshots[0]), frames=frames)\n", "fig.update_layout(\n", " title=\"k-means converging β€” press β–Ά and watch the red βœ•'s find their groups\",\n", " xaxis=dict(title=\"bill length (standardized)\"),\n", " yaxis=dict(title=\"flipper length (standardized)\"),\n", " updatemenus=[dict(\n", " type=\"buttons\", x=0.05, xanchor=\"left\", y=1.15,\n", " buttons=[dict(label=\"β–Ά play\", method=\"animate\",\n", " args=[None, {\"frame\": {\"duration\": 900, \"redraw\": True},\n", " \"transition\": {\"duration\": 500}}])],\n", " )],\n", " sliders=[dict(\n", " currentvalue=dict(prefix=\"iteration \"),\n", " steps=[dict(label=str(i), method=\"animate\",\n", " args=[[str(i)], {\"mode\": \"immediate\",\n", " \"frame\": {\"duration\": 0, \"redraw\": True}}])\n", " for i in range(len(frames))],\n", " )],\n", " showlegend=False, width=700, height=550,\n", ")\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "7b44cb5f", "metadata": {}, "source": [ "### πŸ§ͺ Exercise 1 β€” Let scikit-learn do the clustering\n", "\n", "You watched the loop by hand; now use the real thing. `KMeans` runs those assign/update steps for you until the centroids stop moving.\n", "\n", "- Create `KMeans(n_clusters=3, n_init=10, random_state=SEED)` and fit it to `X2`.\n", "- Store each point's cluster number in `clusters`.\n", "\n", "`n_init=10` just means \"try 10 random starts and keep the best\" β€” k-means can land in a bad spot if it starts unlucky. You should see three clean blobs." ] }, { "cell_type": "code", "execution_count": null, "id": "39ab3d61", "metadata": {}, "outputs": [], "source": [ "from sklearn.cluster import KMeans\n", "import plotly.graph_objects as go\n", "\n", "km = KMeans(n_clusters=3, n_init=10, random_state=SEED)\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: km.fit_predict(X2) fits the model AND returns a cluster number per point\n", "...\n", "\n", "# Interactive result β€” hover any dot to read that penguin's REAL measurements.\n", "hover_data = np.column_stack([feat2, clusters])\n", "fig = go.Figure()\n", "fig.add_trace(go.Scatter(\n", " x=X2[:, 0], y=X2[:, 1], mode=\"markers\",\n", " marker=dict(size=6, color=[CLUSTER_COLORS[c] for c in clusters]),\n", " customdata=hover_data,\n", " hovertemplate=(\"bill %{customdata[0]:.1f} mm Β· \"\n", " \"flipper %{customdata[1]:.0f} mm
\"\n", " \"cluster %{customdata[2]:.0f}\"),\n", "))\n", "fig.add_trace(go.Scatter(\n", " x=km.cluster_centers_[:, 0], y=km.cluster_centers_[:, 1], mode=\"markers\",\n", " marker=dict(size=18, color=\"red\", symbol=\"x\", line=dict(color=\"black\", width=2)),\n", " hovertemplate=\"centroid\",\n", "))\n", "fig.update_layout(\n", " title=\"k-means found 3 groups β€” with no labels (hover the dots!)\",\n", " xaxis=dict(title=\"bill length (standardized)\"),\n", " yaxis=dict(title=\"flipper length (standardized)\"),\n", " showlegend=False, width=650, height=520,\n", ")\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "ba3a2eb6", "metadata": {}, "source": [ "### πŸ§ͺ Exercise 2 β€” The reveal: did it find the species?\n", "\n", "Here's the moment. The machine never saw a single species label β€” it only grouped penguins by shape. So did its groups mean anything?\n", "\n", "- Build a table comparing the machine's `clusters` against the **true species** with `pd.crosstab`.\n", "- Read it: does each cluster line up mostly with one species?\n", "\n", "Use `pd.crosstab(clusters, species_true, ...)`." ] }, { "cell_type": "code", "execution_count": null, "id": "dda02068", "metadata": {}, "outputs": [], "source": [ "species_true = penguins[\"species\"].to_numpy()\n", "\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: pd.crosstab(clusters, species_true) counts how often each pairing occurs\n", "...\n", "print(comparison)" ] }, { "cell_type": "markdown", "id": "reveal-viz-md", "metadata": {}, "source": [ "### πŸ‘€ Where exactly does it go wrong?\n", "\n", "The table gives you the *counts* β€” now let's **see** them. Below are the same penguins twice:\n", "\n", "- **Left:** colored by the machine's clusters. Each cluster borrows the color of the species it mostly captured (majority vote, read straight off your crosstab), so both panels speak the same color language.\n", "- **Right:** colored by the true species.\n", "\n", "Every penguin the machine put in the **wrong** group wears a **red ring** on *both* sides. Hover a ringed dot: almost all of them sit right on the border between two overlapping species β€” the machine wasn't being silly, those penguins really are ambiguous in just these two measurements.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "reveal-viz-code", "metadata": {}, "outputs": [], "source": [ "from plotly.subplots import make_subplots\n", "\n", "# Name each cluster after the species it mostly contains β€” majority vote,\n", "# read straight off the crosstab you just built.\n", "majority = comparison.idxmax(axis=1) # cluster number β†’ species name\n", "predicted_species = majority.loc[clusters].to_numpy()\n", "wrong = predicted_species != species_true\n", "print(f\"{wrong.sum()} of {len(penguins)} penguins landed in the 'wrong' cluster\")\n", "\n", "# Same color for a cluster and the species it captured β†’ panels are comparable.\n", "species_color = {sp: CLUSTER_COLORS[cl] for cl, sp in majority.items()}\n", "\n", "fig = make_subplots(\n", " rows=1, cols=2, shared_yaxes=True, horizontal_spacing=0.06,\n", " subplot_titles=(\"what the machine guessed\", \"the truth (species tags)\"))\n", "\n", "# LEFT β€” machine's view, one trace per cluster\n", "for cl in sorted(np.unique(clusters)):\n", " m = clusters == cl\n", " fig.add_trace(go.Scatter(\n", " x=X2[m, 0], y=X2[m, 1], mode=\"markers\",\n", " marker=dict(size=6, color=CLUSTER_COLORS[cl]),\n", " name=f'cluster {cl} β†’ \"{majority[cl]}\"', legendgroup=f\"c{cl}\",\n", " customdata=feat2[m],\n", " hovertemplate=(\"bill %{customdata[0]:.1f} mm Β· \"\n", " \"flipper %{customdata[1]:.0f} mm
\"\n", " f\"cluster {cl}\"),\n", " ), row=1, col=1)\n", "\n", "# RIGHT β€” the truth, one trace per species\n", "for sp in comparison.columns:\n", " m = species_true == sp\n", " fig.add_trace(go.Scatter(\n", " x=X2[m, 0], y=X2[m, 1], mode=\"markers\",\n", " marker=dict(size=6, color=species_color[sp]),\n", " name=sp, legendgroup=sp,\n", " customdata=feat2[m],\n", " hovertemplate=(\"bill %{customdata[0]:.1f} mm Β· \"\n", " \"flipper %{customdata[1]:.0f} mm
\"\n", " f\"{sp}\"),\n", " ), row=1, col=2)\n", "\n", "# THE CONFUSED ONES β€” a red ring on BOTH panels, hover to see the mix-up\n", "wrong_hover = np.column_stack([feat2[wrong],\n", " predicted_species[wrong], species_true[wrong]])\n", "for col in (1, 2):\n", " fig.add_trace(go.Scatter(\n", " x=X2[wrong, 0], y=X2[wrong, 1], mode=\"markers\",\n", " marker=dict(size=13, symbol=\"circle-open\", color=\"red\",\n", " line=dict(width=2)),\n", " name=\"β­• machine got it wrong\", legendgroup=\"wrong\",\n", " showlegend=(col == 1),\n", " customdata=wrong_hover,\n", " hovertemplate=(\"bill %{customdata[0]:.1f} mm Β· \"\n", " \"flipper %{customdata[1]:.0f} mm
\"\n", " \"machine said %{customdata[2]}, \"\n", " \"actually %{customdata[3]}\"),\n", " ), row=1, col=col)\n", "\n", "fig.update_layout(\n", " title=\"Machine's guess vs the truth β€” red rings mark the penguins it confused\",\n", " width=980, height=520,\n", " legend=dict(orientation=\"h\", y=-0.18),\n", ")\n", "fig.update_xaxes(title_text=\"bill length (standardized)\", row=1, col=1)\n", "fig.update_xaxes(title_text=\"bill length (standardized)\", row=1, col=2)\n", "fig.update_yaxes(title_text=\"flipper length (standardized)\", row=1, col=1)\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "7f0f5370", "metadata": {}, "source": [ "**Sit with that for a second.** Nobody told the machine what a \"Gentoo\" is. It just noticed that some penguins clump together by bill and flipper size β€” and those clumps *are* the species. The structure was hiding in the numbers all along; k-means only had to look.\n", "\n", "(The match isn't perfect β€” a few penguins sit on the border between groups, and two of the species really do overlap in just these two measurements. Later we'll hand k-means *all four* measurements and the picture sharpens.)" ] }, { "cell_type": "markdown", "id": "48f88e55", "metadata": {}, "source": [ "## 3 Β· How many groups are there, really?\n", "\n", "We *told* k-means to find 3. But what if we didn't know? Real mystery crates don't come with the number of groups printed on the side.\n", "\n", "Here's a trick. **Inertia** measures how tightly points hug their centroid β€” smaller is tighter. Add more clusters and inertia always drops (with one cluster per point it hits zero). But watch *how fast* it drops: there's usually a `k` where adding another cluster suddenly stops helping much. The curve bends like an elbow, and that bend is nature's hint at the true number of groups.\n", "\n", "### πŸ§ͺ Exercise 3 β€” Find the elbow\n", "\n", "- For each `k` from 1 to 8, fit `KMeans` and record its `.inertia_`.\n", "- Plot inertia versus `k` and look for the bend.\n", "\n", "You should see the curve drop steeply, then flatten around **k = 3** β€” exactly the number of penguin species." ] }, { "cell_type": "code", "execution_count": null, "id": "8c7fefa2", "metadata": {}, "outputs": [], "source": [ "ks = range(1, 9)\n", "inertias = []\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: after fitting, km_k.inertia_ is the tightness number to append\n", "...\n", "\n", "import plotly.graph_objects as go\n", "\n", "fig = go.Figure(go.Scatter(\n", " x=list(ks), y=inertias, mode=\"lines+markers\", marker=dict(size=9),\n", " hovertemplate=\"k = %{x}
inertia = %{y:.0f}\",\n", "))\n", "fig.add_annotation(x=3, y=inertias[2], text=\"the elbow πŸ‘ˆ\", ax=70, ay=-45,\n", " showarrow=True, arrowhead=2)\n", "fig.update_layout(\n", " title=\"The elbow method β€” hover the points, find the bend\",\n", " xaxis=dict(title=\"k (number of clusters)\"),\n", " yaxis=dict(title=\"inertia (lower = tighter clusters)\"),\n", " width=650, height=430,\n", ")\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "fffc10cb", "metadata": {}, "source": [ "## 4 Β· The visual wow: a photo is just a pile of colors\n", "\n", "Time for the module you're unlocking. Here's a fresh way to look at an image: **a color is just a point in 3-D space** β€” its red, green, and blue amounts. A photo with a million pixels is a cloud of a million points floating in that RGB cube.\n", "\n", "So what happens if we run k-means on the pixels of a photo? It finds the `k` most representative colors β€” the photo's **palette** β€” and we repaint every pixel with its nearest palette color. Fewer colors, almost the same picture. That's **color quantization**, and it's a genuine form of image compression.\n", "\n", "Let's grab a sample photo that ships with scikit-learn (no download needed)." ] }, { "cell_type": "code", "execution_count": null, "id": "7b767014", "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import load_sample_image\n", "\n", "china = load_sample_image(\"china.jpg\") / 255.0 # scale RGB from 0..255 to 0..1\n", "print(\"image shape:\", china.shape, \" (height, width, RGB)\")\n", "\n", "pixels = china.reshape(-1, 3) # one row per pixel\n", "print(\"that's\", len(pixels), \"pixels, each a point in 3-D color space\")\n", "\n", "\n", "def palette_bar(centers, labels, width=400, height=60):\n", " \"\"\"Draw the palette as a horizontal bar, widest (most common) color first.\"\"\"\n", " counts = np.bincount(labels, minlength=len(centers))\n", " order = np.argsort(-counts)\n", " bar = np.zeros((height, width, 3))\n", " x = 0\n", " for idx in order:\n", " w = int(round(counts[idx] / counts.sum() * width))\n", " bar[:, x:x + w] = centers[idx]\n", " x += w\n", " bar[:, x:] = centers[order[-1]] # fill any rounding gap\n", " return bar\n", "\n", "\n", "plt.figure(figsize=(6, 4))\n", "plt.imshow(china)\n", "plt.title(\"the original photo\")\n", "plt.axis(\"off\")\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "ef5cd381", "metadata": {}, "source": [ "### πŸ‘€ See it first: every pixel is a dot in a 3-D color cube\n", "\n", "Before we run any clustering, let's actually *look* at what k-means will see.\n", "\n", "A pixel is just 3 numbers β€” how much **Red**, **Green**, and **Blue** it contains. So every pixel is a **point in 3-D space**: the RGB color cube. The corner (0, 0, 0) is black, (1, 1, 1) is white, and every color lives somewhere in between.\n", "\n", "Our photo has 427 Γ— 640 = **273,280 pixels**, which means 273,280 dots floating in that cube. This is exactly like the penguin scatter plot β€” except each \"penguin\" is now a pixel, and instead of 2 measurements (bill, flipper) it has 3 (R, G, B).\n", "\n", "And just like the penguins, **there are no labels**. Nobody tells the machine \"this pixel is sky\" or \"this pixel is roof.\" But the dots still form natural clumps β€” because a real photo doesn't use random colors. A sky produces hundreds of thousands of pixels that are all *nearly the same blue*, so they pile up in one region of the cube. That hidden clumping is what k-means will discover on its own β€” the definition of **unsupervised learning**." ] }, { "cell_type": "code", "execution_count": null, "id": "997a3c42", "metadata": {}, "outputs": [], "source": [ "# Peek at the pixel cloud: 4,000 random pixels inside the RGB cube.\n", "# This plot is INTERACTIVE β€” drag to rotate πŸ–±οΈ, scroll to zoom, hover a dot to\n", "# read its exact R/G/B values.\n", "try:\n", " import plotly.graph_objects as go\n", "except ImportError: # not installed? grab it, then retry\n", " %pip install -q plotly\n", " import plotly.graph_objects as go\n", "\n", "rng_view = np.random.default_rng(SEED)\n", "peek = pixels[rng_view.choice(len(pixels), size=4_000, replace=False)]\n", "peek255 = (peek * 255).astype(int) # 0..255 for hover text\n", "dot_colors = [f\"rgb({r},{g},{b})\" for r, g, b in peek255]\n", "\n", "fig = go.Figure(go.Scatter3d(\n", " x=peek[:, 0], y=peek[:, 1], z=peek[:, 2],\n", " mode=\"markers\",\n", " marker=dict(size=2.5, color=dot_colors),\n", " customdata=peek255,\n", " hovertemplate=\"R %{customdata[0]} Β· G %{customdata[1]} Β· B %{customdata[2]}\",\n", "))\n", "fig.update_layout(\n", " title=\"the photo's pixels in the RGB color cube β€” drag me to rotate!\",\n", " scene=dict(\n", " xaxis=dict(title=\"Red\", range=[0, 1]),\n", " yaxis=dict(title=\"Green\", range=[0, 1]),\n", " zaxis=dict(title=\"Blue\", range=[0, 1]),\n", " aspectmode=\"cube\",\n", " ),\n", " width=750, height=650, margin=dict(l=0, r=0, t=50, b=0),\n", ")\n", "fig.show()\n", "\n", "print(\"Rotate the cube and look for the clumps:\")\n", "print(\" β€’ a pale blue/white clump near the (1,1,1) corner β†’ the SKY pixels\")\n", "print(\" β€’ a dark clump near (0,0,0) β†’ SHADOW pixels\")\n", "print(\" β€’ a greenish clump in the middle β†’ VEGETATION pixels\")\n", "print(\"Nobody labeled them β€” the clumps exist because photos reuse similar colors.\")" ] }, { "cell_type": "markdown", "id": "f92f7c16", "metadata": {}, "source": [ "### πŸ§ͺ Exercise 4 β€” Compress the photo to 8 colors\n", "\n", "Fitting k-means on all ~270,000 pixels is slow, so we use a classic shortcut: **fit on a random sample, then predict every pixel.** The palette barely changes, but it runs in a snap.\n", "\n", "- Fit `KMeans(n_clusters=8, n_init=4, random_state=SEED)` on the 20,000-pixel `sample`.\n", "- Use the fitted model to `.predict` a cluster for **every** pixel in `pixels` (store it in `labels_all`).\n", "\n", "Then run the given code to repaint the photo. Eight colors β€” see how close it still looks." ] }, { "cell_type": "code", "execution_count": null, "id": "4773efbb", "metadata": {}, "outputs": [], "source": [ "rng = np.random.default_rng(SEED)\n", "sample = pixels[rng.choice(len(pixels), size=20_000, replace=False)]\n", "\n", "km_img = KMeans(n_clusters=8, n_init=4, random_state=SEED)\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: fit on `sample` (fast), then km_img.predict(pixels) to label ALL pixels\n", "...\n", "\n", "palette = km_img.cluster_centers_\n", "repainted = palette[labels_all].reshape(china.shape)\n", "\n", "fig, axes = plt.subplots(1, 2, figsize=(11, 4.5))\n", "axes[0].imshow(china); axes[0].set_title(\"original (thousands of colors)\")\n", "axes[1].imshow(repainted); axes[1].set_title(\"repainted with only 8 colors\")\n", "for ax in axes:\n", " ax.axis(\"off\")\n", "plt.show()\n", "\n", "plt.figure(figsize=(6, 1.2))\n", "plt.imshow(palette_bar(palette, labels_all))\n", "plt.title(\"the 8-color palette k-means chose\")\n", "plt.axis(\"off\")\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "a6f860ae", "metadata": {}, "source": [ "### πŸ‘€ Where did those 8 colors come from? Back to the cube\n", "\n", "Remember the pixel cloud from before? Let's look at it again β€” but now with the 8 centroids k-means just found, drawn as red βœ•'s. **This is the pizza-shop game, played inside the color cube:** each βœ• drifted until it sat in the middle of a clump of similar colors. The centroid of a clump is the *average color* of its pixels β€” and those 8 averages are exactly the palette.\n", "\n", "**Try this:** rotate the cube to a nice angle, then click **AFTER**. Every pixel *snaps* to its nearest βœ• (the gray threads show a few of them flying home) and the whole cloud collapses onto just 8 points. **That snap is quantization** β€” and it's why the repainted photo has only 8 colors. Flip BEFORE/AFTER a few times and watch which colors get merged together." ] }, { "cell_type": "code", "execution_count": null, "id": "c471f911", "metadata": {}, "outputs": [], "source": [ "# One interactive cube, two views β€” use the BEFORE / AFTER buttons on top!\n", "# BEFORE: every pixel in its true color, plus the 8 centroids (red βœ•).\n", "# AFTER: every pixel snapped to its nearest centroid β€” only 8 colors left.\n", "# Thin gray threads show a few pixels \"flying\" to their centroid.\n", "# The camera keeps its angle when you switch, so rotate first, then toggle.\n", "import plotly.graph_objects as go\n", "\n", "peek_labels = km_img.predict(peek) # palette color of each peeked pixel\n", "snapped = palette[peek_labels]\n", "snapped_colors = [f\"rgb({r},{g},{b})\" for r, g, b in (snapped * 255).astype(int)]\n", "\n", "# a few \"snap threads\" from pixel β†’ its centroid (drawing all 4,000 would be a hairball)\n", "thread_idx = rng_view.choice(len(peek), size=120, replace=False)\n", "xs, ys, zs = [], [], []\n", "for i in thread_idx:\n", " c = palette[peek_labels[i]]\n", " xs += [peek[i, 0], c[0], None]\n", " ys += [peek[i, 1], c[1], None]\n", " zs += [peek[i, 2], c[2], None]\n", "\n", "trace_true = go.Scatter3d( # 0 β€” BEFORE cloud\n", " x=peek[:, 0], y=peek[:, 1], z=peek[:, 2], mode=\"markers\",\n", " marker=dict(size=2.5, color=dot_colors), name=\"true colors\",\n", " hovertemplate=\"R %{customdata[0]} Β· G %{customdata[1]} Β· B %{customdata[2]}\",\n", " customdata=peek255,\n", ")\n", "trace_quant = go.Scatter3d( # 1 β€” AFTER cloud (hidden at start)\n", " x=peek[:, 0], y=peek[:, 1], z=peek[:, 2], mode=\"markers\",\n", " marker=dict(size=2.5, color=snapped_colors), name=\"quantized\",\n", " hoverinfo=\"skip\", visible=False,\n", ")\n", "trace_threads = go.Scatter3d( # 2 β€” snap threads (hidden at start)\n", " x=xs, y=ys, z=zs, mode=\"lines\",\n", " line=dict(color=\"rgba(120,120,120,0.4)\", width=1),\n", " name=\"snap\", hoverinfo=\"skip\", visible=False,\n", ")\n", "trace_centroids = go.Scatter3d( # 3 β€” the 8 palette centroids\n", " x=palette[:, 0], y=palette[:, 1], z=palette[:, 2], mode=\"markers\",\n", " marker=dict(size=8, color=\"red\", symbol=\"x\", line=dict(color=\"black\", width=2)),\n", " name=\"palette (centroids)\",\n", " hovertemplate=\"palette color %{pointNumber}\",\n", ")\n", "\n", "fig = go.Figure([trace_true, trace_quant, trace_threads, trace_centroids])\n", "fig.update_layout(\n", " title=\"quantization, live: rotate the cube, then press AFTER and watch the cloud collapse\",\n", " updatemenus=[dict(\n", " type=\"buttons\", direction=\"right\", x=0.5, xanchor=\"center\", y=1.10,\n", " buttons=[\n", " dict(label=\"BEFORE β€” true colors\",\n", " method=\"update\", args=[{\"visible\": [True, False, False, True]}]),\n", " dict(label=\"AFTER β€” snapped to 8 colors\",\n", " method=\"update\", args=[{\"visible\": [False, True, True, True]}]),\n", " ],\n", " )],\n", " scene=dict(\n", " xaxis=dict(title=\"Red\", range=[0, 1]),\n", " yaxis=dict(title=\"Green\", range=[0, 1]),\n", " zaxis=dict(title=\"Blue\", range=[0, 1]),\n", " aspectmode=\"cube\",\n", " ),\n", " uirevision=\"keep-camera\",\n", " showlegend=False,\n", " width=750, height=650, margin=dict(l=0, r=0, t=80, b=0),\n", ")\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "3ba79f91", "metadata": {}, "source": [ "### πŸ’Ύ Why is this \"compression\"? Let's count the bits\n", "\n", "The repainted photo *looks* almost the same β€” but what did we actually save? Count what needs to be stored:\n", "\n", "**Before (the original):** every pixel stores its exact color = 3 numbers (R, G, B), each needing 8 bits. That's **24 bits per pixel**.\n", "\n", "**After (quantized to k = 8):** every pixel only needs to say *which palette color it uses* β€” a number from 0 to 7. Since 2Β³ = 8, that number fits in just **3 bits per pixel**. On top we store the palette itself (8 colors Γ— 24 bits), which is tiny β€” a few dozen bytes for the whole photo.\n", "\n", "From 24 bits down to 3 bits per pixel β‰ˆ **8Γ— smaller**, and the picture is still perfectly recognizable. This isn't a toy trick: it's genuinely how **GIF images** and old 256-color computer displays worked (256 colors = 2⁸ β†’ 8 bits per pixel instead of 24).\n", "\n", "Run the cell below to see the real numbers for our photo." ] }, { "cell_type": "code", "execution_count": null, "id": "0a2038dc", "metadata": {}, "outputs": [], "source": [ "# Let's count the bits ourselves β€” no hand-waving.\n", "n_pixels = len(pixels)\n", "k = 8\n", "\n", "# BEFORE: every pixel stores 3 channels Γ— 8 bits = 24 bits\n", "original_bits = n_pixels * 3 * 8\n", "\n", "# AFTER: every pixel stores only a palette number 0..7 β†’ 3 bits (2**3 = 8)\n", "label_bits = int(np.ceil(np.log2(k))) # bits needed per pixel label\n", "palette_bits = k * 3 * 8 # the 8 palette colors themselves\n", "compressed_bits = n_pixels * label_bits + palette_bits\n", "\n", "print(f\"pixels in the photo: {n_pixels:,}\")\n", "print()\n", "print(f\"BEFORE β€” {n_pixels:,} px Γ— 24 bits = {original_bits:>9,} bits β‰ˆ {original_bits/8/1024:,.0f} KB\")\n", "print(f\"AFTER β€” {n_pixels:,} px Γ— {label_bits} bits + palette = {compressed_bits:>9,} bits β‰ˆ {compressed_bits/8/1024:,.0f} KB\")\n", "print(f\" (the whole palette costs only {palette_bits} bits β‰ˆ {palette_bits//8} bytes!)\")\n", "print()\n", "print(f\"compression ratio: {original_bits / compressed_bits:.1f}Γ— smaller πŸŽ‰\")\n", "print()\n", "print(\"Try it: how many bits per pixel would k = 16 need? k = 256?\")\n", "print(\"(hint: 2**4 = 16, 2**8 = 256 β€” this is exactly how GIF images work)\")" ] }, { "cell_type": "markdown", "id": "1126523d", "metadata": {}, "source": [ "### πŸ§ͺ Exercise 5 β€” How few colors can you get away with?\n", "\n", "Let's make `k` a dial. The `quantize` helper below repaints the photo with any number of colors. You fill in the one line that displays each version.\n", "\n", "- Inside the loop, call `quantize(k)` and show it with `ax.imshow(...)`.\n", "- Compare k = 3, 8, and 16.\n", "\n", "Then answer in the text cell below: what do you *lose* at k = 3? What barely changes between k = 8 and k = 16?" ] }, { "cell_type": "code", "execution_count": null, "id": "f68d516b", "metadata": {}, "outputs": [], "source": [ "def quantize(k, n_sample=20_000):\n", " \"\"\"Repaint `china` using only k colors. Returns an (H, W, 3) image.\"\"\"\n", " samp = pixels[rng.choice(len(pixels), size=n_sample, replace=False)]\n", " model = KMeans(n_clusters=k, n_init=4, random_state=SEED).fit(samp)\n", " return model.cluster_centers_[model.predict(pixels)].reshape(china.shape)\n", "\n", "\n", "fig, axes = plt.subplots(1, 3, figsize=(12, 4))\n", "for ax, k in zip(axes, [3, 8, 16]):\n", " # ==================== YOUR CODE HERE ====================\n", " ### HINT: quantize(k) returns the repainted image; hand it to ax.imshow(...)\n", " ...\n", " ax.set_title(f\"k = {k} colors\")\n", " ax.axis(\"off\")\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "643a286c", "metadata": {}, "source": [ "**Your turn to describe** *(double-click to edit):*\n", "\n", "- At **k = 3**, the photo loses ...\n", "- Between **k = 8** and **k = 16**, ...\n", "\n", "πŸ’‘ You just built a photo compressor β€” and the engine behind your Studio's 🎨 **Pattern Finder** module. Same k-means, running live on whatever photo you upload." ] }, { "cell_type": "markdown", "id": "fc2e5149", "metadata": {}, "source": [ "## 5 Β· Seeing in 4-D: PCA casts a shadow\n", "\n", "Our penguins actually have **four** measurements, not two. But a screen is flat β€” how do you plot a 4-D animal?\n", "\n", "Think of shadows. A 3-D object casts a 2-D shadow on the wall, and if you rotate it to *just the right angle*, the shadow keeps almost everything that makes the shape recognizable. **PCA** (Principal Component Analysis) finds that best angle automatically: it squashes many dimensions down to a few while throwing away as little information as possible.\n", "\n", "### πŸ§ͺ Exercise 6 β€” Flatten 4-D penguins onto a 2-D page\n", "\n", "- Standardize all **four** measurements, then fit `PCA(n_components=2)` and transform them into `coords`.\n", "- We'll color the picture by the *true* species β€” but only to check the result. **PCA never sees the labels;** it arranges the penguins from the measurements alone.\n", "- Print `explained_variance_ratio_` and its sum: the fraction of the original information the 2-D shadow kept.\n", "\n", "You should keep well over 80% of the variance in just two numbers per penguin." ] }, { "cell_type": "code", "execution_count": null, "id": "907aefa2", "metadata": {}, "outputs": [], "source": [ "from sklearn.preprocessing import StandardScaler\n", "from sklearn.decomposition import PCA\n", "import plotly.graph_objects as go\n", "\n", "feat4 = penguins[[\"bill_length_mm\", \"bill_depth_mm\",\n", " \"flipper_length_mm\", \"body_mass_g\"]].to_numpy(dtype=float)\n", "X4 = StandardScaler().fit_transform(feat4)\n", "\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: pca.fit_transform(X4) returns the 2-D coordinates\n", "...\n", "\n", "# Interactive: hover a dot to see the 4 ORIGINAL measurements hiding behind\n", "# each 2-D point; click a species in the legend to hide/show it.\n", "species_codes, species_labels = pd.factorize(penguins[\"species\"])\n", "fig = go.Figure()\n", "for c, name in enumerate(species_labels):\n", " m = species_codes == c\n", " fig.add_trace(go.Scatter(\n", " x=coords[m, 0], y=coords[m, 1], mode=\"markers\", name=name,\n", " marker=dict(size=6),\n", " customdata=feat4[m],\n", " hovertemplate=(name + \"
bill %{customdata[0]:.1f} mm Β· \"\n", " \"depth %{customdata[1]:.1f} mm
\"\n", " \"flipper %{customdata[2]:.0f} mm Β· \"\n", " \"mass %{customdata[3]:.0f} g\"),\n", " ))\n", "fig.update_layout(\n", " title=(\"4 measurements squashed into 2 β€” colored by true species\"\n", " \"
hover a dot for its original 4 numbers Β· click the legend to hide a species\"),\n", " xaxis=dict(title=\"principal component 1\"),\n", " yaxis=dict(title=\"principal component 2\"),\n", " width=750, height=550,\n", ")\n", "fig.show()\n", "\n", "print(\"information kept per axis:\", np.round(pca.explained_variance_ratio_, 3))\n", "print(\"total information kept: \", round(float(pca.explained_variance_ratio_.sum()), 3))" ] }, { "cell_type": "markdown", "id": "ef6a3b7b", "metadata": {}, "source": [ "## 6 Β· A map of handwriting with t-SNE\n", "\n", "PCA is fast and honest, but it can only *rotate and flatten* β€” straight-line moves. Some data is tangled in ways that need a curvier map. **t-SNE** builds one: it pulls similar points close together and shoves different ones apart, untangling structure PCA can miss. It's the go-to tool for *seeing* high-dimensional data.\n", "\n", "Let's aim it at something with 64 dimensions: tiny 8Γ—8 images of handwritten digits (built right into scikit-learn β€” no download)." ] }, { "cell_type": "code", "execution_count": null, "id": "b8851141", "metadata": {}, "outputs": [], "source": [ "from sklearn.datasets import load_digits\n", "\n", "digits = load_digits()\n", "print(digits.images.shape, \"β†’ 1797 images, each 8Γ—8 pixels (64 numbers)\")\n", "\n", "fig, axes = plt.subplots(2, 8, figsize=(10, 2.8))\n", "for ax, img, label in zip(axes.flat, digits.images, digits.target):\n", " ax.imshow(img, cmap=\"gray_r\")\n", " ax.set_title(str(label))\n", " ax.axis(\"off\")\n", "fig.suptitle(\"a handful of the digits\")\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": null, "id": "64c1ca3c", "metadata": {}, "outputs": [], "source": [ "from sklearn.manifold import TSNE\n", "\n", "# t-SNE is slower than PCA. Under a smoke test we use only the first 400 images.\n", "n_show = 400 if SMOKE else len(digits.data)\n", "X_digits = digits.data[:n_show]\n", "y_digits = digits.target[:n_show]\n", "\n", "print(f\"Running t-SNE on {n_show} images… this takes ~20-40s on CPU. Hang tight β˜•\")\n", "tsne = TSNE(n_components=2, random_state=SEED, perplexity=30)\n", "embedding = tsne.fit_transform(X_digits)\n", "print(\"Done β€” here comes the map.\")" ] }, { "cell_type": "markdown", "id": "dbac6f6a", "metadata": {}, "source": [ "### πŸ§ͺ Exercise 7 β€” Draw the map\n", "\n", "`embedding` holds a 2-D point for each digit image. Color each point by which digit it actually is and see whether t-SNE sorted them.\n", "\n", "- For each digit `d` from 0 to 9: make a mask `m = y_digits == d`, then add a `go.Scatter` trace of `embedding[m, 0]` vs `embedding[m, 1]` with `name=str(d)`.\n", "- One trace per digit gives you a **clickable legend**: click a digit to hide it, **double-click to isolate its island**.\n", "\n", "You should see roughly ten islands β€” one per digit β€” that nobody told t-SNE about." ] }, { "cell_type": "code", "execution_count": null, "id": "7644ba02", "metadata": {}, "outputs": [], "source": [ "import plotly.graph_objects as go\n", "\n", "fig = go.Figure()\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: loop d in range(10); mask m = y_digits == d; add a go.Scatter of embedding[m] with name=str(d)\n", "...\n", "fig.update_layout(\n", " title=(\"1797 handwritten digits, sorted into islands with zero labels\"\n", " \"
hover to identify an island Β· click a legend digit to hide it Β· double-click to isolate it\"),\n", " xaxis=dict(title=\"t-SNE axis 1\"),\n", " yaxis=dict(title=\"t-SNE axis 2\"),\n", " legend_title=\"digit\", width=800, height=620,\n", ")\n", "fig.show()" ] }, { "cell_type": "markdown", "id": "b8cefe76", "metadata": {}, "source": [ "**1797 images, 64 pixels each β€” and the map sorts them into islands, one per digit, without ever being told what a digit is.** The pixels alone carry the structure; t-SNE just makes it visible.\n", "\n", "Soon you'll go one step further and **train a network to actually read these digits.**" ] }, { "cell_type": "markdown", "id": "a39ee91f", "metadata": {}, "source": [ "## 7 Β· Optional: UMAP, t-SNE's faster cousin\n", "\n", "**UMAP** does a similar untangling job, often faster and better at keeping the big-picture layout. It's an extra library, so this cell skips itself politely if UMAP isn't installed β€” no worries either way." ] }, { "cell_type": "code", "execution_count": null, "id": "1b2b54c8", "metadata": {}, "outputs": [], "source": [ "try:\n", " import umap # optional β€” present in the Lab's Docker image, maybe not on Colab\n", " import plotly.graph_objects as go\n", "\n", " reducer = umap.UMAP(n_components=2, random_state=SEED)\n", " umap_emb = reducer.fit_transform(X_digits)\n", "\n", " fig = go.Figure()\n", " for d in range(10):\n", " m = y_digits == d\n", " fig.add_trace(go.Scatter(x=umap_emb[m, 0], y=umap_emb[m, 1],\n", " mode=\"markers\", name=str(d), marker=dict(size=5),\n", " hovertemplate=f\"digit {d}\"))\n", " fig.update_layout(\n", " title=\"the same digits, mapped by UMAP β€” same legend tricks apply\",\n", " xaxis=dict(title=\"UMAP axis 1\"),\n", " yaxis=dict(title=\"UMAP axis 2\"),\n", " legend_title=\"digit\", width=800, height=620,\n", " )\n", " fig.show()\n", "except ImportError:\n", " print(\"UMAP isn't installed here β€” that's totally fine, skip this one. πŸ™‚\")" ] }, { "cell_type": "markdown", "id": "3760093b", "metadata": {}, "source": [ "## πŸ”“ Unlock your Studio module\n", "\n", "You've earned it. Save your settings and the 🎨 **Pattern Finder** switches on in your AI Studio. Put your name on your work and pick the number of colors it should reach for by default." ] }, { "cell_type": "code", "execution_count": null, "id": "8e61d75b", "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "# ✏️ Make it yours:\n", "MADE_BY = \"your name here\" # ← type your name between the quotes\n", "DEFAULT_K = 8 # ← default number of color clusters (2–12)\n", "\n", "config_path = MODELS_DIR / \"pattern_finder.json\"\n", "with open(config_path, \"w\", encoding=\"utf-8\") as f:\n", " json.dump({\"default_k\": DEFAULT_K, \"made_by\": MADE_BY}, f, indent=2)\n", "\n", "# make sure the Studio will find it\n", "assert config_path.exists(), \"pattern_finder.json did not save!\"\n", "print(\"πŸ”“ UNLOCKED: Pattern Finder!\")\n", "print(f\" saved {config_path.name} (default_k={DEFAULT_K}, made_by={MADE_BY})\")\n", "print(\" Run your Studio to try it live on any photo:\")\n", "print(\" python ai_studio/app.py (from the repo root)\")" ] }, { "cell_type": "markdown", "id": "6b990a2a", "metadata": {}, "source": [ "## 🏁 Checkpoint\n", "\n", "Look what you can do now β€” with **no answer key at all**:\n", "\n", "- **k-means** to discover groups (and you watched it *rediscover the penguin species*)\n", "- the **elbow method** to guess how many groups exist\n", "- **color quantization** to compress a photo β€” now live in your 🎨 **Pattern Finder**\n", "- **PCA** and **t-SNE** to turn 4-D and 64-D data into pictures you can actually see\n", "\n", "**So far you've borrowed brains (Day 1) and learned the classic tricks (Days 2-3).** Tomorrow the Chief Scientist raises the stakes: **build and train a neural network from nothing but NumPy** β€” no libraries to hide behind. By the end, you'll never see a neural net as a black box again.\n", "\n", "**Next: Day 4 β€” Inside the Brain 🧠**" ] }, { "cell_type": "code", "execution_count": null, "id": "2615965c-28bf-45ff-bd30-95582d1f8c9a", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "8b827df8-0012-450a-9aec-44961ba3e4f6", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "e2ea02d6", "metadata": {}, "source": [ "---\n", "\n", "
\n", " \n", " \"Dr.\n", " \n", "
\n", "\n", "
The Lab Β· AI Studio β€” Intro to AI Β· Β© 2026 Dr. Abdelghafour HALIMI Β· ahalimi.com Β· All rights reserved.
" ] } ], "metadata": { "authors": [ { "name": "Dr. Abdelghafour HALIMI" } ], "copyright": "Β© 2026 Dr. Abdelghafour HALIMI Β· ahalimi.com Β· All rights reserved", "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }