{ "cells": [ { "cell_type": "markdown", "id": "70e9d507", "metadata": {}, "source": [ "# Day 8 โ€” Machines That Read ๐Ÿ’ฌ\n", "\n", "> **Mission briefing:** Your Studio can *see*. Today it learns to *read* โ€” and to *write*. We crack open the machinery behind ChatGPT-class systems: **transformers**, powered by one absurdly simple idea repeated a trillion times โ€” *predict the next word*.\n", "\n", "**Previously in the Lab:** On Day 7 you fine-tuned a giant with transfer learning and unlocked ๐Ÿชจ๐Ÿ“„โœ‚๏ธ **RPS Arena**.\n", "\n", "**Today you will:**\n", "- See how text becomes **tokens** and then **embeddings** (words as arrows in space)\n", "- Watch a language model **predict the next word**, step by step, and learn what *temperature* does\n", "- Tour four **pipelines**: mood, fill-in-the-blank, topic-sorting, and story writing\n", "- Peek inside **attention** โ€” every word choosing which other words to look at\n", "- Learn why models produce *plausible* text, not *true* text\n", "\n", "**Today you unlock:** ๐Ÿ”“ **Language Lab** โ€” four language superpowers in your Studio, running on the transformer models you explore today.\n", "\n", "
A precise reading

GPT-2 predicts the next token, which may be a word fragment. Encoder models such as DistilBERT can use both sides of a masked token. Tokenization can make letter counting harder but does not make it impossible. An attention heatmap shows model weights, not proof of grammatical reference or a complete explanation of a prediction. A lower temperature changes sampling, not factual reliability.

\n" ] }, { "cell_type": "code", "execution_count": null, "id": "6e576b63", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:56:56.457263Z", "iopub.status.busy": "2026-08-17T12:56:56.457111Z", "iopub.status.idle": "2026-08-17T12:56:56.560625Z", "shell.execute_reply": "2026-08-17T12:56:56.559470Z" } }, "outputs": [], "source": [ "# โ•”โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•—\n", "# โ•‘ SETUP โ€” run me first! (identical in every Lab notebook) โ•‘\n", "# โ•šโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•\n", "import os, sys, pathlib\n", "\n", "IN_COLAB = \"google.colab\" in sys.modules\n", "SMOKE = os.environ.get(\"SMOKE_TEST\", \"0\") == \"1\" # tiny fast run for automated tests\n", "\n", "REPO_URL = \"https://github.com/A-Halimi/ai-studio-internship.git\" # instructor: set once\n", "\n", "if IN_COLAB:\n", " if not pathlib.Path(\"/content/ai-studio-internship\").exists():\n", " !git clone -q {REPO_URL} /content/ai-studio-internship\n", " %cd /content/ai-studio-internship/notebooks/day08\n", " %pip -q install transformers==5.13.0 gradio==6.19.0\n", "\n", "HERE = pathlib.Path.cwd()\n", "REPO = HERE.parents[1] # .../notebooks/dayNN -> repo root\n", "DATA_DIR = pathlib.Path(os.environ.get(\"COURSE_DATA_DIR\", REPO / \"data\"))\n", "SAMPLES_DIR = REPO / \"data_samples\" # small datasets shipped with the repo\n", "MODELS_DIR = REPO / \"ai_studio\" / \"models\" # where your Studio modules unlock\n", "for p in (REPO / \"data\", MODELS_DIR):\n", " p.mkdir(parents=True, exist_ok=True)\n", "\n", "import random\n", "import numpy as np\n", "SEED = 42\n", "random.seed(SEED); np.random.seed(SEED)\n", "print(f\"โœ… Setup done | Colab: {IN_COLAB} | Smoke test: {SMOKE}\")\n", "print(f\" data: {DATA_DIR}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e4b07eea", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:56:56.562646Z", "iopub.status.busy": "2026-08-17T12:56:56.562460Z", "iopub.status.idle": "2026-08-17T12:56:57.793484Z", "shell.execute_reply": "2026-08-17T12:56:57.792296Z" } }, "outputs": [], "source": [ "import torch\n", "torch.manual_seed(SEED)\n", "DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", "if DEVICE.type == \"cuda\":\n", " print(f\"โšก Running on GPU: {torch.cuda.get_device_name(0)}\")\n", "else:\n", " print(\"๐Ÿข No GPU found โ€” running on CPU. Everything today still works, just a little slower.\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e0ff3490", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:56:57.795226Z", "iopub.status.busy": "2026-08-17T12:56:57.795015Z", "iopub.status.idle": "2026-08-17T12:56:57.798442Z", "shell.execute_reply": "2026-08-17T12:56:57.797371Z" } }, "outputs": [], "source": [ "# โ”€โ”€ CONFIG โ€” the Lab's control panel โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€\n", "GEN_TOKENS = 20 if SMOKE else 60 # how many new words the story writer adds\n", "\n", "# First run downloads a few small models from Hugging Face (~1.5 GB total:\n", "# gpt2 + three distilbert models). They're pre-baked in the course Docker image;\n", "# on a laptop or Colab they download once and then cache. Grab a coffee. โ˜•" ] }, { "cell_type": "markdown", "id": "cd37efad", "metadata": {}, "source": [ "## 1. The idea that changed everything โšก\n", "\n", "In **2017** a paper called *\"Attention Is All You Need\"* introduced the **transformer**. Within a few years it powered GPT, translation, search, and the chatbots you use today. The wild part: under all the hype sits one tiny training goal โ€” **look at some text, predict the next token, and repeat.** Do that on a big chunk of the internet and the model soaks up grammar, facts, styles, and a surprising amount of reasoning.\n", "\n", "Today we don't just *use* these models โ€” we **look inside** one. We'll use **GPT-2**, the small, open ancestor of ChatGPT (124 million parameters โ€” tiny by today's standards, perfect for learning)." ] }, { "cell_type": "markdown", "id": "b535d38c", "metadata": {}, "source": [ "## 2. Step 1 โ€” text becomes tokens ๐Ÿ”ค\n", "\n", "GPT-2 receives token IDs rather than a separate input for every letter. First it chops text into **tokens** โ€” common chunks that are often whole words, sometimes word-pieces. GPT-2 knows exactly **50,257** of them. Every token maps to a number, and that's all the model ever sees: a list of numbers.\n", "\n", "Run the demo โ€” notice the little `ฤ ` marks a space in front of a token." ] }, { "cell_type": "code", "execution_count": null, "id": "75ee56be", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:56:57.799965Z", "iopub.status.busy": "2026-08-17T12:56:57.799819Z", "iopub.status.idle": "2026-08-17T12:57:05.654767Z", "shell.execute_reply": "2026-08-17T12:57:05.653563Z" } }, "outputs": [], "source": [ "from transformers import AutoTokenizer\n", "\n", "tok = AutoTokenizer.from_pretrained(\"gpt2\")\n", "pieces = tok.tokenize(\"The penguins in the lab love matrix multiplication\")\n", "print(pieces)\n", "print(f\"\\n{len(pieces)} tokens for that sentence.\")" ] }, { "cell_type": "markdown", "id": "f75e2cf2", "metadata": {}, "source": [ "### ๐Ÿงช Exercise 1 โ€” Find a word that shatters\n", "\n", "Some words are a single token; rarer ones get split into pieces. Below, several words are tokenized for you. **Collect the ones that split into 3 or more pieces.**\n", "\n", "- `tok.tokenize(word)` returns the list of pieces\n", "- keep the words where `len(pieces) >= 3`\n", "\n", "You should find that long or unusual words (like a full name) shatter the most." ] }, { "cell_type": "code", "execution_count": null, "id": "4bd8b1b4", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:05.656706Z", "iopub.status.busy": "2026-08-17T12:57:05.656376Z", "iopub.status.idle": "2026-08-17T12:57:05.662103Z", "shell.execute_reply": "2026-08-17T12:57:05.660967Z" } }, "outputs": [], "source": [ "candidates = [\"supercomputer\", \"Abdelghafour\", \"strawberry\", \"penguin\", \"robot\", \"cat\"]\n", "splits = {w: tok.tokenize(w) for w in candidates} # given: tokenize each word\n", "for w, t in splits.items():\n", " print(f\"{w!r:16} -> {len(t)} piece(s) {t}\")\n", "\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: [w for w, t in splits.items() if len(t) >= 3]\n", "...\n", "print(\"\\n3+ pieces:\", three_plus)\n", "assert any(len(t) >= 3 for t in splits.values()), \"Expected at least one word to split into 3+ pieces.\"" ] }, { "cell_type": "markdown", "id": "b0c3ab62", "metadata": {}, "source": [ "### ๐Ÿงช Exercise 2 โ€” The strawberry problem ๐Ÿ“\n", "\n", "Ever seen a chatbot miscount the letters in a word? Here's why. Tokenize **`\"strawberry\"`** and count the pieces. The model never sees ten individual letters โ€” it sees a couple of chunks. Counting the *r*'s means counting something it can't even look at.\n", "\n", "Set `n_pieces` to how many tokens `\"strawberry\"` becomes." ] }, { "cell_type": "code", "execution_count": null, "id": "9ff46c97", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:05.664328Z", "iopub.status.busy": "2026-08-17T12:57:05.664115Z", "iopub.status.idle": "2026-08-17T12:57:05.668185Z", "shell.execute_reply": "2026-08-17T12:57:05.667284Z" } }, "outputs": [], "source": [ "pieces = tok.tokenize(\"strawberry\")\n", "print(\"strawberry ->\", pieces)\n", "\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: n_pieces = len(pieces)\n", "...\n", "print(f\"The model sees {n_pieces} chunk(s), not 10 letters โ€” so 'how many r's?' is genuinely hard for it.\")" ] }, { "cell_type": "code", "execution_count": null, "id": "7dac047c", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:05.669867Z", "iopub.status.busy": "2026-08-17T12:57:05.669732Z", "iopub.status.idle": "2026-08-17T12:57:05.673417Z", "shell.execute_reply": "2026-08-17T12:57:05.672468Z" } }, "outputs": [], "source": [ "# Every token has an ID; encode -> IDs, decode -> back to text (a perfect round-trip).\n", "print(\"Vocabulary size:\", tok.vocab_size) # 50257\n", "ids = tok.encode(\"Matrix multiplication is everywhere.\")\n", "print(\"IDs: \", ids)\n", "print(\"Decoded:\", tok.decode(ids))" ] }, { "cell_type": "markdown", "id": "db52caeb", "metadata": {}, "source": [ "## 3. Step 2 โ€” tokens become arrows ๐Ÿน\n", "\n", "A number like `token 1820` means nothing on its own. So the model gives every token an **embedding**: a list of **768 numbers** โ€” think of it as an arrow pointing to a spot in a 768-dimensional space. The magic is that **meaning becomes geometry**: words used in similar ways end up near each other.\n", "\n", "We can't picture 768 dimensions, so we **squish** them down to 2 with PCA and plot. Watch related words cluster." ] }, { "cell_type": "code", "execution_count": null, "id": "a9b326ac", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:05.675412Z", "iopub.status.busy": "2026-08-17T12:57:05.675279Z", "iopub.status.idle": "2026-08-17T12:57:08.482496Z", "shell.execute_reply": "2026-08-17T12:57:08.481220Z" } }, "outputs": [], "source": [ "from transformers import AutoModelForCausalLM\n", "from sklearn.decomposition import PCA\n", "\n", "gpt2 = AutoModelForCausalLM.from_pretrained(\"gpt2\").to(DEVICE).eval()\n", "emb = gpt2.transformer.wte.weight # (50257, 768): one 768-D arrow per token\n", "\n", "# GPT-2 tokens usually carry a leading space, so we write \" king\" not \"king\".\n", "words = [\" king\", \" queen\", \" man\", \" woman\", \" boy\", \" girl\",\n", " \" cat\", \" dog\", \" pizza\", \" bread\", \" fire\", \" water\",\n", " \" science\", \" music\", \" happy\", \" sad\", \" summer\", \" winter\"]\n", "words = [w for w in words if len(tok.encode(w)) == 1] # keep true single-token words\n", "vecs = torch.stack([emb[tok.encode(w)[0]] for w in words]).detach().cpu().numpy()\n", "\n", "xy = PCA(n_components=2, random_state=SEED).fit_transform(vecs)\n", "import matplotlib.pyplot as plt\n", "plt.figure(figsize=(8, 6))\n", "plt.scatter(xy[:, 0], xy[:, 1])\n", "for (x, y), w in zip(xy, words):\n", " plt.annotate(w.strip(), (x, y), fontsize=11)\n", "plt.title(\"GPT-2 word embeddings (768-D) squished to 2-D\")\n", "plt.xlabel(\"PCA dimension 1\"); plt.ylabel(\"PCA dimension 2\")\n", "plt.tight_layout(); plt.show()" ] }, { "cell_type": "markdown", "id": "43d51b80", "metadata": {}, "source": [ "### ๐Ÿงช Exercise 3 โ€” Put your own words on the map\n", "\n", "Add **three of your own words** to `my_words`. Two rules so nothing breaks:\n", "\n", "- start each with a **space** (e.g. `\" ocean\"`), and\n", "- pick words likely to be a **single token** (short, common words are safest โ€” the code quietly drops any that aren't).\n", "\n", "Re-run and see where they land. Do the neighbors make sense? **Meaning = geometry:** each word is really 768 numbers, and closeness on the plot hints at closeness in meaning." ] }, { "cell_type": "code", "execution_count": null, "id": "91e18c52", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:08.484411Z", "iopub.status.busy": "2026-08-17T12:57:08.484097Z", "iopub.status.idle": "2026-08-17T12:57:08.564406Z", "shell.execute_reply": "2026-08-17T12:57:08.563134Z" } }, "outputs": [], "source": [ "# ==================== YOUR CODE HERE ====================\n", "### HINT: three single-token words, each starting with a space, e.g. \" ocean\"\n", "...\n", "\n", "all_words = words + [w for w in my_words if len(tok.encode(w)) == 1]\n", "vecs2 = torch.stack([emb[tok.encode(w)[0]] for w in all_words]).detach().cpu().numpy()\n", "xy2 = PCA(n_components=2, random_state=SEED).fit_transform(vecs2)\n", "\n", "new_set = {w for w in my_words if len(tok.encode(w)) == 1}\n", "plt.figure(figsize=(8, 6))\n", "for (x, y), w in zip(xy2, all_words):\n", " is_new = w in new_set\n", " plt.scatter([x], [y], color=\"crimson\" if is_new else \"steelblue\")\n", " plt.annotate(w.strip(), (x, y), fontsize=11,\n", " color=\"crimson\" if is_new else \"black\")\n", "plt.title(\"Your words (red) among the neighbors\")\n", "plt.xlabel(\"PCA dimension 1\"); plt.ylabel(\"PCA dimension 2\")\n", "plt.tight_layout(); plt.show()" ] }, { "cell_type": "markdown", "id": "71ddcf01", "metadata": {}, "source": [ "## 4. โ˜… The whole secret โ€” predict the next word ๐Ÿ”ฎ\n", "\n", "Here it is, the entire engine. Given some text, GPT-2 outputs a **probability for every one of its 50,257 tokens** โ€” how likely each is to come next. Pick one, stick it on the end, and ask again. That loop is *all* that \"generating text\" means.\n", "\n", "Let's watch it. The helper below returns the model's **top-5 guesses** for what comes after a prompt.\n", "\n", "
๐Ÿ”ข Math Corner โ€” how much computation is one word? (optional)\n", "\n", "Each next-word guess flows through GPT-2's **12 transformer blocks**, each with an **attention** step and a small **MLP** โ€” hundreds of matrix multiplications across **124 million** parameters, *per token*. Generating a sentence does this dozens of times. Making those multiplications scream on a GPU is the whole field of high-performance computing.\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": null, "id": "3e0842ad", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:08.566206Z", "iopub.status.busy": "2026-08-17T12:57:08.566050Z", "iopub.status.idle": "2026-08-17T12:57:09.301251Z", "shell.execute_reply": "2026-08-17T12:57:09.300000Z" } }, "outputs": [], "source": [ "import torch.nn.functional as F\n", "\n", "@torch.no_grad()\n", "def next_token_probs(prompt, k=5):\n", " \"\"\"Return the model's top-k (token, probability) guesses for what comes next.\"\"\"\n", " ids = tok.encode(prompt, return_tensors=\"pt\").to(DEVICE)\n", " logits = gpt2(ids).logits[0, -1] # scores for the token AFTER the prompt\n", " probs = F.softmax(logits, dim=0) # turn scores into probabilities\n", " top = torch.topk(probs, k)\n", " return [(tok.decode([int(i)]), float(p)) for p, i in zip(top.values, top.indices)]\n", "\n", "print(\"Prompt: 'The best thing about science is'\\n\")\n", "for tk, p in next_token_probs(\"The best thing about science is\"):\n", " print(f\"{p:6.1%} {tk!r}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "7ac8616f", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:09.303047Z", "iopub.status.busy": "2026-08-17T12:57:09.302896Z", "iopub.status.idle": "2026-08-17T12:57:09.360696Z", "shell.execute_reply": "2026-08-17T12:57:09.359493Z" } }, "outputs": [], "source": [ "# Greedily glue on the single most likely token, five times. Watch the sentence assemble.\n", "text = \"The best thing about science is\"\n", "for step in range(5):\n", " top = next_token_probs(text, k=5)\n", " nxt = top[0][0] # greedy = always pick #1\n", " options = \", \".join(f\"{t.strip()!r}:{p:.0%}\" for t, p in top)\n", " print(f\"step {step + 1}: ...{text[-40:]!r}\")\n", " print(f\" choices -> {options}\\n\")\n", " text += nxt\n", "print(\"Final sentence:\", repr(text))" ] }, { "cell_type": "markdown", "id": "943e80de", "metadata": {}, "source": [ "### ๐Ÿงช Exercise 4 โ€” Predict from your own prompt\n", "\n", "Set `my_prompt` to any sentence-start you like and look at the model's top-5 next-word guesses. Try something factual (`\"The capital of France is\"`) and something silly โ€” the shape of the probabilities changes with how \"sure\" the model is." ] }, { "cell_type": "code", "execution_count": null, "id": "559f5fc1", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:09.362505Z", "iopub.status.busy": "2026-08-17T12:57:09.362346Z", "iopub.status.idle": "2026-08-17T12:57:09.375518Z", "shell.execute_reply": "2026-08-17T12:57:09.374516Z" } }, "outputs": [], "source": [ "# ==================== YOUR CODE HERE ====================\n", "### HINT: any string, e.g. \"In the year 3000, computers will\"\n", "...\n", "\n", "for tk, p in next_token_probs(my_prompt):\n", " print(f\"{p:6.1%} {tk!r}\")" ] }, { "cell_type": "markdown", "id": "a1409cd3", "metadata": {}, "source": [ "## 5. Four superpowers โ€” the pipeline tour ๐Ÿงฐ\n", "\n", "Writing model code by hand is a lot. Hugging Face **pipelines** wrap a model + tokenizer into a single call. We'll tour four, each a different transformer trained for a different job. First up: **text-classification** โ€” reading the *mood* (sentiment) of a sentence." ] }, { "cell_type": "code", "execution_count": null, "id": "a7a1b74c", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:09.377251Z", "iopub.status.busy": "2026-08-17T12:57:09.377110Z", "iopub.status.idle": "2026-08-17T12:57:11.784015Z", "shell.execute_reply": "2026-08-17T12:57:11.782822Z" } }, "outputs": [], "source": [ "from transformers import pipeline\n", "\n", "# device=DEVICE puts it on the GPU when there is one; plain CPU works fine too.\n", "clf = pipeline(\"text-classification\",\n", " model=\"distilbert-base-uncased-finetuned-sst-2-english\", device=DEVICE)\n", "\n", "for s in [\"This course is the best thing ever!\",\n", " \"I am so bored and tired today.\",\n", " \"The GPU cluster caught fire again.\"]:\n", " r = clf(s)[0]\n", " print(f\"{r['label']:8} {r['score']:.0%} <- {s}\")" ] }, { "cell_type": "markdown", "id": "42c699d6", "metadata": {}, "source": [ "### ๐Ÿงช Exercise 5 โ€” Fool the mood detector ๐Ÿ˜\n", "\n", "These models read surface words, not deep meaning. Write a **sarcastic** sentence โ€” cheerful words, sour meaning โ€” that gets labeled **POSITIVE** even though a human hears the opposite. (Something like *\"Oh wonderful, another bug.\"*)" ] }, { "cell_type": "code", "execution_count": null, "id": "524b93d1", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:11.785741Z", "iopub.status.busy": "2026-08-17T12:57:11.785572Z", "iopub.status.idle": "2026-08-17T12:57:11.794181Z", "shell.execute_reply": "2026-08-17T12:57:11.793234Z" } }, "outputs": [], "source": [ "# ==================== YOUR CODE HERE ====================\n", "### HINT: sarcasm works well โ€” happy words, unhappy meaning\n", "...\n", "\n", "r = clf(tricky)[0]\n", "print(f\"{r['label']} ({r['score']:.0%}) <- {tricky}\")\n", "print(\"Did it read the sarcasm, or just the happy words?\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e0faeac3", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:11.795546Z", "iopub.status.busy": "2026-08-17T12:57:11.795412Z", "iopub.status.idle": "2026-08-17T12:57:14.046917Z", "shell.execute_reply": "2026-08-17T12:57:14.045307Z" } }, "outputs": [], "source": [ "# Superpower 2 โ€” fill-mask: guess a hidden word. Write the blank as ___ and we swap in\n", "# the model's special mask token (that replace trick comes straight from the Studio module).\n", "fm = pipeline(\"fill-mask\", model=\"distilbert-base-uncased\", device=DEVICE)\n", "\n", "sentence = \"Students in the lab love to ___ every day.\"\n", "masked = sentence.replace(\"___\", fm.tokenizer.mask_token)\n", "print(sentence, \"\\n\")\n", "for r in fm(masked)[:5]:\n", " print(f\"{r['score']:.1%} {r['token_str'].strip()!r}\")" ] }, { "cell_type": "code", "execution_count": null, "id": "e40d6156", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:14.049935Z", "iopub.status.busy": "2026-08-17T12:57:14.049779Z", "iopub.status.idle": "2026-08-17T12:57:16.005621Z", "shell.execute_reply": "2026-08-17T12:57:16.004456Z" } }, "outputs": [], "source": [ "# Superpower 3 โ€” zero-shot: sort text into ANY labels you invent, with no extra training.\n", "zs = pipeline(\"zero-shot-classification\",\n", " model=\"typeform/distilbert-base-uncased-mnli\", device=DEVICE)\n", "\n", "res = zs(\"The team trained a neural network on the GPU cluster overnight.\",\n", " candidate_labels=[\"science\", \"sports\", \"cooking\", \"gossip\"])\n", "for lab, score in zip(res[\"labels\"], res[\"scores\"]):\n", " print(f\"{score:5.0%} {lab}\")" ] }, { "cell_type": "markdown", "id": "47053456", "metadata": {}, "source": [ "### ๐Ÿงช Exercise 6 โ€” Two temperatures ๐ŸŒก๏ธ\n", "\n", "Superpower 4 โ€” **text-generation**: the next-word loop from section 4, automated. The knob that controls it is **temperature**: how adventurously the model picks from its list of likely next words.\n", "\n", "- **Low** (0.2) โ†’ almost always the top pick: safe, repetitive, sometimes boring\n", "- **High** (1.2) โ†’ happily grabs unlikely words: creative, surprising, sometimes nonsense\n", "\n", "Generate the same story starter **twice** โ€” once at `temperature=0.2`, once at `temperature=1.2` โ€” and read the difference." ] }, { "cell_type": "code", "execution_count": null, "id": "8c52727f", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:16.007501Z", "iopub.status.busy": "2026-08-17T12:57:16.007355Z", "iopub.status.idle": "2026-08-17T12:57:19.683278Z", "shell.execute_reply": "2026-08-17T12:57:19.681979Z" } }, "outputs": [], "source": [ "gen = pipeline(\"text-generation\", model=\"gpt2\", device=DEVICE)\n", "prompt = \"Late one night in the lab, a small robot woke up and said\"\n", "torch.manual_seed(SEED) # seed so the demo is the same for everyone\n", "\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: call gen(...) twice โ€” temperature=0.2 for `cool`, temperature=1.2 for `wild`\n", "...\n", "\n", "print(\"๐ŸงŠ temperature 0.2 (plays it safe):\\n\", cool, \"\\n\")\n", "print(\"๐Ÿ”ฅ temperature 1.2 (takes big risks):\\n\", wild)" ] }, { "cell_type": "markdown", "id": "0fc5b710", "metadata": {}, "source": [ "## 6. Peeking inside โ€” attention ๐Ÿ‘€\n", "\n", "How does a model know that \"it\" refers to the penguin and not the fish? **Attention.** At every layer, each word gets to **look at every other word** and decide how much to borrow from each. That table of \"who looks at whom\" is literally a grid of numbers โ€” and computing it is (you guessed it) **matrix multiplication**, the same operation HPC engineers make fly on the GPU.\n", "\n", "Let's visualize it. We run a small model (**DistilBERT**) with `output_attentions=True` and draw one attention head as a heatmap: bright square at row *it*, column *penguin* means \"*it* is paying attention to *penguin*.\"" ] }, { "cell_type": "code", "execution_count": null, "id": "6d5dbe3a", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:19.685359Z", "iopub.status.busy": "2026-08-17T12:57:19.685139Z", "iopub.status.idle": "2026-08-17T12:57:21.160239Z", "shell.execute_reply": "2026-08-17T12:57:21.159048Z" } }, "outputs": [], "source": [ "from transformers import AutoModel\n", "\n", "dtok = AutoTokenizer.from_pretrained(\"distilbert-base-uncased\")\n", "dmodel = AutoModel.from_pretrained(\"distilbert-base-uncased\",\n", " output_attentions=True).to(DEVICE).eval()\n", "\n", "def show_attention(sentence, layer=4, head=0, title=None):\n", " enc = dtok(sentence, return_tensors=\"pt\").to(DEVICE)\n", " with torch.no_grad():\n", " out = dmodel(**enc)\n", " grid = out.attentions[layer][0, head].cpu().numpy() # (tokens, tokens)\n", " toks = dtok.convert_ids_to_tokens(enc[\"input_ids\"][0])\n", " plt.figure(figsize=(6.5, 5.5))\n", " plt.imshow(grid, cmap=\"viridis\")\n", " plt.xticks(range(len(toks)), toks, rotation=90)\n", " plt.yticks(range(len(toks)), toks)\n", " plt.xlabel(\"attending TO\"); plt.ylabel(\"word doing the looking (FROM)\")\n", " plt.title(title or f\"Attention โ€” layer {layer + 1}, head {head + 1}\")\n", " plt.colorbar(label=\"attention weight\"); plt.tight_layout(); plt.show()\n", "\n", "show_attention(\"The penguin ate the fish because it was hungry\",\n", " title=\"'it' should point at the PENGUIN (it was hungry)\")" ] }, { "cell_type": "markdown", "id": "1227e89b", "metadata": {}, "source": [ "### ๐Ÿงช Exercise 7 โ€” Follow the pronoun\n", "\n", "Change the ending so **\"it\" now means the fish** instead of the penguin โ€” for example, *\"...because it was **delicious**.\"* Re-draw the heatmap and compare: does the bright spot in the *it* row shift toward *fish*?\n", "\n", "(Attention is subtle and no single head is perfect โ€” but you'll often see the pattern move. Set `sentence2` below.)" ] }, { "cell_type": "code", "execution_count": null, "id": "54031c98", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:21.162009Z", "iopub.status.busy": "2026-08-17T12:57:21.161846Z", "iopub.status.idle": "2026-08-17T12:57:21.222426Z", "shell.execute_reply": "2026-08-17T12:57:21.221436Z" } }, "outputs": [], "source": [ "# ==================== YOUR CODE HERE ====================\n", "### HINT: keep the start, change the last word so \"it\" = the fish (e.g. \"delicious\")\n", "...\n", "\n", "show_attention(sentence2, title=\"Now where does 'it' look?\")" ] }, { "cell_type": "markdown", "id": "f810153b", "metadata": {}, "source": [ "## 7. Reality check โš ๏ธ\n", "\n", "One minute of honesty before you unlock the module. A language model predicts **plausible** text โ€” words that *sound* right following the prompt. That is **not** the same as **true** text. Watch GPT-2 answer something it knows, then something it can't possibly know โ€” and say both with the exact same confidence." ] }, { "cell_type": "code", "execution_count": null, "id": "277521c0", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:21.223927Z", "iopub.status.busy": "2026-08-17T12:57:21.223787Z", "iopub.status.idle": "2026-08-17T12:57:22.256820Z", "shell.execute_reply": "2026-08-17T12:57:22.255719Z" } }, "outputs": [], "source": [ "for prompt in [\"The largest planet in our Solar System is\",\n", " \"In the year 3000, computers will be able to\"]:\n", " out = gen(prompt, max_new_tokens=GEN_TOKENS, do_sample=False,\n", " pad_token_id=50256)[0][\"generated_text\"]\n", " print(out)\n", " print(\"-\" * 65)" ] }, { "cell_type": "markdown", "id": "8c30dd51", "metadata": {}, "source": [ "The first answer is probably fine (that fact appeared all over GPT-2's training text). The second is **invented** โ€” GPT-2 never saw 2026, so it confidently makes something up. This is called a **hallucination**, and it's the single biggest gotcha when using these tools: *always verify facts.*\n", "\n", "So how do modern systems (ChatGPT, Claude) get so much more reliable and helpful? They add an extra ingredient on top of next-word prediction: **training from human feedback** โ€” rewarding good answers, penalizing bad ones. **Rewards.** Which is *exactly* tomorrow's topic. ๐Ÿ•" ] }, { "cell_type": "markdown", "id": "c5d2d8d3", "metadata": {}, "source": [ "## ๐Ÿ”“ Unlock your Studio module\n", "\n", "The **Language Lab** module bundles all four superpowers into your Studio. The transformer models load themselves from the cache โ€” all you ship is a small **settings token** that personalizes it. Fill in your name and (optionally) rewrite the story starters, then run the cell." ] }, { "cell_type": "code", "execution_count": null, "id": "5d507974", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:22.258611Z", "iopub.status.busy": "2026-08-17T12:57:22.258468Z", "iopub.status.idle": "2026-08-17T12:57:22.270193Z", "shell.execute_reply": "2026-08-17T12:57:22.269117Z" } }, "outputs": [], "source": [ "import json\n", "\n", "MADE_BY = \"an AI explorer at the Lab\" # โ† put YOUR name here\n", "DEFAULT_TEMPERATURE = 0.9 # the Story Writer's starting creativity\n", "STORY_STARTERS = [ # write at least 3 โ€” make them yours!\n", " \"Late one night in the lab, a small robot woke up and said\",\n", " \"The last penguin on the iceberg looked at the satellite and\",\n", " \"In the year 3000, the fastest supercomputer on Earth whispered\",\n", "]\n", "\n", "cfg = {\"made_by\": MADE_BY,\n", " \"default_temperature\": DEFAULT_TEMPERATURE,\n", " \"story_starters\": STORY_STARTERS}\n", "(MODELS_DIR / \"language_lab.json\").write_text(json.dumps(cfg, indent=2), encoding=\"utf-8\")\n", "\n", "REQUIRED_FILES = [\"language_lab.json\"]\n", "missing = [f for f in REQUIRED_FILES if not (MODELS_DIR / f).exists()]\n", "assert not missing, f\"Something didn't save: {missing}\"\n", "assert len(STORY_STARTERS) >= 3, \"Write at least 3 story starters before unlocking.\"\n", "print(\"๐Ÿ”“ UNLOCKED: Language Lab! Run your Studio and try all four tabs:\")\n", "print(\" python ai_studio/app.py (from the repo root) -> ๐Ÿ’ฌ Language Lab\")" ] }, { "cell_type": "markdown", "id": "262fb60d", "metadata": {}, "source": [ "## ๐Ÿ Checkpoint\n", "\n", "Today you looked *inside* a language model. You watched text become **tokens** and then **embeddings** (meaning as geometry), saw the one trick behind it all โ€” **predict the next token** โ€” and learned that **temperature** tunes how wild the guessing gets. You toured four pipelines, peeked at **attention** (every word choosing what to look at โ€” matrix multiplication again), and learned the crucial caveat: models produce *plausible*, not *true*, text.\n", "\n", "**Studio unlocked:** ๐Ÿ’ฌ Language Lab โ€” mood, fill-in-the-blank, topic-sorting, and a story writer, all in your app.\n", "\n", "Seeing โœ“. Reading โœ“. But the Studio still can't **act**. Tomorrow: **machines that learn by trial, error, and reward** โ€” like training a puppy with treats. Welcome to reinforcement learning. ๐Ÿ•" ] }, { "cell_type": "markdown", "id": "97f8dba6", "metadata": {}, "source": [ "---\n", "\n", "
\n", " \n", " \"Dr.\n", " \n", "
\n", "\n", "
The Lab ยท AI Studio โ€” Intro to AI ยท ยฉ 2026 Dr. Abdelghafour HALIMI ยท ahalimi.com ยท All rights reserved.
" ] } ], "metadata": { "authors": [ { "name": "Dr. Abdelghafour HALIMI" } ], "copyright": "ยฉ 2026 Dr. Abdelghafour HALIMI ยท ahalimi.com ยท All rights reserved", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }