The classical dense matmul counts about 2nยณ floating-point operations, but measured timing curves need not have slope 3 at every size. Compare the same size and precision, warm up, synchronize the GPU, and state whether transfer time is included. Speedups are measurements, not guarantees. Matrix products power many neural layers; tabular Q-learning is a scalar update, and other operations still matter.
The classical triple loop uses approximately nยณ multiply-adds, or 2nยณ floating-point operations. Doubling n multiplies this work by about 8; tripling it by 27. This is O(nยณ) work.
Wall-clock timing is different: startup overhead, cache, parallelism and hardware utilization can bend the curves. A log-log slope near 3 is a possible large-size regime, not a requirement for all three measured lines. Compare speedups only at matching sizes and precision.
" ] }, { "cell_type": "markdown", "id": "30ba8f66", "metadata": {}, "source": [ "### ๐งช Exercise 6 โ put a number on the gap\n", "\n", "At the largest size measured by **both** the pure-Python and NumPy runs, compute two numbers:\n", "\n", "- **speedup** = (pure-Python time) รท (NumPy time) โ how many times faster NumPy is.\n", "- **GFLOP/s** = $2 n^3 / t / 10^9$ โ billions of floating-point operations per second (a matmul does about $2n^3$ of them)." ] }, { "cell_type": "code", "execution_count": null, "id": "16153877", "metadata": { "execution": { "iopub.execute_input": "2026-08-17T12:57:54.230001Z", "iopub.status.busy": "2026-08-17T12:57:54.229494Z", "iopub.status.idle": "2026-08-17T12:57:54.235398Z", "shell.execute_reply": "2026-08-17T12:57:54.234308Z" } }, "outputs": [], "source": [ "n_common = max(set(PURE_SIZES) & set(NP_SIZES)) # biggest size both methods ran\n", "t_py = pure_times[PURE_SIZES.index(n_common)]\n", "t_np = numpy_times[NP_SIZES.index(n_common)]\n", "# ==================== YOUR CODE HERE ====================\n", "### HINT: a matmul does ~2*n^3 flops; GFLOP/s = (2*n^3) / time / 1e9\n", "...\n", "\n", "print(f\"At {n_common} x {n_common}, the SAME multiplication:\")\n", "print(f\" pure Python : {t_py*1000:12.1f} ms\")\n", "print(f\" NumPy : {t_np*1000:12.3f} ms โ {speedup:,.0f}x faster, {gflops_np:,.1f} GFLOP/s\")\n", "if gpu_sizes and n_common in gpu_sizes:\n", " t_gpu = gpu_times[gpu_sizes.index(n_common)]\n", " gflops_gpu = 2 * n_common**3 / t_gpu / 1e9\n", " print(f\" GPU : {t_gpu*1000:12.3f} ms โ {t_py/t_gpu:,.0f}x faster than pure Python, {gflops_gpu:,.1f} GFLOP/s\")" ] }, { "cell_type": "markdown", "id": "6dd6cb3a", "metadata": {}, "source": [ "**Same formula. Same answer. A factor of thousands โ sometimes ~10,000ร โ in speed.** Nobody changed the math. They changed *how the hardware runs it*: memory layout, cache blocking, vectorized instructions, and thousands of GPU cores working in parallel.\n", "\n", "That is exactly what **high-performance computing** is about. **Keep this plot.** When you meet HPC for real, your mission: take the gap you just measured and *close it yourself*." ] }, { "cell_type": "markdown", "id": "31845565", "metadata": {}, "source": [ "Neural inference combines matrix products with activations, normalization, memory access and other work. Its cost depends on the architecture, context length and generated token count; one FLOP is not one whole matrix multiplication.
At large scale, improving kernels, memory movement and batching can lower latency and cost. Use the experiment to identify which part of a measured workload is limiting performance, rather than assuming every operation is A @ B.