top of page


Neural Network Search - NAS
For years, neural network architectures were designed by hand. The space of possible designs is enormous: how many layers, how wide each one is, how many branches, which kernel sizes, what input resolution. Turning all of those knobs manually, and getting them right together, is famously difficult. Neural architecture search (NAS) hands the tuning to an algorithm, which explores those knobs under whatever accuracy and efficiency constraints we set. That saves a great deal of
NOURA ALSHAREEF
Oct 2624 min read


The building blocks NAS chooses from
Oct 23, 2026 · @Nora 1. Quick recap: what a network is made of Say we have data for 100,000 patients, and for every patient five measurements. Those five numbers are the inputs x₀…x₄ of one data point. We do not feed patients one by one; we take them in batches, maybe 100 at a time, so one batch is a table with 100 rows and 5 columns. In the picture below we use 2 at a time so it stays readable: patient A in front, patient B behind. The data decide the values in the circles.
NOURA ALSHAREEF
Oct 2415 min read


Quantization, Part 2
Oct 22, 2026 · @Nora 1. Where we stopped Last time we laid a ruler over the weights: evenly spaced marks, every weight snaps to the nearest one, and two numbers (scale and zero point) bring it back. That is linear quantization, and it is what lets a layer run on integers. But the ruler left three questions open, and they decide whether the quantized model keeps its accuracy: One ruler or many? A single ruler for a whole layer, or one per channel, or one per small group of wei
NOURA ALSHAREEF
Oct 2313 min read


NN Quantization
1. Why fewer bits? Pruning made the network smaller by deleting weights. Quantization keeps every weight but stores each one in fewer bits. The two stack: a pruned network can also be quantized. Why it pays off: Memory. A weight in 32-bit float takes 4 bytes. The same weight in 8-bit integer takes 1 byte: the model is 4× smaller and moves 4× less data from memory, which in Why Is My Neural Network Slow? we saw is where most of the time and energy go. Energy. An 8-bit integer
NOURA ALSHAREEF
Oct 2310 min read


How Much to Prune, and How to Recover
1. Where we stopped last time In Part 1 we saw what pruning is: delete the weights the network does not need. In Part 2 we answered two of the four questions: in what shape do we delete (granularity), and which weights go (criterion). Two questions are left, and they are the ones you meet first when you actually prune a model: How much do we prune in each layer? Pruning every layer by the same amount is simple, but it is not the best. Some layers can lose 90% of their weights
NOURA ALSHAREEF
Oct 239 min read


A Deep Dive into Pruning Patterns
A simple guide to choosing the pattern and the weights to prune In this article Where we stopped last time Granularity: fine vs coarse Five patterns in a convolution layer 3.1 Fine-grained: any weight, anywhere 3.2 Pattern-based: zeros in allowed shapes 3.3 Vector-level: a whole row of a kernel 3.4 Kernel-level: a whole 3×3 kernel 3.5 Channel-level: a whole filter Criterion: which weights go? 4.1 Magnitude 4.2 Wanda: weights × inputs Test yourself: one panel, five patterns 1.
NOURA ALSHAREEF
Oct 1511 min read


NN Pruning
A simple guide to removing the weights a neural network does not need In the last post we used the roofline to find what slows a network down. One of the fixes was a single line: pruning means removing weights that don't matter. Fewer weights, fewer MACs. This post explains that line. It answers two questions: What is pruning? How do we write pruning as a math problem? Why remove weights at all? Every weight costs us twice. It takes bytes in memory, and it takes one MAC every
NOURA ALSHAREEF
Oct 138 min read


Why Is My Neural Network Slow?
A simple guide to matrix multiplication, hardware, and the roofline 1. It is all matrix multiplication A neural network does a lot of math. Most of that math is one thing: matrix multiplication. A layer in a network takes an input and multiplies it by a table of numbers called weights. That is a matrix multiplication. Most of the time a network runs, it is doing this. So if we want fast AI, we need fast matrix multiplication. الشبكة العصبية تقضي معظم وقتها في ضرب المصفوفات، ل
NOURA ALSHAREEF
Sep 2512 min read
Intro to LLMs
Andrej Karpathy's "Intro to Large Language Models" came out in 2023, and I still go back to it. The details have aged, but the story hasn't. It gives you the full picture of what an LLM is and how the ideas connect, from "a model is just two files" to pretraining, finetuning, and RLHF. Every time I rewatched it, the same questions came up: what actually happens inside the network? How does text become numbers? What is attention doing? So I turned the first part of the talk in
NOURA ALSHAREEF
Sep 241 min read


An Interactive Guide to CNNs
I always wanted one place with everything about CNNs: the details, the visualizations, and the chance to play with the numbers. So I built it: an interactive reference where every step, from pixels to prediction, is computed live and explained in plain words. I hope you enjoy it. You can also experiment with a larger width at: https://nora-alshareef.github.io/cnn-interactive-guide/
NOURA ALSHAREEF
Sep 191 min read

STAY IN THE KNOW
bottom of page