🕹️ Practice arcade

💡 Self-Information & Units Sprint

Compute -log2(p) and convert bits to nats -- the base of the logarithm sets the unit, not a separate formula.

🎲 Binary Entropy Sprint

Find H(X) for a biased coin -- the full two-term formula, not just the bigger probability's own share.

🔢 Hamming Distance Blitz

Count disagreeing positions between two bit-strings -- fast, exact, no partial credit for close.

🔗 Mutual Information & Divergence Sprint

Read a joint table for I(X;Y), or compare two point distributions with D(P||Q) -- both built from the same log-ratio sum.

🛰️ Syndrome Decoding Sprint

Reverse a (7,4) Hamming code's syndrome to (row 3, row 2, row 1) and read the error position straight off as binary.

📉 Rate-Distortion & Stationary Distributions Sprint

Find R(D)=h(p)-h(D) for a lossy Bernoulli source, or pi_1 for a two-state Markov chain -- two different chapters' own closed-form results.

🧩 Joint & Conditional Entropy Sprint

Sum -p(x,y)log2 p(x,y) over a joint table for H(X,Y), then subtract off H(X) for H(Y|X) -- the chain rule, worked both ways.

📐 Cross-Entropy & Gibbs' Inequality Sprint

Compute H(P,Q) directly, then judge whether D(P||Q) can dip below 0 -- it can't, and Gibbs' inequality is why.

🌲 Huffman Merge Blitz

Spot the two least probable symbols and add their probabilities -- the one move Huffman's algorithm always makes first.

📦 Typical Sets & Source Coding Sprint

Scale entropy by block length for the AEP's typical-set exponent, or subtract H(X) from an achieved code length for its redundancy.

📡 Differential Entropy & Channel Capacity Sprint

log2 of an interval's width for a continuous source, or 1 minus a channel's own uncertainty for its capacity -- three closed forms, one course.

🧮 Code Parameters Sprint

The smallest Hamming parity-check count, the Singleton bound on minimum distance, or a linear code's own rate k/n -- three ways to size up a code.

🔗 Chain Rule & Conditional MI Sprint

Add H(X) and H(Y|X) for the chain rule, or subtract two conditional entropies for I(X;Y|Z) -- both closed forms, no joint table needed.

📏 Code Length & Entropy Bound Sprint

Compute a code's expected length by weighting each codeword by how often it's used, then judge whether a proposed L could ever beat the source's own entropy floor.

🍰 Shannon-Fano & Arithmetic Coding Sprint

Run this course's own top-down pie-slicing rule to find a Shannon-Fano codeword length, or multiply probabilities down to find arithmetic coding's narrowed interval width.

🗜️ LZW Dictionary Blitz

Add new phrases to the starting alphabet -- the one arithmetic behind every LZW dictionary's growth.

📊 Gaussian Differential Entropy Sprint

Find h(X) for a Gaussian from its variance alone -- the distribution that maximizes differential entropy at any fixed variance.

🔲 Generator & Parity-Check Matrix Sprint

Encode a message as x=uG, or compute a received vector's syndrome as xH^T -- both mod-2 matrix arithmetic, read off as one binary number.

🛡️ CRC Remainder Blitz

Shift a message left by the generator's degree and divide by XOR -- the exact division CRC error detection relies on.

🎚️ Uniform Quantizer Sprint

Step size squared over 12 -- the mean-squared distortion every uniform scalar quantizer pays for chopping a continuous range into finitely many levels.

🗝️ One-Time Pad Blitz

XOR a message against its key, bit by bit -- the entire one-time pad, as fast as you can read off the ciphertext.

📈 Fisher Information & Cramer-Rao Sprint

Find a Bernoulli trial's own Fisher information, then turn it into the Cramer-Rao floor on any unbiased estimator's variance.

🤖 ML Information Theory Sprint

Cross-entropy loss on the true label, information gain from a decision-tree split, or softmax with a temperature knob -- three machine-learning ideas, all this course's own vocabulary underneath.

🌳 Kraft Inequality Survival

Sum 2^(-l_i) and check it against 1 -- one wrong verdict about a proposed prefix code ends the round.

🔻 Data Processing Inequality Survival

For X -> Y -> Z, I(X;Z) is capped by the SMALLER of I(X;Y) and I(Y;Z) -- never their sum, never their difference. One wrong bound ends the round.

📶 Noisy-Channel Coding Theorem Survival

Reliable communication is possible exactly when the rate sits below capacity -- one wrong call about which side of R=C you're on ends the round.

🧾 McMillan Inequality Survival

Sum 2^(-l_i) and check it against 1 -- McMillan's inequality extends the same test to every uniquely decodable code, not just prefix codes. One wrong verdict ends the round.

🔒 Perfect Secrecy Survival

H(K) has to be at least H(M), never the other way around -- one wrong call about whether a key's entropy is enough ends the round.