Section 6 introduced the iterate superscript x^{(k)} informally, just long enough to talk about limits. This closing section makes it official and pairs it with the other piece of bookkeeping every later chapter leans on: summation notation, for adding up a whole family of terms -- every data point's error, every constraint's contribution to a penalty, every variable's share of a total -- without writing a growing list of plus signs. Both notations are ordinary tools this course simply commits to using consistently from here forward.
Think of x^{(k)} as a dated photograph in an album -- x^{(0)} is the photo taken on day zero, x^{(1)} the one taken on day one, and so on, with the number in parentheses always meaning "which day," never "raised to a power." Summation notation solves a completely different annoyance: writing out a long list of additions by hand. Instead of writing a_1+a_2+a_3+\cdots+a_{100} and trusting the reader to fill in the pattern with the dots, \sum_{i=1}^{100}a_i says the exact same thing without any ambiguity about what belongs in the middle. Put the two together and a sentence like "the total cost after five steps" becomes \sum_{j=0}^{5}\text{cost}(x^{(j)}) -- add up the cost of every dated photo from day zero through day five, one clean symbol standing in for six separate additions.
The iterate sequence x^{(0)},x^{(1)},x^{(2)},\dots names the guess an algorithm holds after each step, with x^{(0)} the starting point and x^{(k)} the guess after k steps; the parenthesized superscript is notation for "which step," never an exponent, and this course never reuses it to mean anything else. Summation notation \sum_{i=1}^n a_i=a_1+a_2+\cdots+a_n compresses a sum of n terms into one symbol, with i the index running from the lower limit to the upper limit, one term per value of i. The two notations combine constantly: an algorithm's total cost after k steps might be written \sum_{j=0}^{k}\text{cost}(x^{(j)}), summing one cost value per iterate produced so far, and an objective built from many data points is routinely written f(\mathbf x)=\sum_{i=1}^m \ell_i(\mathbf x), one loss term \ell_i per data point, summed into a single number.
A running total, one term at a time
Toggle the partial-sums view and watch the running total accumulate exactly one new term per step -- that running total, stopped after n terms, is exactly what \\sum_{i=1}^{n}a_i means.
Loading interactive…
The iterate sequence x^{(k)} = A^k x^{(0)}
Increase k and watch x^{(k)} march toward a fixed point one superscript at a time -- each new k produces the next dated entry in the sequence, never a new variable name.
Loading interactive…
Summing three iterates directly
This is \\sum_{k=1}^{3}x^{(k)}, three design points added tip to tail into a single vector -- drag any one of them and watch the sum arrow update live.
Loading interactive…
Worked example
Evaluating a sum over an iterate sequence
-
1
An algorithm produces the iterate sequence x^{(0)}=1, x^{(1)}=1.5, x^{(2)}=1.75, x^{(3)}=1.875. Evaluate \displaystyle\sum_{k=0}^{3}\bigl(x^{(k)}\bigr)^2.
-
2expanding the sum first, before computing, avoids losing track of a term
Write out one term per value of k from 0 to 3: (x^{(0)})^2+(x^{(1)})^2+(x^{(2)})^2+(x^{(3)})^2.
expanding the sum first, before computing, avoids losing track of a term
-
3
Square each iterate: 1^2=1, 1.5^2=2.25, 1.75^2=3.0625, 1.875^2=3.515625.
-
4
Add all four: 1+2.25+3.0625+3.515625=9.828125.
-
5
\boxed{\displaystyle\sum_{k=0}^{3}\bigl(x^{(k)}\bigr)^2=9.828125} Four dated values, squared and totaled by one compact expression -- exactly the form later chapters use for a running sum of squared errors or squared step sizes across an entire run of iterates.
Where this goes next
Every algorithm and every loss function in this course
This is the last piece of shared vocabulary standing between this chapter and the rest of the course. An algorithm's convergence history is a sequence of iterates; its total cost is a sum over that sequence; a model's training objective is a sum over every data point's individual loss. From chapter 1 forward, every objective function, every algorithm's update rule, and every convergence argument in this course is written in exactly the two notations this section just fixed.
Check your understanding (counts toward your progress)
Why does this course write x^{(0)} for the starting point instead of just calling it x?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Both refer to the same kind of object, a design point -- the notation difference is about position in a sequence, not a different mathematical type.
The superscript (0) has nothing to do with the VALUE being zero -- x^{(0)}=(4,-1,7) is a perfectly ordinary starting point.
This course is specific about the superscript -- x^{(0)} signals membership in an iterate sequence, which plain x does not.
Why
Labeling the starting point x^{(0)} fits it into the same x^{(0)},x^{(1)},x^{(2)},\dots pattern every later iterate uses, making the whole sequence's notation consistent from step zero onward.
What does \sum_{i=1}^{n}a_i mean?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Multiplying every term is a product, usually written with \\prod -- \\sum specifically denotes addition.
Summation adds every term in the range, not just the largest one -- picking a single maximum is a different operation entirely.
Every term from i=1 through i=n contributes -- none of the earlier terms are discarded.
Why
\sum is summation notation: add one term per value of the index, from the lower limit to the upper limit inclusive.
An algorithm's total cost after k steps is written \sum_{j=0}^{k}\text{cost}(x^{(j)}). What does this expression compute?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
A sum over j=0 through k adds every one of those iterates' costs together -- it isn't reporting just the last one.
An average would divide the total by the number of terms -- this expression is the raw total, with no division.
Each term in the sum is \\text{cost}(x^{(j)}), a DIFFERENT iterate's cost for each j -- it isn't the same value repeated.
Why
The lower and upper limits (j=0 to j=k) name exactly which iterates' costs get added -- every one from the start through step k.
Explain why writing \sum_{i=1}^{100}a_i is preferable to writing a_1+a_2+a_3+\cdots+a_{100} with dots standing in for the missing terms.
AI grading isn't configured on this deployment.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
First mistake
Correct answer
Also accepted:
Why
Think about what could go wrong if a reader has to guess the pattern behind the dots.
What does the superscript in x^{(k)} mean in this course's notation?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
The parentheses are exactly what rules out the exponent reading -- x^{(k)} and x^k are deliberately different symbols.
A component index is usually written as a subscript, x_k, not this parenthesized superscript -- the two notations are kept separate on purpose.
The parenthesized superscript has one fixed meaning throughout this course -- which step of an iterative process produced this value.
Why
The parentheses around k are the whole point -- they mark k as a step label, not an exponent, so x^{(2)} reads as "the guess after step 2," never as x squared.
Check your understanding (counts toward your progress)
An algorithm records the iterates x^{(0)}=5, x^{(1)}=3, x^{(2)}=8, and x^{(3)}=2. What total do you get by adding every one of them together?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
5+3+8+2=18.
Correct answer
Also accepted:
Why
Compute \displaystyle\sum_{i=1}^{5}2i, a sum of a plain multiple of the index.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
2(1)+2(2)+2(3)+2(4)+2(5)=2+4+6+8+10=30.
Correct answer
Also accepted:
Why
An iterate sequence adds a constant each step: x^{(k+1)}=x^{(k)}+3 with x^{(0)}=2. What is x^{(4)}?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
x^{(1)}=5, x^{(2)}=8, x^{(3)}=11, x^{(4)}=14 -- adding 3 each step, four steps total from x^{(0)}=2.
Correct answer
Also accepted:
Why
Starting from x^{(0)}=3 and applying the update x^{(k+1)}=2x^{(k)}-1 twice, what value does the sequence reach?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
x^{(1)}=2(3)-1=5, then x^{(2)}=2(5)-1=9.
Correct answer
Also accepted:
Why
Compute \displaystyle\sum_{i=0}^{3}i^2.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
0^2+1^2+2^2+3^2=0+1+4+9=14.
Correct answer
Also accepted:
Why
Practice: Sums and iterates, worked back to back (3 problems · doesn’t affect your progress)
Compute \displaystyle\sum_{i=1}^{4}(2i+1), a sum where each term also gets a constant added.
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
3+5+7+9=24.
Correct answer
Also accepted:
Why
An iterate sequence halves the previous term and adds a constant: x^{(k+1)}=x^{(k)}/2+1 with x^{(0)}=10. What is x^{(2)}?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
Why
x^{(1)}=10/2+1=6; x^{(2)}=6/2+1=4.
Correct answer
Also accepted:
Why
Which expression correctly writes "the sum of the squared distance between each iterate and the target x^\ast, from step 0 through step k"?
Right on try — credit. Right — the answer had already been shown, so no credit this time. Next try is worth Answer shown after three tries — this one no longer earns credit. You chose to see the answer — this one earns no credit.
That's only the LAST term's squared distance, not a sum over every step from 0 through k.
The squaring has to happen BEFORE summing -- this subtracts x^\\ast only once, after the sum, which isn't the same quantity.
This just repeats the first term's squared distance k times rather than actually summing k+1 different terms.
Why
Summation notation with the squared term inside the sum, \sum_{j=0}^{k}(x^{(j)}-x^\ast)^2, is exactly "add up each step's own squared distance."
\\sum_{j=0}^{k}\\bigl(x^{(j)}-x^\\ast\\bigr)^2