Formula & theorem reference

For samples of size n drawn from a population with standard deviation \sigma, the standard error of the mean is SE = \frac{\sigma}{\sqrt n}. It is the standard deviation of the sampling distribution of \bar x itself, not of any one individual observation. It measures how much a sample mean is expected to vary from sample to sample, a different question from how much individual data points vary within one sample. When \sigma is unknown, the sample standard deviation s substitutes directly: SE \approx s/\sqrt n.

Built from the standard error, the margin of error for a confidence interval for a mean, when \sigma is known, is ME = z^* \cdot SE = z^* \cdot \frac{\sigma}{\sqrt n}, where z^* is the critical value fixed by the chosen confidence level (1.96 for 95% confidence, for instance). The resulting interval is \bar x \pm ME. When \sigma is unknown, s estimates it instead. That same shape then holds with t^* (at n-1 degrees of freedom) in place of z^*.

Built from the standard error, the margin of error for a confidence interval for a mean, when \sigma is known, is ME = z^* \cdot SE = z^* \cdot \frac{\sigma}{\sqrt n}, where z^* is the critical value fixed by the chosen confidence level (1.96 for 95% confidence, for instance). The resulting interval is \bar x \pm ME. When \sigma is unknown, s estimates it instead. That same shape then holds with t^* (at n-1 degrees of freedom) in place of z^*.

For samples of size n drawn from a population with standard deviation \sigma, the standard error of the mean is SE = \frac{\sigma}{\sqrt n}. It is the standard deviation of the sampling distribution of \bar x itself, not of any one individual observation. It measures how much a sample mean is expected to vary from sample to sample, a different question from how much individual data points vary within one sample. When \sigma is unknown, the sample standard deviation s substitutes directly: SE \approx s/\sqrt n.

Built from the standard error, the margin of error for a confidence interval for a mean, when \sigma is known, is ME = z^* \cdot SE = z^* \cdot \frac{\sigma}{\sqrt n}, where z^* is the critical value fixed by the chosen confidence level (1.96 for 95% confidence, for instance). The resulting interval is \bar x \pm ME. When \sigma is unknown, s estimates it instead. That same shape then holds with t^* (at n-1 degrees of freedom) in place of z^*.

Built from the standard error, the margin of error for a confidence interval for a mean, when \sigma is known, is ME = z^* \cdot SE = z^* \cdot \frac{\sigma}{\sqrt n}, where z^* is the critical value fixed by the chosen confidence level (1.96 for 95% confidence, for instance). The resulting interval is \bar x \pm ME. When \sigma is unknown, s estimates it instead. That same shape then holds with t^* (at n-1 degrees of freedom) in place of z^*.

For paired data (x_i, y_i), the correlation coefficient is r = \frac{n\sum x_iy_i - \sum x_i \sum y_i}{\sqrt{\left[n\sum x_i^2 - \left(\sum x_i\right)^2\right]\left[n\sum y_i^2 - \left(\sum y_i\right)^2\right]}}, a quantity always satisfying -1 \le r \le 1, computed by calculator or software in practice, exactly like m and b themselves. What matters is reading the value it returns:

• r close to +1: a strong positive linear relationship. As x goes up, y reliably goes up too, and the points hug the line tightly.
• r close to -1: a strong negative linear relationship. As x goes up, y reliably goes down, points still hugging the line tightly.
• r close to 0: little to no linear relationship. The points are scattered too loosely (or don't trend in a straight line at all) for a line to say much of anything.
• The sign of r always matches the sign of the slope m of the fitted line. Both record the same fact, that y rises or falls with x.

For n data points (x_1,y_1), \dots, (x_n,y_n), the line y = mx+b minimizing the sum of squared vertical errors E(m,b) = \sum_{i=1}^n \big(y_i - (mx_i+b)\big)^2 has slope and intercept m = \frac{n\sum x_iy_i - \sum x_i \sum y_i}{n\sum x_i^2 - \left(\sum x_i\right)^2}, \qquad b = \frac{\sum y_i - m\sum x_i}{n}.

For a least-squares fit to n paired observations (x_i, y_i), total variation splits into two parts -- an explained part and an unexplained part. Let \hat y_i = mx_i + b be the fitted values, and \bar y the mean. Then \underbrace{\sum_{i=1}^n (y_i - \bar y)^2}_{SST} = \underbrace{\sum_{i=1}^n (\hat y_i - \bar y)^2}_{SSR} + \underbrace{\sum_{i=1}^n (y_i - \hat y_i)^2}_{SSE}, and from this, R^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST}. R^2 is the share of y's total variation that the line accounts for. It always satisfies 0 \le R^2 \le 1. For a simple linear regression, with one predictor, R^2 equals the square of the Pearson correlation coefficient: R^2 = r^2. Both formulas describe the same number, read two ways -- r measures how tightly the points hug a line, and R^2 measures how much of y's spread that line accounts for.

For a least-squares fit to n paired observations (x_i, y_i), total variation splits into two parts -- an explained part and an unexplained part. Let \hat y_i = mx_i + b be the fitted values, and \bar y the mean. Then \underbrace{\sum_{i=1}^n (y_i - \bar y)^2}_{SST} = \underbrace{\sum_{i=1}^n (\hat y_i - \bar y)^2}_{SSR} + \underbrace{\sum_{i=1}^n (y_i - \hat y_i)^2}_{SSE}, and from this, R^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST}. R^2 is the share of y's total variation that the line accounts for. It always satisfies 0 \le R^2 \le 1. For a simple linear regression, with one predictor, R^2 equals the square of the Pearson correlation coefficient: R^2 = r^2. Both formulas describe the same number, read two ways -- r measures how tightly the points hug a line, and R^2 measures how much of y's spread that line accounts for.

Plain R^2 = SSR/SST never decreases when another predictor is added to a regression model. This holds even for a predictor with no real relationship to the response -- it can only help the fit by chance. For n observations and p predictors, adjusted R^2 corrects for this. It rescales the unexplained share 1 - R^2 by the ratio of degrees of freedom used to degrees of freedom remaining: R^2_{\text{adj}} = 1 - (1 - R^2)\frac{n-1}{n-p-1}. Adding a predictor that genuinely improves the fit raises R^2_{\text{adj}}. Adding one that does not can lower it, even while plain R^2 still rises. That is what makes R^2_{\text{adj}}, not R^2, the fairer way to compare two regression models with a different number of predictors.

An F-statistic always compares two variance-like quantities. It is built as a ratio: a mean square explained by a model or a factor, over a mean square left over as unexplained error: F = \frac{\text{mean square explained}}{\text{mean square error}} = \frac{MS_{\text{explained}}/df_{\text{explained}}}{MS_{\text{error}}/df_{\text{error}}}. In one-way ANOVA this is F = MS_{\text{between}}/MS_{\text{within}}. Here df_{\text{between}} = k-1, for k groups, and df_{\text{within}} = N-k, for N total observations. In a regression's own overall F-test, it is F = MSR/MSE. This compares the variation the model's predictors explain to the variation left in the residuals. Under the null hypothesis -- no real group differences, or no real predictor effect -- F follows an F-distribution with those two degrees of freedom, and only stays near 1. A large F signals that the explained variation is too large, relative to the leftover error variation, to be due to chance alone. Only large values count as evidence against the null, so an F-test is always tested with a right-tailed p-value.

An F-statistic always compares two variance-like quantities. It is built as a ratio: a mean square explained by a model or a factor, over a mean square left over as unexplained error: F = \frac{\text{mean square explained}}{\text{mean square error}} = \frac{MS_{\text{explained}}/df_{\text{explained}}}{MS_{\text{error}}/df_{\text{error}}}. In one-way ANOVA this is F = MS_{\text{between}}/MS_{\text{within}}. Here df_{\text{between}} = k-1, for k groups, and df_{\text{within}} = N-k, for N total observations. In a regression's own overall F-test, it is F = MSR/MSE. This compares the variation the model's predictors explain to the variation left in the residuals. Under the null hypothesis -- no real group differences, or no real predictor effect -- F follows an F-distribution with those two degrees of freedom, and only stays near 1. A large F signals that the explained variation is too large, relative to the leftover error variation, to be due to chance alone. Only large values count as evidence against the null, so an F-test is always tested with a right-tailed p-value.

For n paired (or single-sample, against a hypothesized median) differences, first drop every difference that is exactly 0. This leaves n' nonzero differences. The sign test statistic S is simply the number of POSITIVE differences among those n': S = \#\{\text{differences} > 0\}. Under the null hypothesis, the population median of the differences is 0. Each nonzero difference is then equally likely to be positive or negative. So S \sim \text{Binomial}(n', 0.5). The p-value is read directly from that distribution. For larger n', a normal curve approximates it well enough to use instead. Either way, the p-value never comes from a z or t table. The sign test uses only each difference's sign. It discards the size of the difference entirely. That is exactly what makes it valid under far weaker conditions than a paired t-test. The cost is lower power, whenever the sizes do carry real information.

For n paired gaps, drop any gap exactly 0. Then rank the REMAINING gaps by size, smallest (1) to largest, ignoring their sign. Split any tied sizes by their average rank. Let W^+ be the sum of ranks for positive gaps, and W^- the sum for negative gaps. The two always satisfy W^+ + W^- = \frac{n'(n'+1)}{2}, for n' nonzero gaps. The Wilcoxon signed-rank statistic is the smaller of the two: W = \min(W^+, W^-). A small W means one sign's ranks run larger than the other's. That is evidence the gaps aren't centered at 0. Unlike the sign test, W also uses each gap's rank, not just its sign. That rank shows how big the gap is. This is why W is often stronger than the sign test, as long as the gaps are roughly even on both sides.

For two independent groups of size n_A and n_B, combine every observation from both groups. Rank them together from smallest (1) to largest. Split any tied values by their average rank. Let R_A be the sum of ranks landing in group A, and R_B the sum landing in group B. Convert each group's rank sum into its own U value: U_A = R_A - \frac{n_A(n_A+1)}{2}, \qquad U_B = R_B - \frac{n_B(n_B+1)}{2}, which always satisfy U_A + U_B = n_A n_B. The Mann-Whitney U statistic is the smaller of the two: U = \min(U_A, U_B). A small U signals that one group's values rank higher than the other's, across the combined ordering. Assume both groups' spreads have a similar shape. Then this reads as a comparison of the two groups' MEDIANS -- the same comparison a two-sample t-test makes with means.

Let two groups, A and B, each be split into the SAME set of subgroups. The split is by some third, confounding variable. Group A can have a higher rate than group B within EVERY single subgroup. Yet group B can still have the higher rate OVERALL, once the subgroups are combined. This happens whenever the confounding variable is skewed between A and B. Say A's cases sit mostly in a harder subgroup, and B's in an easier one. Then the combined, weighted average can reverse a direction that holds in every single subgroup.

Both the combined comparison and every within-subgroup comparison can be worked out correctly. The paradox is a real result of how weighted averages combine unevenly-sized groups. It is not a mistake in either piece of math. Guarding against it means checking whether a likely confounding variable is skewed across the groups being compared. Report the within-subgroup comparison whenever one is found.

Let X and Y be two measurements of the same units, like a first test and a second test. Their correlation is r, with |r| < 1. So the two measurements are related, but not perfectly. Now pick units because their X value was extreme, far from X's own mean. Their average Y value moves back toward Y's mean, but it closes only part of that gap. The closed fraction is exactly r, in standardized terms: \frac{E[Y \mid X = x] - \mu_Y}{\sigma_Y} = r \cdot \frac{x - \mu_X}{\sigma_X}. Since |r| < 1, the right side is always a SHRUNKEN version of how extreme x was. The predicted follow-up score drifts back toward the mean. This drift comes purely from the imperfect correlation, with no underlying change, intervention, or explanation required. This is why a group selected for an extreme first score tends to look "more average" on a second, independent-ish measurement. Think of the worst test-takers, or the best-performing funds -- nothing about the group actually changed.

Suppose m independent hypothesis tests are each run at significance level \alpha. Suppose also that every one of their null hypotheses happens to be exactly true. The probability that at least one of the m tests comes out "significant" purely by chance is the family-wise error rate: P(\text{at least one false positive}) = 1 - (1 - \alpha)^m, since (1-\alpha)^m is the probability that all m tests correctly fail to reject, given independence. This grows fast with m. At \alpha = 0.05, running m=6 independent tests already gives roughly a 26\% chance of at least one false "significant" result. At m=20 that chance is roughly 64\%. Both numbers come purely from running enough tests, with no real effect anywhere. This is why we need a fix for running many tests at once, instead of applying \alpha separately to each one.