The The Power Method finds only the eigenvalue largest in size. To reach any other, apply it to a different matrix with the same eigenvectors. If $A\mathbf{v} = \lambda\mathbf{v}$ and $s$ is a number (the shift), then $$(A - sI)^{-1}\mathbf{v} = \frac{1}{\lambda - s}\,\mathbf{v}.$$
Key idea
Shifted inverse iteration runs the power method on $(A - sI)^{-1}$. It converges to the eigenvector whose eigenvalue $\lambda$ is closest to $s$, and the scale factors settle at $\mu = \frac{1}{\lambda - s}$, so $$\lambda = s + \frac{1}{\mu}.$$ Each step solves $(A - sI)\mathbf{y} = \mathbf{x}_k$ using one LU factorization made at the start; it never forms the inverse.
Why it works
The eigenvalues of $(A - sI)^{-1}$ are the numbers $\frac{1}{\lambda_i - s}$. The largest in size comes from the $\lambda_i$ with the smallest $|\lambda_i - s|$, so that is the one the power method finds. Its convergence ratio is the ratio of the two largest sizes, $\frac{|\lambda_{\text{near}} - s|}{|\lambda_{\text{next}} - s|}$, which is tiny when $s$ is a good guess. Factoring $A - sI$ costs about $\frac{2}{3}n^{3}$ once; each step then costs only two triangular solves, about $2n^{2}$.
Worked example: The smallest eigenvalue
Use inverse iteration with $s = 0$ on $A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}$ from $\mathbf{x}_0 = (1, 0)$.
- Solve $A\mathbf{y} = (1, 0)$: $\mathbf{y} = \left(\frac{2}{3}, -\frac{1}{3}\right)$, so $\mu_1 = \frac{2}{3}$ and $\mathbf{x}_1 = (1, -0.5)$.
- Solve $A\mathbf{y} = (1, -0.5)$: $\mathbf{y} = \left(\frac{5}{6}, -\frac{2}{3}\right)$, so $\mu_2 = \frac{5}{6}$ and $\mathbf{x}_2 = (1, -0.8)$.
- $\mu$ heads to $1$, so $\lambda = 0 + \frac{1}{1} = 1$, the smallest eigenvalue, with eigenvector $(1, -1)$.
Answer: $\lambda = 1$
Worked example: Recovering λ
A matrix has eigenvalues near $1$, $4$ and $10$. With $s = 3.8$, the scale factors settle at $\mu = 5$. Which eigenvalue was found, and how fast was the convergence?
- $\lambda = 3.8 + \frac{1}{5} = 4$.
- Distances from $s$: $0.2$ to the eigenvalue $4$, $2.8$ to $1$, $6.2$ to $10$.
- Ratio $\frac{0.2}{2.8} = \frac{1}{14}$: more than one new digit every step.
Answer: $\lambda = 4$, ratio $\frac{1}{14}$
Worked example: A shift too close?
If $s$ is extremely close to an eigenvalue, $A - sI$ is nearly singular. Does that spoil the method?
- The computed $\mathbf{y}$ has a large error, but the error is almost entirely along the wanted eigenvector, the direction $(A - sI)^{-1}$ stretches most.
- Scaling removes the size, so the direction is still right. A nearly singular $A - sI$ is the method working well, not failing.
Answer: no
Updating the shift every step to the current Rayleigh quotient gives Rayleigh quotient iteration, which for symmetric matrices triples the number of correct digits each step, at the price of a new factorization each time.
Common mistake
Reporting the eigenvalue $\mu$ of $(A - sI)^{-1}$ as the answer. The eigenvalue of $A$ is $s + \frac{1}{\mu}$. Check: your answer should be close to the shift (inverse iteration finds the eigenvalue nearest $s$), and $A\mathbf{x} \approx \lambda\mathbf{x}$ should hold for the final vector.