Taylor and Maclaurin Series

The idea

Near a point $a$, a smooth function looks like a polynomial whose derivatives at $a$ match those of $f$:

$$f(x) = f(a) + f'(a)(x - a) + \frac{f''(a)}{2!}(x - a)^2 + \dots + \frac{f^{(n)}(a)}{n!}(x - a)^n + R_n(x).$$

• Taylor series about $a$: $\displaystyle\sum_{k=0}^\infty\frac{f^{(k)}(a)}{k!}(x - a)^k$.
• Maclaurin series: the special case $a = 0$.
• The coefficient of $(x - a)^k$ is $f^{(k)}(a)/k!$ — so reading coefficients off a known series gives derivatives for free: $f^{(k)}(a) = k!\times\text{coefficient}$.

Series to know by heart

Manipulating series

Rather than differentiating repeatedly, build new series from known ones:

• Substitute: $e^{-x^2} = 1 - x^2 + \frac{x^4}{2} - \cdots$.
• Multiply and collect terms up to the order you need ($e^x\sin x$ in the practice questions).
• Differentiate or integrate term by term inside the radius of convergence: integrating $\frac{1}{1 + t^2} = \sum(-1)^nt^{2n}$ gives $\tan^{-1}x$.
• Divide by long division or by matching coefficients.

The remainder and error bounds

Lagrange form: for some $\xi$ between $a$ and $x$,

$$R_n(x) = \frac{f^{(n+1)}(\xi)}{(n+1)!}(x - a)^{n+1}.$$

If $\lvert f^{(n+1)}\rvert \le M$ on the interval, then $\lvert R_n(x)\rvert \le \frac{M}{(n+1)!}\lvert x - a\rvert^{n+1}$. For an alternating series with decreasing terms, the error is at most the first omitted term.

The notation $O((x - a)^{n+1})$ ("big-O") records that the error shrinks like $\lvert x - a\rvert^{n+1}$ as $x \to a$.

Convergence

A power series $\sum c_n(x - a)^n$ converges for $\lvert x - a\rvert < R$ and diverges for $\lvert x - a\rvert > R$, where $R$ is the radius of convergence: $\frac1R = \lim\left\lvert\frac{c_{n+1}}{c_n}\right\rvert$ (ratio test) when the limit exists. Endpoints must be checked separately. Within $R$ the series can be differentiated and integrated term by term.

Why Taylor series matter in data science

• Linear approximation: $f(x) \approx f(a) + f'(a)(x - a)$ — sensitivity analysis, the delta method in statistics, linearising non-linear models.
• Gradient descent minimises the first-order model plus a step-size penalty; Newton's method minimises the second-order model $f(a) + f'(a)h + \frac12f''(a)h^2$, giving $h = -f'(a)/f''(a)$ — see maxima and minima.
• Second-derivative test for extrema comes straight from the quadratic term.
• Numerical methods: finite-difference derivatives, numerical integration error analysis.
• Activation functions: near 0, $\tanh x \approx x$ and $\sigma(x) \approx \frac12 + \frac x4$.

Quick revision