Functions of a Single Variable and Limits

Functions

A function $f: D \to \mathbb R$ assigns to each $x$ in its domain $D$ exactly one value $f(x)$; the set of values is the range.

Functions that appear constantly in data science: polynomials, $e^x$ and $\ln x$, the sigmoid $\sigma(x) = 1/(1 + e^{-x})$ (range $(0,1)$, $\sigma(-x) = 1 - \sigma(x)$, $\sigma' = \sigma(1 - \sigma)$), $\tanh x$, ReLU $\max(0, x)$, $\lvert x\rvert$, and the floor/ceiling functions.

Limits

$\lim_{x\to a}f(x) = L$ means $f(x)$ can be made as close to $L$ as we like by taking $x$ close enough (but not equal) to $a$. Formally: for every $\varepsilon > 0$ there is a $\delta > 0$ with $0 < \lvert x - a\rvert < \delta \Rightarrow \lvert f(x) - L\rvert < \varepsilon$.

• The value $f(a)$ itself is irrelevant — it may not even be defined.
• One-sided limits: $\lim_{x\to a^-}$ and $\lim_{x\to a^+}$. The two-sided limit exists iff both exist and are equal.

Limit laws

If $\lim f = L$ and $\lim g = M$: sums, differences, products and constant multiples behave as expected; $\lim f/g = L/M$ if $M \ne 0$; $\lim f^n = L^n$; and for continuous $h$, $\lim h(f(x)) = h(L)$.

Standard limits to memorise

Growth hierarchy as $x \to \infty$: $\ln x \ll x^k \ll a^x\ (a > 1) \ll x! \ll x^x$.

Indeterminate forms

$\frac00, \frac\infty\infty, 0\cdot\infty, \infty - \infty, 0^0, 1^\infty, \infty^0$ — the form alone does not determine the limit.

L'Hôpital's rule

If $\lim\frac{f}{g}$ is $\frac00$ or $\frac{\pm\infty}{\pm\infty}$, and $\lim\frac{f'}{g'}$ exists (or is $\pm\infty$), then $\lim\frac fg = \lim\frac{f'}{g'}$. Apply repeatedly if the form persists. Check the form first — applying it to a determinate form gives wrong answers.

Squeeze (sandwich) theorem

If $g(x) \le f(x) \le h(x)$ near $a$ and $\lim g = \lim h = L$, then $\lim f = L$. Example: $-x^2 \le x^2\sin(1/x) \le x^2$, so $x^2\sin(1/x) \to 0$ as $x\to0$.

Limits at infinity and asymptotes

• Rational functions $\frac{a_mx^m + \dots}{b_nx^n + \dots}$ as $x\to\pm\infty$: 0 if $m < n$; $a_m/b_n$ if $m = n$; $\pm\infty$ if $m > n$.
• Horizontal asymptote $y = L$ if $\lim_{x\to\pm\infty}f(x) = L$ (the sigmoid has $y = 0$ and $y = 1$).
• Vertical asymptote $x = a$ if $f(x) \to \pm\infty$ as $x \to a^\pm$ (e.g. $\ln x$ at $x = 0$).
• Oblique asymptote $y = mx + c$ with $m = \lim f(x)/x$ and $c = \lim(f(x) - mx)$.

Sequences

$\lim_{n\to\infty}a_n$ follows the same rules. Useful facts: $r^n \to 0$ for $\lvert r\rvert < 1$; $n^{1/n} \to 1$; $\left(1 + \frac{x}{n}\right)^n \to e^x$; a monotone bounded sequence converges.

Quick revision