| title |
author |
institute |
classoption |
fleqn |
abstract |
Fréchet Derivative |
Keith A. Lewis |
KALX, LLC |
fleqn |
true |
Infinite dimensional derivatives |
\newcommand\bm[1]{\boldsymbol{#1}}
\newcommand\RR{\boldsymbol{R}}
\newcommand\BB{\mathcal{B}}
\newcommand\LL{\mathcal{L}}
\newcommand\MM{\mathcal{L}}
\newcommand\Var{\operatorname{Var}}
\newcommand\Cov{\operatorname{Cov}}
If $F\colon X\to Y$ is a function between normed linear spaces
the Fréchet derivative ${DF\colon X\to\BB(X,Y)}$ is defined by
$$
F(x + h) = F(x) + DF(x)h + o(h)
$$
for $x,h\in X$ where $\BB(X,Y)$ is the set of bounded linear operators from $X$ to $Y$.
The "little $o$" notation means
$|F(x + h) - F(x) - DF(x)h|/|h| \to 0$ as $|h| \to 0$.
The function $F$ can be approximated near $x$ by the linear operator $DF(x)$.
A norm is a function $|\cdot|\colon X\to\RR$ satisfying
$|ax| = |a||x|$, $|x + y| \le |x| + |y|$, $|x|\ge0$
and $|x| = 0$ implies $x = 0$ for $a\in\RR$, $x,y\in X$.
If $X = Y = \RR$ then $DF\colon\RR\to\BB(\RR,\RR)$ where $DF(x)h = F'(x)h$.
If $X = \RR^n$ and $Y = \RR^m$ then $DF(x)\colon\RR^n\to\RR^m$ is the Jacobian.
Exercise. If $T\colon X\to X$ is a bounded linear operator then $DT = T$.
If $G\colon Y\to Z$ then $D(G\circ F)(x) = DG(F(x))DF(x)$
Exercise: Prove the chain rule for Frechet derivatives.
Hint: $G\circ F(x + h) = G(F(x + h)) = G(F(x) + DF(x)h + o(h)) = G(F(x)) + DG(F(x))DF(x)h + o(h)$.
Define the dual of a normed linear space $X$ by the vector space
of bounded linear functionals $X^* = \BB(X,\RR)$.
If ${X^I = {x\colon I\to\RR}}$ is the set of functions from the set $I$
to $\RR$ we can define an inner product $(\cdot,\cdot)\colon X\times
X\to\RR$ by $(x, y) = \sum_{i\in I}x(i) y(i)$ if $I$ is finite. We also
write $x_i$ for $x(i)$.
Exercise. Show the inner product is bilinear.
Hint: Show $(ax + y, z) = a(x,y) + (y,z)$ and $(x, ay + z) = a(x,y) + (x,z)$
for $a\in\RR$, $x,y,z\in X$ using $(x,y) = \sum_{i\in I}x_i y_i$.
The inner product defines a norm by $|x| = \sqrt{(x, x)}$.
Exercise. Show $|(x,y)| \le |x| |y|$ for $x,y\in X$.
Hint: Use $0\le|ax + y|$ for $a\in\RR$, $x,y\in X$ and minimize over $a$.
Exercise. Show if $|(x,y)| = |x| |y|$ and $x\not=0$ then $y = ax$ for some $a\in\RR$.
Exercise. Show $|\cdot|$ is a norm.
The canonical basis of $\RR^I$ is $e_i\in\RR^I$, $i\in I$, defined by $e_i(j) = \delta_{ij}$,
where $\delta_{ij} = 1$ if $i = j$ and is zero otherwise.
Exercise: Show $x = \sum{i\in I} x(i)e_i$ for $x\in\RR^I$_.
Hint: Compute $x(j)$, $j\in I$.
Define the dual basis
$e_j^\in(\RR^I)^$, $j\in I$, by $e_j^*(e_i) = \delta_{ji}$.
Exercise: Show $x^* = \sum{j\in I} x^(e_j) e_j^$ for $x^\in X^$_.
Hint: Compute $x^*(e_i)$, $i\in I$.
The dual map $\colon\RR^I\to(\RR^I)^$ defined by $e_i\mapsto e_i^$ allows us to identify
$(\RR^I)^$ with $\RR^I$.
Exercise. Show the dual map is an isometric isomorphism.
Exercise: _Show $D|x|^p = p|x|^{p-1}x^*$ for $x\in\RR^I$.
Hint: Compute $D|x|^2$ and use $|x|^p = \exp(p/2 \log|x|^2)$.
Exercise: Show $(Dx^2)h = hx + xh$ if $x$ is a linear operator in $\BB(V,V)$.
For $x\in\BB(V,V)$ define right and left multiplication by $R_xy = xy$ and $L_xy = yx$.
Exercise: Show $Dx^n = \sum{j=0}^{n - 1} R_x^{n - j} L_x^j$_.
Hint: The previous exercise establishes the case $n = 2$. Use induction.