Examples of Using Automatic Differentiation

by SeHyoun Ahn, June 2016

Contents

An html file properly marked up comments is avaiable at http://sehyoun.com/EXAMPLE_AutoDiff_Syntax.html

% Add folder containing <@myAD>. In this case, it is in the parent folder.
addpath('../');

Simple Example

$$f_1(x,y,z) = x^2+cos(y)*(x+z)+e^z$$ $$f_2(x,y,z) = yz$$ at $(x,y,z) = (1,2,3)$

v = myAD([1;2;3]);
x=v(1); y=v(2); z=v(3);
f=[x^2+cos(y)*(x+z)+exp(z);
    y*z];
disp('The function evaluated at (1,2,3) is');
disp(getvalues(f));
disp('the derivatives evaluated at (1,2,3) are');
disp(full(getderivs(f)));

The function evaluated at (1,2,3) is
   19.4209
    6.0000

the derivatives evaluated at (1,2,3) are
    1.5839   -3.6372   19.6694
         0    3.0000    2.0000

Example of Matrix Vector Multiplication

$$ f = \begin{bmatrix} x & y & 0 \\ y& y& z \\ 0 & z & z \end{bmatrix} \begin{bmatrix}x\\y\\z \end{bmatrix} = \begin{bmatrix} x^2+y^2\\ xy+y^2+z^2 \\ yz+z^2\end{bmatrix} $$ evaluated at $(x,y,z)=(1,2,3)$

v=myAD([1:3]');
A=spdiags(v,0,3,3)+spdiags(v(2:3),-1,3,3)+spdiags(v,1,3,3);
f=A*v;
disp('f is');
disp(getvalues(f));
disp('The Jacobian of f evaluated at (1,2,3) is')
disp(full(getderivs(f)));

f is
     5
    15
    15

The Jacobian of f evaluated at (1,2,3) is
     2     4     0
     2     5     6
     0     3     8

If you plan to use matrix multiplication in high dimensions, refer to the documentation to compile c-files for speed gains

Example of Using fsolve

Given an implicitly defined variable $z$ from the relationship $$x^2+y^4+x\cdot (z-0.2)\cdot e^z=5 $$ solving for $\frac{dz}{dx}$ and $\frac{dz}{dy}$ evaluated at $(x,y)=(1,1)$

% Refer to README.pdf for syntactic requirement on definition of functions.
% x = v(2), y = v(3), z = v(1)
func = @(v) v(2).^2+v(3).^4+v(2)*(v(1)-0.2)*exp(v(1))-5;
v0 = myAD([1;1]);            % Initialize x and y
z=fsolve(func, 0.5 ,v0);     % Refer to documentation for syntax
disp('Solution of z is given by');
disp(getvalues(z));
disp('The derivatives at (x,y)=(1,1) are');
disp(getderivs(z));

Equation solved.

fsolve completed because the vector of function values is near zero
as measured by the default value of the function tolerance, and
the problem appears regular as measured by the gradient.



Solution of z is given by
    1.1500

The derivatives at (x,y)=(1,1) are
   (1,1)      -0.8119
   (1,2)      -0.6496

Published with MATLAB® R2016b on 2017-04-04
    (and adjusted not to clash with Bootstrap)

SYSTEM INFORMATION FOR REPLICATION
Computer:
	Version: ThinkPad X230

CPU:
	Version: Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz

Memory:
	Number Of Devices: 2
Memory Device
	Size: 4096 MB
	Type: DDR3
	Speed: 1600 MHz
	Configured Clock Speed: 1600 MHz
Memory Device
	Size: 4096 MB
	Type: DDR3
	Speed: 1600 MHz
	Configured Clock Speed: 1600 MHz

OS:
Description:	Linux Mint 18 Sarah

MATLAB:
9.1.0.441655 (R2016b)