$ \DeclareMathOperator{\arccosh}{arccosh} \DeclareMathOperator*{\argmin}{arg\,min} \DeclareMathOperator{\Exp}{Exp} \newcommand{\geo}[2]{\gamma_{\overset{\frown}{#1,#2}}} \newcommand{\geoS}{\gamma} \newcommand{\geoD}[2]{\gamma_} \newcommand{\geoL}[2]{\gamma(#2; #1)} \newcommand{\gradM}{\nabla_{\M}} \newcommand{\gradMComp}[1]{\nabla_{\M,#1}} \newcommand{\Grid}{\mathcal G} \DeclareMathOperator{\Log}{Log} \newcommand{\M}{\mathcal M} \newcommand{\N}{\mathcal N} \newcommand{\mat}[1]{\mathbf{#1}} \DeclareMathOperator{\prox}{prox} \newcommand{\PT}[3]{\mathrm{PT}_{#1\to#2}#3} \newcommand{\R}{\mathbb R} \newcommand{\SPD}[1]{\mathcal{P}(#1)} \DeclareMathOperator{\Tr}{Tr} \newcommand{\tT}{\mathrm{T}} \newcommand{\vect}[1]{\mathbf{#1}} $

The gradient descent algorithm on a manifold

Based on a manifold , a starting point , and a gradient (or descent) direction of a function this function performs a gradient descent algorithm

with step size , until a stopping criterion is fulfilled.

All functions involved are provided as anonymous functions: The gradient gradF is a function based on @(x), the step size is a function @(x,descentDir,iter,initial) based on the current iterate, descent direction , the iteration and an initial step size to start seaching with. This value is set to the last step size within the iterations. Finally the stoppingCriterion is a function based on @(x,xold,iter) that returns false as long as the algorithm should run, hence the algorithm stops if the stopping criterion is fulfilled for the first time.

Optional Parameters

Debug
If set to an anonymous functional depending on @(x,xold,iter) this is called in each cycle, for example to produce text output or save iteration data to a file.
Record
If set to a functional @(x,iter) returning a column vector, and calling the cyclicProximalPoint with a second return value, this function records these column vectors in a matrix returned in the second return value recData.

There are several gradients available.

Matlab Documentation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
% gradientDescent(M,x,gradF,stepSizeRule,stoppingCriterion)
%    compute a gradient descent based on
%
% INPUT
%    M                : a manifold M
%    x                : initial data
%    gradF            : a function @(x) returning the gradient of F at x
%    stepSize         : a function @(x,eta,iter,initial) returning the
%                       stepsize at x with direction eta corresponding to a
%                       certain rule (e.g. Amijo) based on iterate point x,
%                       descent direction eta, iteration iter and initial
%                       stepsize initial.
%   stoppingCriterion : a function @(x,xold,iter) determining whether to
%                       stop or not
%
% OPTIONAL
%   Debug             : ([]) a function @ (x,xold,iter) producing debug
%                          output
%   Record            : (@(x,iter)) a function returning a column vector,
%                       if there's a second return value and this function
%                       is given, data is recorded in an array and returned
% ---
% MVIRT | R. Bergmann | 2018-01-22

See also