This method is rather raw and requires quite a lot of
computing resources, but it has the advantage that it does not
need any evaluation of the cost function's gradient, and that
it is quite easily implemented. First, we choose N+1
starting points, given here by a starting point $
\mathbf{P}{0} $ and N points such that
$$
\mathbf{P}{\mathbf{i}}=\mathbf{P}{0}+\lambda \mathbf{e}{\mathbf{i}},
$$
where $ \lambda $ is the problem's characteristic length scale). These
points will form a geometrical form called simplex.
The principle of the downhill simplex method is, at each
iteration, to move the worst point (highest cost function value)
through the opposite face to a better point. When the simplex
seems to be constrained in a valley, it will be contracted
downhill, keeping the best point unchanged.
Multi-dimensional simplex class
This method is rather raw and requires quite a lot of computing resources, but it has the advantage that it does not need any evaluation of the cost function's gradient, and that it is quite easily implemented. First, we choose N+1 starting points, given here by a starting point $ \mathbf{P}{0} $ and N points such that $$ \mathbf{P}{\mathbf{i}}=\mathbf{P}{0}+\lambda \mathbf{e}{\mathbf{i}}, $$ where $ \lambda $ is the problem's characteristic length scale). These points will form a geometrical form called simplex. The principle of the downhill simplex method is, at each iteration, to move the worst point (highest cost function value) through the opposite face to a better point. When the simplex seems to be constrained in a valley, it will be contracted downhill, keeping the best point unchanged.