In short, a derivative gives you the slope (or rate of change ... If we think of the loss function as being plotted on the graph, we want to move in increments toward the minimum of a function.

In calculus, the derivative of a function can be interpreted as the instantaneous slope of that function’s graph at any point. Students spend the better part of a year in calculus class learning how ...

The FD= and FDHESSIAN= options specify the use of finite difference approximations of the derivatives. The FD= option specifies that all derivatives are approximated using function evaluations, and ...

You need to understand derivatives and ... Here is a plot of that function along with some points on the curve. In this case, the average velocity (the slope) going from point 2 to 3 is different ...