Directional Derivatives and the Gradient Vector
The directional derivative of f at in the direction of unit vector is
if this limit exists.
If f is a function of two variables x and y, then the gradient of f is the vector function defined by
Hence,
More generally (in higher dimensions),
If f is a differentiable function of x and y, then f has a directional derivative in the direction of any unit vector and
Suppose f is a differentiable multivariate function. The maximum value of the directional derivative is and it occurs when has the same direction as the gradient vector .
The gradient vector will be very important in optimization problems.
We shouldn't be a stick in the mud about the orientation of our axes: why x and y, and not some other pair of directions which are mutually perpendicular? Perhaps we are interested in the slope of the surface along some direction other than x or y: hence the idea behind directional derivatives. At a given point at which a function is differentiable, one natural choice for two directions might be the direction in which the function is increasing fastest, and the direction perpendicular to this.
Because the function is essentially planer at a differentiable point, it might not be surprising that the partial derivative of f wrt (with respect to) a given direction other than x and y is some combination of the partial derivatives of f wrt x and y.
The gradient vector is the vector with these two partials (wrt x and y) weighting the respective axes unit vectors and , and is used to give a convenient inner product definition of the directional derivative. It points in the direction of fastest increase of the function, which simultaneously means that it is perpendicular to level curves.
The importance of this vector cannot be overestimated: it is used for minimization, as the derivative is used in the univariate case. It is via the gradient that we are able to minimize multivariate functions.