Scribd
Upload
10 views27 pages

Chapter 4 Vector Calculus

This document discusses gradient computation in machine learning. It covers differentiation of univariate functions, partial differentiation and gradients, gradients of vector-valued functions, and gradients of matrices. The key topics include computing directional derivatives and partial derivatives to determine the gradient, which represents the direction of steepest ascent. Gradients are generalized from real-valued to vector-valued functions by taking partials of each component function. Gradients of matrices with respect to vectors result in multidimensional tensors whose entries are the partial derivatives.

Uploaded by

Tuan Anh Tran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views27 pages

Chapter 4 Vector Calculus

This document discusses gradient computation in machine learning. It covers differentiation of univariate functions, partial differentiation and gradients, gradients of vector-valued functions, and gradients of matrices. The key topics include computing directional derivatives and partial derivatives to determine the gradient, which represents the direction of steepest ascent. Gradients are generalized from real-valued to vector-valued functions by taking partials of each component function. Gradients of matrices with respect to vectors result in multidimensional tensors whose entries are the partial derivatives.

Uploaded by

Tuan Anh Tran
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 27

In this chapter, we will discuss how to compute

gradients of functions, which is often essential to


facilitate learning in machine learning models since
the gradient points in the direction of steepest ascent.
Therefore, vector calculus is one of the fundamental
mathematical tools we need in machine learning.
4.1 Differentiation of Univariate Functions

4.2 Partial Differentiation and Gradients

4.3 Gradients of Vector-Valued Functions

4.4 Gradients of Matrices


Lesson 4.1
Differentiation of Univariate Functions


I. Definitions

II. Taylor Series


Lesson 4.2: Partial Differentiation and
Gradients
I. Definitions

Partial Differentiation and Gradients

Partial Differentiation and Gradients

II. Basic Rules of Partial Differentiation

II. Basic Rules of Partial Differentiation


II. Basic Rules of Partial Differentiation


II. Basic Rules of Partial Differentiation



Lesson 4.3:
Gradients of Vector-Valued Functions

Thus far, we discussed



partial derivatives and gradients of
functions : ℝ → ℝ mapping to the real numbers. In the
following, we will generalize the concept of the gradient to vector-
valued functions (vector fields) : ℝ → ℝ , where ≥ 1 and
> 1.
Gradients of Vector-Valued Functions

Writing the vector-valued function in this way allows us to view a
vector valued function : ℝ → ℝ as a vector of functions
, ,…, , : ℝ → ℝ that map onto ℝ.
Gradients of Vector-Valued Functions


Therefore, we obtain the gradient of : ℝ → ℝ with respect to
∈ ℝ by collecting these partial derivatives:
Gradients of Vector-Valued Functions


Gradients of Vector-Valued Functions


Gradients of Vector-Valued Functions


Gradients of Vector-Valued Functions


EXERCISES

 Exercises: Ex 5.1 – 5.5 (P 170-171)
Lesson 4.4
Gradients of Matrices

We will encounter situations where we need to take gradients of
matrices with respect to vectors (or other matrices), which
results in a multidimensional tensor. We can think of this tensor
as a multidimensional array that collects partial derivatives. For
example, if we compute the gradient of an × matrix A with
respect to a × matrix B, the resulting Jacobian would be
( × ) × ( × ), i.e., a four-dimensional tensor J, whose
entries are given as = / .
Lesson 4.4
Gradients of Matrices

Lesson 4.4
Gradients of Matrices

Gradients of Matrices


Gradients of Matrices


Gradients of Matrices

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy