Last updated on September 14 at 7:00 PM
Also brush up on your complex numbers if you've studied them before.
Vectors are usually thought of as living in a vector space--the set of all possible values that such a vector could take on. For example, the space of all possible two-dimensional vectors is the two-dimensional Cartesian plane. However, there are many more vector spaces of arbitrary complexity.
A vector space is closed: by taking a linear combination of vectors
in the space you always get another vector in the space.
A set of vector from which you can construct any other vector
in the space is called a spanning set. For example,
by combining some amount of the vectors 
, 
, and 
,
one can construct any other vector in the 2-D plane.
A minimal spanning set (one that contains as few vectors as possible) is called a basis set. In the previous example, one really only needs any two of the three vectors to span the space.
It is useful for the vectors in a basis set to have no components
in the direction of each another (orthogonal). In the previous example,
and 
are an orthogonal basis set while the other pairs aren't.
Finally, since we're spanning the space through linear combinations of vectors, the actual length of the vectors isn't really important--the direction is. To simplify things, we'll usually work with unit-length vectors for our basis sets. Basis sets that are orthogonal and normalized to unit length are called orthonormal.
Notice that a single vector space has many, many, many possible sets of
basis vectors.
The two-dimensional plane can be spanned by 
and 
,
or it can be spanned by 
and 
.
We also aren't strictly limited to Cartesian representation of the space.
For example, the 2-D plane can be spanned by the Cartesian vectors
and 
or by the polar basis vectors 
and 
.
Let 
...
be a set of orthonormal basis vectors for a
vector space, then (as we have seen) any vector v in that space can
be represented by
and
is the jth element of vector v
and
is the jth element of vector 
.
In other words, any vector v is a weighted sum of the basis vectors
,
and the weight 
of each vector is the dot product between the
vector v and the basis vector 
.
In this way, any vector can be transformed from a set of components 
to a set of components 
in a coordinate system whose basis vectors
are 
. This is known as a change of coordinates
or a coordinate transformation.
In the next lecture, we will talk about expressing image functions as their corresponding weights in a transformed space. Such an operation is called a transform.
Suppose that we had a set of vectors that we needed to transform into the
same space.
We could put these vectors into a single matrix V whose columns
are the vectors to be transformed and take the dot product of each column
of V with each row of A to give a set of vectors.
What we get is a new matrix 
whose columns are the transformed vectors:
Notice that this is simply matrix multiplication as you learned it in high school, linear algebra, or elsewhere. To more precisely put it, matrix multiplication is a compact way of transforming a vector into a new coordinate system. The columns of the first matrix are vectors to be transformed (expressed in the old coordinate system) and the rows of the second matrix are the new coordinate basis vectors (also expressed in the old coordinate system).
One can also do the same thing by transposing each matrix and reversing
their order. (Remember from linear algebra: 
.)
You will find that their are two distinct camps that span graphics,
image processing, and computer vision:
the ``row vector'' people and the ``column vector'' people.
It's all a matter of taste really.
Example: Rotation using matrix multiplication
Suppose that you wanted to transform a set of vectors into one
coordinate system and then from there into another expressed in terms
of the new one.
One can just multiply the matrix of initial vectors by the first
transformation matrix and then multiply the result by the second:
(Notice that to add a transformation, we add a matrix on the left
side of the existing matrices.)
However, matrix multiplication is commutative, so
This means that the result of the two transformations is the same
as one single transformation whose matrix is the product of the other
two.
This is called matrix composition: the collapsing of a set of transformations into a single (usually more complex) one. Notice that for a large set of initial vectors it usually more efficient than applying each transformation successively.
Sometimes when you apply a transformation matrix to a vector, it returns exactly the same vector but perhaps shortened or stretched. (We'll ignore the null vector, which always transforms to itself.) Such vectors are special for that matrix and form a sort of natural coordinate system for the transformation.
These vectors satisfy the equation
where A is an arbitrary matrix, v is called an
eigenvector of A, and 
is a corresponding
eigenvalue of A.
(Together, the eigenvalues and eigenvectors are sometimes
called an eigensystem.
Eigenvectors are vectors that are only stretched or contracted by
the matrix; eigenvalues are the amount of that stretch (
)
or contraction (
).
For now, don't worry about how to solve for eigenvalues and eigenvectors of a matrix. You may need to look up how to do this for later in the course, but right now I want you to get a more intuitive feel for what they are.
We define the 
norm (denoted as 
) of a vector as
That is, raise the absolute value of all elements of the vector to the
p power, add them, and take the p-th root of the total.
The 
norm is the square root of the sum of the squares. This
is what we usually think of as length in a Euclidean space.
In fact, if we ever write simply 
in this class, it
will (by default) mean the 
norm.
However, for many vectors this isn't the most useful measure.
The 
norm is simply the sum of the elements. For example,
the 
distance between two pixels p and q is the 
norm of
the vector p-q.
The 
norm can be shown to be the element of the vector with the
largest magnitude.
For example, the 
distance between two pixels p and q is the
norm of the vector p-q.
where i is an integer.
That is, for every integral value of 
there is a
unique corresponding ``value'' of 
.
In other words, a vector is a function from the integers 1...n to
some arbitrary set (the values of the components).
For simplicity, we will only consider here vectors whose components are
real numbers.
But what if i wasn't integral? What if we allowed it to be a real number? We then have a function that maps from the real numbers to the real numbers, something we are very familiar with. In other words, a function is a vector, and we can do anything with a function that we do with a vector.
For example, we can compute the inner (dot) product between two vectors by
point-wise multiplying them together and adding up the total:
We can likewise to the same with functions: point-wise multiply them
together and add up (i.e., integrate) the result:
norm of a function is thus
Likewise we can compute the norm of a function as
If we are taking the 
norm,
we usually square both sides of Eq. 4.11 and simply write
where the function 
(called the Krönecker delta function)
is defined as
This is simply a compact notation for expressing the familiar definition that orthonormal vectors have length one and are orthogonal to each other.
We can likewise say that a set of functions 
is orthonormal if
If the set of orthonormal functions 
spans a
(functional) vector space,
they form a basis set for that space
(just like basis vectors for any other vector space).
Since Eq. 4.1 and Eq. 4.2 hold,
that means we can express any function as a weighted sum of some
set of other functions:
or, if there are an infinite number of functions and we let i be
a continuous rather than discrete subscript,
and the weights 
can be computed by
Notice the symmetry in these equations.
It will appear again later.
A complex number is one of the form a + bi where 
.
Complex numbers can be thought of as vectors in the complex plane
with basis vectors (1,0) and (0,i).
The a coefficient is called the real part of the complex number and the b coefficient is called the imaginary part.
When you add two complex numbers, the real parts and imaginary parts
add independently:
When you multiply two complex numbers, you cross-multiply them like
you would polynomials:
The length is called the magnitude of the number and is
denoted by 
:
The direction is the angle from the real-number axis.
This is called the phase and is denoted by 
:
When you multiply two complex numbers, their magnitudes multiply:
A real number is a complex number with the same magnitude and zero phase, so multiplying a complex number by a real one simply multiplies the magnitude of the complex number by the real number.
If x =a + bi is a complex number, then its complex conjugate
is 
.
The complex conjugate 
has the same magnitude but opposite phase
of x.
When you multiply x by 
, you get the real number equation
to 
.
This follows from Eq 4.24, but also from
simple algebra:
Going back to our earlier discussion of
functions as vectors,
computing the inner product of two complex functions involves multiplying
one of the functions not by the other but by the complex conjugate of the
other. In this way, the inner product of a function with itself
(the 
norm squared) is a real number.
Eq. 4.18 can be written for complex numbers as
where 
is the complex conjugate of 
.
Notice that Eq. 4.18 is a special case of
Eq. 4.26 because the complex conjugate of
a real number is itself (i.e., the imaginary part is zero, so inverting
its sign has no effect.).
Euler's formula states that 
.
In other words, 
is the vector with magnitude one and phase
.
As part of your homework assignment, you'll verify that magnitudes multiply and phases add by using Euler's formula.
The Dirac delta function is defined as
and
When one multiplies a delta function by a constant,
and
When one multiplies a ``normal'' function 
by a delta function,
one gets
This is another delta function whose ``height'' has been multiplied
by 
, similar to
a delta function multiplied by a constant.
When this is integrated,
By extension, one can shift the delta function with similar results:
This last equation
is called the Sifting Property: one can extract
(sample) the value of a function 
at point a by integrating the
product of the function with a delta function shifted by a.
As one might guess, our discussion of discrete sampling will involve delta functions.
Many people don't differentiate between the Krönecker and Dirac delta functions in casual communication--the context of the usage generally makes it clear.
be defined as the vector
One applies this unary operator by applying each element individually
to some differentiable function:
The length of the operator is assumed to be the number of variables
in the domain of the function to which it is being applied.
The result of the application of the operator is a vector of the same length.
Geometrically, 
is the direction of steepest ascent of the function 
and its length is the ``steepness'' of this ascent.
This direction (which may be different for every point) is called the
gradient of 
.
For this reason, 
is often called grad.
Because this is a vector, you can do with it anything you normally do
with vectors.
Specifically, one can take the inner product of the operator with itself:
This is also an operator, except that this returns a scalar (like the dot
product of any two vectors does).
So,
This operator 
is called the Laplacian.
vector 
to give a 
matrix 
.
Multiplying 
by 
gives an 
matrix of second derivatives:
For example, the matrix of second derivatives for a function of two
variables is
Notice that this matrix is symmetric because
.
By the way, what do you think 
is?
There are other operators in vector calculus
(e.g., 
),
but these are the three we'll use in this course.
© Bryan S. Morse, 1995
