A very quick guide to Linear Algebras-I
General:
(1) things we are interested in: matrix and system of linear equations
(2) key points: operations satisfying linear property: 
%20%3Dc_1%20f(x_1)%2Bc_2f(x_2)%2C%20%5Cforall%20x_1%2C%20x_2%20%5Cin%20X%2C%20%5Cforall%20c_1%2C%20c_2%20%5Cin%20%5Cmathbb%7BR%7D#card=math&code=f%3A%20X%20%5Cto%20Y%2C%20f%28c_1x_1%2Bc_2x_2%29%20%3Dc_1%20f%28x_1%29%2Bc_2f%28x_2%29%2C%20%5Cforall%20x_1%2C%20x_2%20%5Cin%20X%2C%20%5Cforall%20c_1%2C%20c_2%20%5Cin%20%5Cmathbb%7BR%7D&id=QLdTS), provided that it makes sense to define 
for 
.
(3) Other operations and properties (but we will not discuss).
A Real (or Complex if you like) Matrix, is an 
array.
Example:
.
: 
, 
: 
—— the numbers of Row and Column sometimes are explicitly written out.
Matrix entries (or elements).
Definition: (1) As matrices, 
if they have same numbers of rows and columns, and they are equal elementwise.
. 
as matrices.
(2) Let 
be two matrices of the same size (say 
). Define their sum (difference) in the obvious way: the sum 
also has size 
, and its entries are obtained as the sum of the entries of 
and 
correspondingly.
Example: 
.
The difference is defined similarly.
Proposition: Two matrices 
iff 
is the zero matrix. (all entries are 0).
Remark: there are zero matrices in every size. But we usually call them the zero matrix in general. When we must emphasis the size of a zero matrix, we can write 
.
(3) (The first key point). Now suppose 
%7Bm%20%5Ctimes%20n%7D%2C%20B%3D%20(b%7Bij%7D)%7Bp%20%5Ctimes%20q%7D#card=math&code=A%3D%28a%7Bij%7D%29%7Bm%20%5Ctimes%20n%7D%2C%20B%3D%20%28b%7Bij%7D%29%7Bp%20%5Ctimes%20q%7D&id=o8AdT), where 
are positive integers. For example, let us take 
as above. Then .
One can define the product 
only if 
. When it is the case, the product 
#card=math&code=C%3DAB%20%3D%20%28c_%7Bij%7D%29&id=I6dJ9) has size 
, and is determined by the following formula:
.
That is, the 
#card=math&code=%28i%2C%20j%29&id=usGV1)-entry in the product 
, is given by the “product” of the 
th row entries of 
and the 
th column entries of 
.
Example:
We can multiply 
. But 
is not well-defined. This shows that matrix multiplication is NOT commutative.
.
(4) Scalar multiplication of matrices: Let #card=math&code=A%3D%28a%7Bij%7D%29&id=wurVt) be a matrix, and 
a scalar, the scalar multiplication 
is a matrix whose entries are #card=math&code=%28%5Clambda%20a_%7Bij%7D%29&id=fNORy) .
Example: 
,
M_2%20%3D%20M_1%20%3D%20%5Cbegin%7Bpmatrix%7D%201%20%26%200%20%263%2F2%20%5C%5C%200%20%26%202%20%26%200%20%5Cend%7Bpmatrix%7D#card=math&code=%281%2F2%29M_2%20%3D%20M_1%20%3D%20%5Cbegin%7Bpmatrix%7D%201%20%26%200%20%263%2F2%20%5C%5C%200%20%26%202%20%26%200%20%5Cend%7Bpmatrix%7D&id=bG74b).
Remarks: We have the following rules to compute matrices addition and multiplications:
(1) 
suppose the equation makes sense.
(2) If we have an arbitrary real number 
, then 
%20%3D%20%5Calpha%20M%20%2B%20%5Calpha%20N#card=math&code=%5Calpha%28M%2BN%29%20%3D%20%5Calpha%20M%20%2B%20%5Calpha%20N&id=EcJ25), where 
is defined as 
multiplying all entries of 
.
(3) Suppose 
and 
are matrices such that 
is well-defined (which means the number of columns of 
is equal to the number of rows of 
). We have 
N%20%3D%20M%20(%5Calpha%20N)%20%3D%20%5Calpha%20(MN)#card=math&code=%28%5Calpha%20M%29N%20%3D%20M%20%28%5Calpha%20N%29%20%3D%20%5Calpha%20%28MN%29&id=powF2) .
(4) Warning: For two matrices, if 
is well-defined and 
, this does not imply that 
or 
. Example: 
.
Now we look at an important application of matrices. We use it to solve systems of linear equations. A system of 
linear equations in 
variables is given as follows:
here the variables are 
. “Linear” means that all equations are polynomials in 
of degree 
.
The algorithm is called “the Gaussian elimination”.
Write down two matrices from the above system:
the coefficient matrix 
,
and
the augmented matrix 
.
Let us write 
, and 
. Moreover, we let 
, then the original system can be written in matrix notations as
The Gaussian elimination algorithm is based on the very imporant
Elementary Row Transformations
for matrices. There are 3 kinds of elementary row transformations.
(i) switching two rows;
(ii) multiplying a row by a (non-zero) constant;
(iii) a row multiplying a non-zero constant and then add to another row. (but the first mentioned row is not changed).
Examples: Let 
(
is called the identity matrix of dimension 
.)
Now let us apply these elementary row transformations to 
:
(i) We switch row 1 and row 3 (denoted 
): 
.
(ii) We multiply the first row by 4 (denoted 
): 
.
(iii) 
will result in 
.
The Gaussian elimination method:
- Starting with the augmented matrix
. - Using elementary row transformations, to change
into an echelon form.
echelon
%20%3D%5Cbegin%7Bpmatrix%7D%201%20%26%200%20%26%200%20%26%20%5Cvdots%20%26%201%5C%5C%200%20%26%201%20%26%20%200%20%26%5Cvdots%20%26%20%200%5C%5C%200%20%26%200%20%26%201%20%26%20%5Cvdots%20%26%200%5Cend%7Bpmatrix%7D#card=math&code=%28A%29%20%3D%5Cbegin%7Bpmatrix%7D%201%20%26%200%20%26%200%20%26%20%5Cvdots%20%26%201%5C%5C%200%20%26%201%20%26%20%200%20%26%5Cvdots%20%26%20%200%5C%5C%200%20%26%200%20%26%201%20%26%20%5Cvdots%20%26%200%5Cend%7Bpmatrix%7D&id=CdUpQ). - Now determine whether there is a solution.
The criterion is: the number of non-zero rows in the echelon(A) = the number of non-zero rows in echelon(C).
If this holds, then there are solutions. Otherwise, no solution. - Now read off the solution:
, which means 
.
Echelon form:
Rule 1: All-zero-rows must put below non-zero rows. (suppose there are all-zero-rows)
Rule 2: For all 
, the first non-zero element in the 
-th row must on the right to the first non-zero element in the 
#card=math&code=%28k-1%29&id=WippT)-th row.
A matrix satisfying these two rules is called an echelon form.
Example: For the matrix 
, one of its echelon form is 
.
r_2%7D#card=math&code=%5Cxrightarrow%7B%281%2F38%29r_2%7D&id=gptgw) 
r_1%7D#card=math&code=%5Cxrightarrow%7B%28-1%2F2%29r_1%7D&id=GUejD) 
r_2%7D#card=math&code=%5Cxrightarrow%7Br_1%20%2B%20%283%2F2%29r_2%7D&id=vq1f3) 
.
Conclusion: Echelon form of a matrix is not necessarily unique.
An enhancement to the (row) Echelon form, called the “Reduced Row Echelon Form”, RREF , is defined as follows:
Rule 1 (for RREF): It is already an echelon form.
Rule 2 (for RREF): To all (non-zero) rows, the first non-zero element must be 
.
We call these “1” pivots.
Rule 3 (for RREF): In the columns where the pivots “1” in Rule 2 are in, all other entries in these columns must be zero. (The pivoting columns must only have a pivot “1” and other entries 0).
Proposition: RREF of a matrix is uniquely determined by this matrix. (More precisely, given an arbitrary matrix 
, one can write down a matrix in RREF attached to 
.)
Example: Using RREF to solve system of linear equations. The augmented matrix is
its RREF is
The pivots are in the first two columns.
There are two non-zero rows in the augmented matrix, as well as in the coefficient matrix.
Conclusion: there must be a solution!
Q: How to write down the solution(s)?
A: Using the pivots in the RREF!
(1) Locate the columns 
where the pivots lie in. We then choose 
as main variables, and
(2) we use other variables to express the main varibles.
Then we can write down all solutions!
Here: 
are main variables, 
are free variables.
x_3%20-%20(3%2F4)x_4%20%2B%205%2F4#card=math&code=x_1%20%3D%20%283%2F2%29x_3%20-%20%283%2F4%29x_4%20%2B%205%2F4&id=E0Pjk)
x_3%20%2B%20(7%2F4)x_4%20-%201%2F4#card=math&code=x_2%20%3D%20%283%2F2%29x_3%20%2B%20%287%2F4%29x_4%20-%201%2F4&id=hP9TM)
So that when 
are determined, so are 
.
An example without solution
Explanation: The last row says that 
.
This is an inconsistent equation. Thus the original system has no solution.
Remark: If the constant column 
of the system of linear equations is identically zero, the system certainly has a trivial solution: 
. (the zero solution). So in this case, we are interested in the question that “are there any other solution”? Moreover, if there is a nonzero solution 
(say), then any multiple of 
is also a solution: let 
be an arbitrary constant, then
%20%3D%20%5Clambda%20(Cv)%3D%5Clambda%20%5Cmathbf%7B0%7D%20%3D%20%5Cmathbf%7B0%7D.%0A#card=math&code=C%28%5Clambda%20v%29%20%3D%20%5Clambda%20%28Cv%29%3D%5Clambda%20%5Cmathbf%7B0%7D%20%3D%20%5Cmathbf%7B0%7D.%0A&id=Yta1I)
The algorithm to this question is still the Gaussian elimination. It is easy to see that the existence of free variables will give numerous solution.
The ultimate result is:
Theorem: Denote the augmented matrix of a system of linear equations 
by 
. Suppose that 
is an 
matrix 
and has 
#card=math&code=r%28C%29&id=DBV3R) pivots in its RREF. Suppose 
has 
#card=math&code=r%28A%29&id=JKgNI) pivots in its RREF. Then the system has
- no solution if and only if %20%3C%20r(A)#card=math&code=r%28C%29%20%3C%20r%28A%29&id=AuUs6);
- a unique solution if and only if %3Dr(A)%3Dn#card=math&code=r%28C%29%3Dr%28A%29%3Dn&id=R9fcV);
- more than one solution if and only if %3Dr(A)%3Cn#card=math&code=r%28C%29%3Dr%28A%29%3Cn&id=lwFDz).
Let us illustrate this theorem by using the following toy example: we consider the equation 
, where 
are constants. Then 
. Here 
.
- If %3Cr(A)#card=math&code=r%28C%29%3Cr%28A%29&id=tem59), this can only happend if
but 
. In this case, the equation 
becomes 
, which is inconsistent, and hence no solution. - If %3Dr(A)%3D1#card=math&code=r%28C%29%3Dr%28A%29%3D1&id=ojTAq), this can only happend if
(Note that if 
, then whehter 
is zero or not, is not a matter). In this case, the equation 
has a unique solution 
. - If %3Dr(A)%3D0#card=math&code=r%28C%29%3Dr%28A%29%3D0&id=Up051), this can only happen if
. The equation becomes 
. Clearly, any number is a solution. So numerous solutions.
