Skip to the content of the web site.

Linear Systems of Equations

What does it actually mean to solve the system Mx = b?

For some students, it's entirely likely that after doing all the math in your linear algebra course, you forgot the original reason why you defined these matrices, and why you're learning Gaussian elimination.

One Dimension

For the one dimensional case, one solves mx = b for x. In this case, the solution is simple: x = b/m.

Two Dimensions

Let's write out the matrix vector equaiton Mx = b. Given the matrix

and the vectors x = (x1, x2)T and b = (b1, b2)T, then expanding we get:

It's important here to remember that all the values mi, j and bi are given, a concrete example, is useful.

Example of a 2×2 System of Linear Equations

What does Mx = b mean when

?

All this is, in disguise, is

4x1 + 2x2 = 3
3x1 + 5x2 = 4

which is equivalent to

4x1 + 2x2 − 3 = 0
3x1 + 5x2 − 4 = 0

However, note that the left hand side describes a plane. In Figure 1, we show f1(x) = 4x1 + 2x2 − 3, and in Figure 2, we show f2(x) = 3x1 + 5x2 − 4.

Figure 1. The function f1(x) = 4x1 + 2x2 − 3.

Figure 2. The function f2(x) = 3x1 + 5x2 − 4.

The first function, f1(x) is zero on the line x1 = 3/4 − 2/4x2, while the second function, f2(x) is zero on the line x1 = 4/3 − 5/3x2. What it means to solve a linear system is that we are finding a value of x = (x1, x2)T such that both f1(x) = 0 and f2(x) = 0.

In this example, there is one unqiue point, and that point may be seen in Figure 3.

Figure 3. Both function f1(x) and f2(x).

These two functions are simultaneously zero at only one point, seen in Figure 3 as the point where the grey (z = 0), red, and blue planes intersect. From the linear algebra you learned, you could solve this system to find that the solution is x = (0.5, 0.5)T.

Thus, solving a system of n linear equations is no more than finding simultaneous roots of n different linear functions. This is why we can use such techniques as when we perform Newton's method in n dimensions: we convert a system of n nonlinear functions into a simplified problem with n linear functions, find the root of the linear functions and show that, under the appropriate conditions, the solution to the linear functions is a good approximation to the nonlinear problem.

Why are we Solving Systems of Linear Equations?

The most obvious example comes in examples such as trying to solve circuit in Figure 4. Kirchhoff's voltage laws tells us that the sum of the voltages in each of the loops is zero.

Figure 4. A simple circuit.

In the first loop, we get 7 V − 1 (i1 + i2) − 2 i1 = 0, and in the second, we get 2 i1 − 4 i2 = 0. Solving these yeilds the values i1 = 2 A and i2 = 1 A.