Chances are you’ve seen a neural network diagram like this one with circles and arrows:

Figure: A neural network. Source: Medium

While that looks fancy, it’s basically a matrix. More generally, it’s a tensor, but a special kind of tensor which makes it just a matrix.

I was reading this Medium article comparing matrices and tensors which models the following matrix multiplication as the two-layer neural network shown above:

This helped me realize that any linear map can be modeled as a neutral network. Consider rotating a vector around the X, then Y, and Z-axes:

\begin{aligned} M_x &= Rot_x(15°) \\ M_y &= Rot_y(15°) \\ M_z &= Rot_z(15°) \\ V &= [1,2,3] \\ V' &= M_xV \\ &= [1, \pmb{1.155}, \pmb{3.415}] \\ V'' &= M_yV' \\ &= [\pmb{1.850}, 1.155, \pmb{3.042}] \\ V''' &= M_zV'' \\ &= [\pmb{1.488}, \pmb{1.595}, 3.042] \\ \end{aligned}

Here is a neural network visualizing the linear mapping sequence $M_zM_yM_xV$:

By tracing the paths through the layers of the neural network below, one can gain an intuition of how each transformation operation contributes to the resulting vector $V_{rot}$.