Chances are you’ve seen a neural network diagram like this one with circles and arrows:

*Figure: A neural network. Source:* Medium

While that looks fancy, it’s basically a matrix. More generally, it’s a tensor, but a special kind of tensor which makes it just a matrix.

I was reading this Medium article comparing matrices and tensors which models the following matrix multiplication as the two-layer neural network shown above:

This helped me realize that any linear map can be modeled as a neutral network. Consider rotating a vector around the X, then Y, and Z-axes:

$\begin{aligned}
M_x &= Rot_x(15°) \\
M_y &= Rot_y(15°) \\
M_z &= Rot_z(15°) \\
V &= [1,2,3] \\
V' &= M_xV \\
&= [1, \pmb{1.155}, \pmb{3.415}] \\
V'' &= M_yV' \\
&= [\pmb{1.850}, 1.155, \pmb{3.042}] \\
V''' &= M_zV'' \\
&= [\pmb{1.488}, \pmb{1.595}, 3.042] \\
\end{aligned}$

Here is a neural network visualizing the linear mapping sequence $M_zM_yM_xV$:

*Figure: Made with draw.io*

By tracing the paths through the layers of the neural network below, one can gain an intuition of how each transformation operation contributes to the resulting vector $V_{rot}$.

*Figure: Made with draw.io*

### Discussion

Please feel free to start a discussion on GitHub.

Main page