Chances are you’ve seen a neural network diagram like this one with circles and arrows:

Image for post
Figure: A neural network. Source: Medium

While that looks fancy, it’s basically a matrix. More generally, it’s a tensor, but a special kind of tensor which makes it just a matrix.

I was reading this Medium article comparing matrices and tensors which models the following matrix multiplication as the two-layer neural network shown above:

Image for post

This helped me realize that any linear map can be modeled as a neutral network. Consider rotating a vector around the X, then Y, and Z-axes:

Mx=Rotx(15°)My=Roty(15°)Mz=Rotz(15°)V=[1,2,3]V=MxV=[1,1.155,3.415]V=MyV=[1.850,1.155,3.042]V=MzV=[1.488,1.595,3.042] \begin{aligned} M_x &= Rot_x(15°) \\ M_y &= Rot_y(15°) \\ M_z &= Rot_z(15°) \\ V &= [1,2,3] \\ V' &= M_xV \\ &= [1, \pmb{1.155}, \pmb{3.415}] \\ V'' &= M_yV' \\ &= [\pmb{1.850}, 1.155, \pmb{3.042}] \\ V''' &= M_zV'' \\ &= [\pmb{1.488}, \pmb{1.595}, 3.042] \\ \end{aligned}

Here is a neural network visualizing the linear mapping sequence MzMyMxVM_zM_yM_xV:

Figure: Made with

By tracing the paths through the layers of the neural network below, one can gain an intuition of how each transformation operation contributes to the resulting vector VrotV_{rot}.

Figure: Made with


Please feel free to start a discussion on GitHub.

Main page