Chances are you’ve seen a neural network diagram like this one with circles and arrows:

Image for post
Figure: A neural network. Source: Medium

While that looks fancy, it’s basically a matrix. More generally, it’s a tensor, but a special kind of tensor which makes it just a matrix.

I was reading this Medium article comparing matrices and tensors which models the following matrix multiplication as the two-layer neural network shown above:

Image for post

This helped me realize that any linear map can be modeled as a neutral network. Consider rotating a vector around the X, then Y, and Z-axes:

Mx=Rotx(15°)My=Roty(15°)Mz=Rotz(15°)V=[1,2,3]V=MxV=[1,1.155,3.415]V=MyV=[1.850,1.155,3.042]V=MzV=[1.488,1.595,3.042] \begin{aligned} M_x &= Rot_x(15°) \\ M_y &= Rot_y(15°) \\ M_z &= Rot_z(15°) \\ V &= [1,2,3] \\ V' &= M_xV \\ &= [1, \pmb{1.155}, \pmb{3.415}] \\ V'' &= M_yV' \\ &= [\pmb{1.850}, 1.155, \pmb{3.042}] \\ V''' &= M_zV'' \\ &= [\pmb{1.488}, \pmb{1.595}, 3.042] \\ \end{aligned}

Here is a neural network visualizing the linear mapping sequence MzMyMxVM_zM_yM_xV:


Figure: Made with draw.io

By tracing the paths through the layers of the neural network below, one can gain an intuition of how each transformation operation contributes to the resulting vector VrotV_{rot}.


Figure: Made with draw.io


Discussion

Please feel free to start a discussion on GitHub.


Main page