We don’t usually think of linear algebra being compatible with derivatives, but it very useful to be able to know the derivative of certain elements in order to adjust them—basically using gradient-descent.
We might therefore ask ourselves questions like, how do I compute the derivative of a scalar product relative to one of the vector elements? Of a matrix-vector product? Through a matrix inverse? Through a product of matrices? Let’s answer the first two questions for now.
2 Comments | 
algorithms, machine learning, Mathematics | Tagged: Derivative, dot product, Machine Learning, Matrix, scalar product, Vector | 
Permalink
Posted by Steven Pigeon
A Taylor series for a function 
or
where 
Have you ever wondered where the coefficients in a Taylor series come from? Well, let’s see!
Leave a Comment » | algorithms, hacks, Mathematics | Tagged: Analytic, Analytical Function, Derivatives, Numerical Approximation, numerical method, Taylor, Taylor Series | Permalink
Posted by Steven Pigeon
Getting good text data for language-model training isn’t as easy as it sounds. First, you have to find a large corpus. Second, you must clean it up!
Leave a Comment » | algorithms, Bash (Shell), hacks | Tagged: ASCII, bash, CRLF, grep, Gutenberg project, ISO-8859-15, sed, tr, UTF-8, Windows codepage | Permalink
Posted by Steven Pigeon
About a week ago, some dude drops on IRC that he’s beat memcpy “by a lot”. That’d be interesting, except that we couldn’t get neither code nor test methodology out of him. But, how hard can making a better memcpy be? Turns out, harder than you think!
If you think this is a typical case of “reinventing the wheel”, I mostly agree with you. But while reinventing will be hard, can improvements be made?
Leave a Comment » | C-plus-plus, hacks | Tagged: assembly language, C, code optimization, Compiler, memcpy, movsb, movsd, movsq, vmovdqu | Permalink
Posted by Steven Pigeon
Harder, Better, Faster, Stronger 