by incrudible on 2/6/25, 2:09 PM
Category theory always reminds me of this quip by Feynman:
https://www.youtube.com/watch?v=B-eh2SD54fM
When I hear people talking in the jargon of category theory, I do not understand what they say, but I have a suspicion that it is something rather mundane that I would understand if they were using the more specific terms for the given context. I understand the appeal of generalization from a mathematical perspective, but from a practical programming perspective, I fail to understand the value proposition.
by abeppu on 2/5/25, 4:54 PM
So, learning about the categorical structure is interesting, but is there a specific advantage to seeing these concepts directly informing the implementation vs as a post-hoc way of explaining what autodiff is doing? E.g. Tensorflow is creating and transforming graphs of computation. Despite being written before most of the categorical work cited was done, isn't it doing the "same" thing, but we just wouldn't find names or comments in the code that closely align with the categorical work?
by tripplyons on 2/5/25, 5:37 PM
How does this compare to the XLA's ability to compile full training steps from JAX?
by catgary on 2/5/25, 6:22 PM
This is a nice project, but “You only linearize once” is more-or-less the type theory version of “Reverse Derivative Categories”, so JAX really does this already.
by drumnerd on 2/6/25, 7:37 AM
I think about cones all the time when doing machine learning. If I have an object O that can be mapped to A and B I can learn a function from A to B or B to A if I can generate Os. That’s my view of self supervised learning.
by bguberfain on 2/5/25, 6:14 PM
by cyanydeez on 2/5/25, 9:31 PM
i always find it fascinating when projects like these can seem to find a proper overview for the AI curious non-PhD students.