This post on the MIT CSAIL blog really upset me today. It's a fascinating and potentially important finding that throws doubt into many assumptions about how deep learning works, but it's claims are hyperbolic and anthropomorphizing to the point of being unscientific.
Here they show that a particular LLM could do several "neat tricks" in interpreting, modifying, and generating the weights for another neural network. This result is extraordinary, because these weights are notoriously unintelligable, and generally can only be set by arduous, expensive training processes. I'm genuinely shocked that this works (assuming it replicates). This could have profound consequences for all sorts of things, from efficiency, to development, and interpretability.
What they did not do is show that LLMs can understand weights, read and write them, or optimize them using reasoning.
#science #llm #ai
(1/2)
