§ feed · storyline
Multimodal neurons in artificial neural networks
OpenAI discovers multimodal neurons in CLIP that respond to concepts presented literally, symbolically, or conceptually, offering insight into how the model classifies unusual visual renditions.
We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP’s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn.
§ sources1 publication · timeline below