Similar Links:

• DALL·E 1: Creating Images from Text: We’ve trained a neural network called DALL·E that creates images from text captions for a wide range of concepts expressible in natural language

• The Scaling Hypothesis

• Gato: A Generalist Agent

• Decision Transformer: Reinforcement Learning via Sequence Modeling

• What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

• Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization

• Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition

• Critical Data Size of Language Models from a Grokking Perspective

• Grokking: Generalization Beyond Overfitting On Small Algorithmic Datasets

• Mimetic Initialization of Self-Attention Layers

• Hopfield Networks is All You Need

• CT0: Fine-tuned Language Models are Continual Learners

• Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?

• Why are Sensitive Functions Hard for Transformers?

• Learning Transformer Programs

• Perceiver IO: A General Architecture for Structured Inputs & Outputs

• Absolute Unit NNs: Regression-Based MLPs for Everything

• Towards Better LLM Creative Writing

• Search: GS; Google; site