The Problem
Standard transformers process sequences. Proteins require reasoning about pairwise relationships between residues simultaneously with evolutionary signals across many homologous sequences — a 2D problem, not 1D. No standard transformer block handles both.
The Key Insight
Treat protein structure prediction as a graph inference problem. Maintain two representations jointly: a Multiple Sequence Alignment (MSA) matrix capturing evolutionary correlations, and a pair representation matrix capturing pairwise spatial relationships. Update both simultaneously using attention — with geometric constraints (triangle inequality) baked into the update operations.
What’s Clever
Three novel attention patterns: row attention (within each sequence), column attention (across all sequences at one position), and triangle self-attention (updating pairwise distances using triangles of three residues, enforcing geometric consistency). The outer product from the MSA into the pair representation continuously pushes evolutionary signal into spatial predictions.
Key Sources
- alphafold-2-protein-structure-prediction — introduces Evoformer as the core architectural innovation in AlphaFold2
Related Concepts
- attention — row/column/triangle attention are Evoformer’s core operations
- transformer — Evoformer is a transformer variant adapted for graph inference on protein sequences
- protein-structure — the application domain Evoformer was designed for