MODEL LEARNING PATH
Enformer
Enformer compresses a long DNA window with convolution and learned pooling, then uses relative attention to connect distant sequence features. Species-specific heads predict central assay profiles.
Loading the interactive page…
A landmark in bringing long-range attention into sequence-to-regulatory-function modeling, with Borzoi explicitly building on its core architecture. Its original benchmark gains should be read alongside a 2023 independent evaluation showing weak prediction of expression variation between personal genomes.
Chapters
Compress sequence context
The convolutional stem and pooling tower convert bases into wider, lower-resolution sequence features.
Track how local sequence features become lower-resolution pooled representations.
Connect distant regions
Transformer blocks use signed genomic-distance features alongside content attention.
Combine sequence-content attention with signed genomic-distance information.
Predict the central profiles
Crop the contextual representation and apply the species-specific output heads.
Connect the central contextual representation to assay-specific output profiles.
Learn from assay counts
Count-valued labels supervise the predicted assay profiles.
Apply the masked count objective to valid observed assay values.
All exercises in this guide
- Dense 1-D dilated convolution
- Masked Sum
- Masked Mean
- Layer Normalization
- BatchNorm training, evaluation and functional running statistics
- Sigmoid GELU Approximation
- Affine Projection
- Masked Softmax
- Window Attention Pooling
- Learned softmax pooling for sequence windows and pair rows
- Multi-head attention with masks and additive bias
- Enformer genomic relative attention
- ReLU
- Feedforward Block
- Activations and gated feed-forward transitions
- LayerNorm and RMSNorm from reductions
- Affine projection and embedding lookup
- Reverse-complement sequences and align strand-aware outputs
- Crop, orient and average genomic predictions
- Edit and crop an alternate-allele DNA window
- Stable masked Poisson objective for genomic counts