Skip to content

BioTorchby Chris Hayduk

Problems Models Review Profile

Sign in Create free account

RAW PYTORCH · TENSOR PRIMITIVES

THE CURRICULUM

Problems

Learn the mathematical building blocks of biological AI. One function at a time, from raw PyTorch tensors.

Loading the interactive page…

Foundations

Masked Sum easy
Masked Mean easy
Weighted Masked Mean easy
Masked sums, means and weighted reductions medium
Masked Softmax easy
Masked Log Softmax easy
Masked Log-Sum-Exp easy
Stable logsumexp, log-softmax and masked softmax medium
Affine Projection easy
Token Embedding Lookup easy
Affine projection and embedding lookup easy
Layer Normalization easy
RMS Normalization easy
LayerNorm and RMSNorm from reductions medium
ReLU easy
Exact GELU easy
SiLU easy
Tanh GELU Approximation easy
Sigmoid GELU Approximation easy
Feedforward Block easy
Gated Feedforward Block medium
Activations and gated feed-forward transitions medium
Multi-head attention with masks and additive bias medium
Categorical Cross-Entropy medium
Binary Cross-Entropy with Logits easy
Stable categorical and binary classification losses medium
Stable vector norms, normalization, dot and cross features easy
Ragged segment sum and mean easy
Masked Layer Normalization easy
AlphaFold 3 Adaptive Normalization medium
Conditioned RMS Normalization medium
Adaptive LayerNorm and RMS condition modulation medium
Categorical and numeric molecular feature encoding medium
BatchNorm training, evaluation and functional running statistics medium

Geometry and structural losses

Masked pairwise distances and hard distance-bin targets easy
Batched rigid transforms medium
Right-handed local frame from three points medium
Express Points in Local Frames easy
Reduce Local Frame Point Errors medium
Frame-aligned point error (FAPE) hard
Local distance difference test (lDDT) medium
Weighted proper rigid alignment hard
Geometric confidence targets and observation masks hard
Rigid Permutation Alignment hard
Ambiguous Atom Renaming medium
Symmetry assignment: rigid fits and ambiguous atom renaming hard
Peptide Geometry Violation hard
Within-Residue Distance Violation medium
Between-Residue Clash Loss medium
Polymer Clash Summary medium
Peptide geometry, steric violations and polymer clashes hard

MSA and pair representations

MSA row attention with pair bias hard
MSA column attention medium
Masked outer-product mean medium
Triangle multiplicative update: incoming and outgoing hard
Triangle attention: starting and ending node hard
MSA pair-weighted averaging medium
Tied Pair Row Attention hard
Tied Pair Column Attention hard
Sequence-Weighted MSA Attention hard
RoseTTAFold Pair Block hard
RFdiffusion tied axial and sequence-weighted attention hard
Reference-coordinate and space-ID rotary features medium
Diagonal recurrent-state discretization and update medium
Global-column MSA attention with shared keys and values hard
MSA cluster profiles and deletion features hard
Row Attention Pooling easy
Chain-Aware Relative Position Buckets easy
ESMFold Relative Position Buckets easy
AlphaFold 3 Relative Position Buckets medium
ESMFold 2 Relative Position Buckets medium
Chain, entity and token-relative position buckets medium
ESMFold Single-to-Pair Features medium
ESMFold 2 Single-to-Pair Features hard
ESM3 Single-to-Pair Features medium
Reference Atom Pair Features hard
Noisy Atom Pair Features hard
Trunk-Conditioned Atom Pair Features hard
Single-to-pair features and atom reference conditioning hard
Conditioned pair-bias attention and gated transition block hard

Structure module, templates and recycling

Invariant point attention (IPA) expert
AlphaFold 2 Template Pair Features hard
AlphaFold 3 Template Pair Features hard
RFdiffusion Template Pair Features expert
Template geometry and pair features expert
Template Pair Cross-Attention medium
Template Pair Mean easy
Template-axis attention and mean aggregation hard
AlphaFold 2 Recycling Update hard
ESMFold Recycling Update medium
AlphaFold 3 Recycling Update medium
Clean Template Recycling easy
Recycling representations and clean-template state hard
Learned torsions and hierarchical atom reconstruction expert
Local Quaternion Frame Update medium
Global Quaternion Frame Update medium
Normalized-quaternion backbone updates hard

All-atom folding

Atom–Token Broadcast and Pool medium
Atom Layout Gather easy
Atom-layout gather and masked atom-to-token pooling medium
AlphaFold 3 Atom Attention Encoder hard
AlphaFold 3 Atom Attention Decoder hard
ESMFold 2 Atom Attention Encoder hard
ESMFold 2 Atom Attention Decoder hard
Local atom attention, encoder pooling and decoder skip/readout hard

Protein sequence and inverse folding

Rotary positions for queries and keys medium
Tied vocabulary head and masked/causal token losses medium
ProteinMPNN node-and-edge message-passing layer hard
Arbitrary-order autoregressive graph mask medium
Nearest structural-code quantization easy
Attention-derived protein contact prediction medium
Protein backbone edge geometry and 25 RBF blocks medium
Sequence-conditioned ProteinMPNN decoder stack hard
Constrained categorical sampling with tied positions hard
Iterative masked-token reveal with fixed prompts medium
Autoregressive KV-cache append and causal attention hard
Weighted Layer Mixture easy
Projected Layer Mixture medium
Learned language-model layer mixtures easy
ESM3 multitrack embedding and confidence fusion medium
ESM-2 token-dropout scaling easy

Molecular graphs, equivariance and docking

Masked batched kNN graphs and neighbor gather medium
Geometric vector perceptron (GVP) hard
Distance basis expansion with smooth finite support easy
One EGNN scalar-feature and coordinate update medium
Differentiate energy into forces and train through forces hard
Project coordinates and Gaussian noise to zero center of gravity medium
Geometric direction-and-distance attention hard
Irrep-safe batch and instance normalization hard
Periodic Radius Graph medium
RFdiffusion Full Graph easy
RFdiffusion Refinement Graph medium
Typed radius, bond-prefix and sequence-neighbor graph assembly medium
Origin-safe Bessel channels with a compact polynomial cutoff medium
Real spherical harmonics from Cartesian polynomial tables medium
Real-irrep Clebsch–Gordan paths and channel tensor products hard
Radial-weighted irrep convolution with receiver aggregation hard
Species-conditioned self connections and parity-aware gates hard
Learned irrep query-key attention and tensor values hard
Ligand translation, rotation, bond torsion and confidence heads expert
Directed molecular bonds and reverse-edge bookkeeping medium
Nonbacktracking directed-bond recurrence and atom readout hard

Diffusion and flow matching

EDM coordinate-denoiser preconditioning medium
Variance-Preserving Forward Noising medium
Gaussian Noising and Score easy
Gaussian forward corruption and score targets medium
DDPM posterior from a clean-coordinate prediction medium
Stable SO(3) exponential and logarithm maps hard
SO(3) interpolation and tangent target hard
Masked translational and rotational flow-matching loss hard
One Lie-group Euler step for frame flow hard
Power-schedule EDM initialization and sampler step hard
Shared rigid coordinate augmentation and Haar rotations medium
Wrapped torsion corruption, density and score normalization hard
SO(3) heat-kernel scores and supplied-variate angle sampling hard
Three-space docking initialization and reverse diffusion step hard
Sequential directed-bond torsion rotations hard
RFdiffusion rotational reverse step hard
DiffDock three-space score-matching loss hard
Fourier noise features and learned conditioning projection medium

Genomics and RNA

Reverse-complement sequences and align strand-aware outputs easy
Dense 1-D dilated convolution medium
Stable masked Poisson objective for genomic counts medium
Causal long convolution with FFT medium
Windowed max pooling and coordinate routing easy
Window Attention Pooling medium
Learned softmax pooling for sequence windows and pair rows medium
Enformer genomic relative attention hard
Crop, orient and average genomic predictions medium
Align insertion/deletion predictions to reference coordinates medium
Splice Variant Effects medium
DeepSEA Variant Effects hard
Splice and chromatin variant-effect scores hard
Causal depthwise filtering with a FIR cache medium
Real-imaginary modal filters and recurrent state medium
Complete Evo-style Hyena gated block expert
Edit and crop an alternate-allele DNA window medium

Single-cell and perturbations

Stable negative-binomial log probability medium
Gaussian reparameterization, KL, and a minibatch ELBO medium
Library-size-scaled rates and batch-specific dispersion medium
Keep missing, masked, padded, and truly zero values separate medium
Zero-inflated negative-binomial likelihood medium
Posterior predictive moments and total variance medium
Posterior differential-expression evidence and expected FDP hard
Corpus-normalized gene rank tokenization medium
Delete Gene Tokens medium
Promote Gene Tokens medium
Gene-token deletion and rank promotion medium
Paired gene-embedding perturbation statistics medium

Structure confidence heads

ReLU pLDDT Head medium
Linear-Stack pLDDT Head medium
Normalized Linear pLDDT Head medium
GELU pLDDT Head medium
Linear pLDDT Head easy
Resolved-Atom Head easy
Normalized Resolved-Atom Head easy
Distogram Linear Head easy
Distogram GELU Head medium
PAE Linear Head easy
PAE GELU Head medium
PDE Linear Head medium
PDE GELU Head medium
pLDDT Confidence Summary easy
Predicted TM Score medium
Interface TM Score medium
Directed Chain-Pair TM Scores hard
Bidirectional Chain-Pair TM Scores hard
Interface pLDDT Summary easy
Relative Accessible Surface Area medium
TM Ranking Score easy
AlphaFold 3 Ranking Score easy
Learned structural confidence heads and model-aware ranking expert

Small functions. Deep understanding. Sponsors Privacy and support

Sponsored by Lyra Labs