Genomics and RNA

Causal depthwise filtering with a FIR cache

MediumRaw PyTorchAbout 20 min2 public examples plus private tests

Evo 1StripedHyenaHyenaDNA

Free account with email, Google or GitHub. Your code runs in an isolated PyTorch sandbox.

Your task

Filter a sequence chunk independently by channel using its preceding input cache. Return the causal output and the updated cache needed to continue with the next chunk.

Function signature

causal_depthwise_convolution(
    u: torch.Tensor,
    kernel: torch.Tensor,
    bias: torch.Tensor,
    valid_mask: torch.Tensor,
    state: torch.Tensor | None = None,
) -> tuple[torch.Tensor, torch.Tensor]

Worked example

The cache starts at zero. The last kernel coefficient multiplies the current token: outputs are $10\cdot0+1=1$ , $10\cdot1+2=12$ , and $10\cdot2+3=23$ . Return y with shape (1, 3, 1) and state = [[[3]]] with shape (1, 1, 1).

import torch

u = torch.tensor([[[1.0], [2.0], [3.0]]], dtype=torch.float64, requires_grad=True)
kernel = torch.tensor([[10.0, 1.0]], dtype=torch.float64, requires_grad=True)
bias = torch.tensor([0.0], dtype=torch.float64, requires_grad=True)
valid_mask = torch.tensor([[True, True, True]], dtype=torch.bool)
state = None

result = causal_depthwise_convolution(u=u, kernel=kernel, bias=bias, valid_mask=valid_mask, state=state)

Expected return

(torch.tensor([[[1.0], [12.0], [23.0]]], dtype=torch.float64), torch.tensor([[[3.0]]], dtype=torch.float64))

Parameters

Parameter	Shape and meaning
`u`torch.Tensor	$[B, L, D]$ `float` $[B,L,D]$ , current projected inputs
`kernel`torch.Tensor	$[D, K]$ `float` $[D,K]$ , oldest-to-current coefficients
`bias`torch.Tensor	$[D]$ `float` $[D]$ , per-channel `bias`
`valid_mask`torch.Tensor	$[B, L]$ `bool` $[B,L]$ , `False` sanitizes input and suppresses output while advancing time
`state`torch.Tensor \| NoneDefault `None`	$[B, K-1, D]$ optional `float` $[B,K-1,D]$ , chronological FIR input history; defaults to zeros

Return

tuple[torch.Tensor, torch.Tensor]

Return these tensors in the stated order.

Required behavior

$u [B,L,D]$ , kernel $[D,K]$ with $K\ge 1$ , bias $[D]$ , valid_mask $[B,L]$ , and optional state $[B,K-1,D]$ are supplied. State is chronological, oldest to newest, immediately preceding this chunk; absent state means zeros.
Kernel is also chronological: its LAST coefficient multiplies the current token, matching depthwise cross-correlation, not a lag-ordered convolution kernel. Concatenate state with $u$ after replacing invalid $u$ vectors by zero.
For each $t$ , $y_t=\sum_{k=0}^{K-1}\mathrm{joined}[:,t+k,:]\odot\mathrm{kernel}[:,k]+\mathrm{bias}$ ; set invalid output positions to zero. Return $y [B,L,D]$ and the final $K-1$ chronological inputs as state (shape $[B,0,D]$ when $K=1$ ).
Missing positions still consume time and enter the cache as zeros; the mask is not a skip-update instruction. $L=0$ returns an empty $y$ and unchanged state. No channel mixing, groups argument, convolution primitive or input mutation is allowed.

Why this matters

A cache of past inputs makes chunked inference reproduce the same filter as a full sequence pass.

Starter code

You write the body of this function in the BioTorch workspace. Your code runs in an isolated PyTorch sandbox; Run checks the public examples and Submit grades the full test suite, including gradients.

import torch


def causal_depthwise_convolution(
    u: torch.Tensor,  # (B, L, D)
    kernel: torch.Tensor,  # (D, K)
    bias: torch.Tensor,  # (D,)
    valid_mask: torch.Tensor,  # (B, L)
    state: torch.Tensor | None = None,  # (B, K-1, D)
) -> tuple[torch.Tensor, torch.Tensor]:
    """Return tuple[torch.Tensor, torch.Tensor]:
    
    y: (B, L, D).
    state: (B, K-1, D)."""
    # Implement the operation. Inputs and test setup are supplied by BioTorch.
    raise NotImplementedError("Implement causal_depthwise_convolution")

Ready to implement it?

3 progressive hints, a solution walkthrough, and grading that checks values, shapes and gradients are waiting in the workspace.

Write and run your solution Sign in

Free account with email, Google or GitHub. Your code runs in an isolated PyTorch sandbox.

Sources and further reading

A primitive exercise, not an end-to-end reproduction of the named models. No affiliation or endorsement is implied.