Foundations

Binary Cross-Entropy with Logits

EasyRaw PyTorchAbout 10 min1 public example plus private tests

Free account with email, Google or GitHub. Your code runs in an isolated PyTorch sandbox.

Your task

Treat each logit as an independent probability p=sigmoid(logit) of a positive target. Before masking and weighting, each loss is −target·log(p)−(1−target)·log(1−p). Compute the equivalent stable expression from logits, then apply the requested reduction.

Function signature

binary_cross_entropy_with_logits(
    logits: torch.Tensor,
    targets: torch.Tensor,
    mask: torch.Tensor | None = None,
    weights: torch.Tensor | None = None,
    reduction: str = 'mean',
    eps: float = 1e-08,
) -> torch.Tensor

Worked example

The same score, opposite targets

Use logits=[log(3),log(3)], targets=[1,0], reduction="none", with mask and weights omitted. Both positive-class probabilities are 3/4. The positive target costs −log(3/4) ≈ 0.287682; the negative target costs −log(1/4) ≈ 1.386294. Return approximately [0.287682,1.386294]. Use the stable logit formula below in the implementation.

Parameters

Parameter	Shape and meaning
`logits`torch.Tensor	Exactly targets.shape.One unnormalized positive-class score per target; there is no class softmax axis.
`targets`torch.Tensor	Exactly logits.shape.Floating positive-target fractions in [0,1], including hard labels 0 and 1.
`mask`torch.Tensor \| NoneDefault `None`	Any shape broadcastable to targets.shape.Optional validity mask; True includes an example or binary target.
`weights`torch.Tensor \| NoneDefault `None`	Any shape broadcastable to targets.shape.Optional nonnegative element weights; omission means one.
`reduction`strDefault `'mean'`	Choose "none" for targets-shaped losses, "sum" for one scalar sum, or "mean" for one globally weighted scalar mean.
`eps`floatDefault `1e-08`	Nonnegative offset added once to the total included weight for a mean reduction. It does not modify logits, probabilities, or individual losses.

Return

torch.Tensor · targets.shape for reduction="none"; scalar tensor, shape (), for "sum" or "mean".

Return Masked weighted loss after the requested reduction.

Required behavior

Binary targets are floating values in $[0,1]$ with the same shape as logits. Compute each loss as $\max(z,0)-zy+\log(1+e^{-|z|})$ , without first forming probabilities.
mask is optional, boolean, and broadcastable to targets; True includes an element. Sanitize ignored logits/targets before arithmetic. weights is optional, nonnegative, and broadcastable to the unreduced loss; omission means one.
reduction="none" returns the masked weighted element losses; "sum" returns their scalar sum. "mean" (default) divides that sum by the total valid weight plus eps (default $10^{-8}$ ); return exact scalar zero when the valid weight sum is zero.
Floating logits, optional weights, and binary targets remain differentiable. Ignore masks and class IDs are nondifferentiable.
Use CPU float32/float64 tensors with one shared floating dtype/device. Accept noncontiguous input layouts and do not mutate supplied tensors.
Use raw tensor operations, not a ready-made implementation of the requested primitive. Supplied parameters and random variates must not be reinitialized.

Why this matters

Working in logit space avoids saturated probabilities and preserves useful gradients even for confident incorrect predictions.

Starter code

You write the body of this function in the BioTorch workspace. Your code runs in an isolated PyTorch sandbox; Run checks the public examples and Submit grades the full test suite, including gradients.

import torch


def binary_cross_entropy_with_logits(
    logits: torch.Tensor,  # Exactly targets.shape.
    targets: torch.Tensor,  # Exactly logits.shape.
    mask: torch.Tensor | None = None,  # Any shape broadcastable to targets.shape.
    weights: torch.Tensor | None = None,  # Any shape broadcastable to targets.shape.
    reduction: str = 'mean',
    eps: float = 1e-08,
) -> torch.Tensor:
    """Return a tensor with shape targets.shape for reduction="none"; scalar tensor, shape
        (), for "sum" or "mean"."""
    # Implement the operation. Inputs and test setup are supplied by BioTorch.
    raise NotImplementedError("Implement binary_cross_entropy_with_logits")

Ready to implement it?

3 progressive hints, a solution walkthrough, and grading that checks values, shapes and gradients are waiting in the workspace.

Write and run your solution Sign in

Free account with email, Google or GitHub. Your code runs in an isolated PyTorch sandbox.

Related exercises

Sources and further reading

A primitive exercise, not an end-to-end reproduction of the named models. No affiliation or endorsement is implied.