Why We Replaced Bilinear with Bicubic Sampling (and How Catmull-Rom Works)

When rotating an image 45 degrees with the free transform tool, vertical edges looked like jagged staircases. The culprit was bilinear interpolation — Metal's hardware sampler blends only the 4 nearest texels, which isn't enough to anti-alias diagonal edges.

Bilinear filtering works by finding the four texels surrounding the sample point and doing weighted averaging based on distance. At 0 or 90 degrees this is fine because edges align with the pixel grid. But at 45 degrees, every edge pixel is sampling across a diagonal boundary, and 4 samples can't capture the smooth transition.

The fix was a 4x4 Catmull-Rom bicubic kernel implemented directly in the compute shader. Instead of Metal's hardware sampler, we read 16 texels manually and weight them using the Catmull-Rom spline:

For distance t from 0 to 1: w(t) = 1.5t^3 - 2.5t^2 + 1
For distance t from 1 to 2: w(t) = -0.5t^3 + 2.5t^2 - 4t + 2

We tried B-spline bicubic too — it's C2 continuous (smoother mathematically) but since it's an approximating filter rather than an interpolating one, the result looked noticeably blurry. Catmull-Rom is C1 continuous and interpolating, meaning it passes exactly through the original sample points. The result: smooth diagonal edges without any softening of the image.

One subtlety: the shader clamps the output to [0, 1] because Catmull-Rom can overshoot. Without the clamp, bright pixels next to dark edges would produce values above 1.0, causing subtle halo artifacts.

The existing catmullRomWeight function was already in our shader file for the Image Size resize dialog, so the affine transform kernel just reuses it — 16 texture reads per pixel instead of 4, but on Apple Silicon GPUs this is imperceptible even on large images.