What Mustache Distortion Is and What Makes It Hard to Correct

Nov 16, 2026 - 08:06
0 0
What Mustache Distortion Is and What Makes It Hard to Correct

Photograph a brick wall with a 20mm lens, and one mortar line can bend two different directions inside the same frame, bowing one way through the middle of the picture and then hooking back the other way as it runs into the corner. That double bend is mustache distortion, and it is why a correction that straightens the middle of your frame pushes the corners out of true. LensTip measured the Sony FE 20-70mm f/4 G at -10.23% at 20mm in uncorrected full frame raw, with mustache deformation pronounced enough that the ends of a straight line visibly lift, and the camera's own corrected JPEG still came in at -1.65%.

A Line That Bends Both Ways

Distortion is a magnification error that depends on how far a point sits from the center of the frame. Under barrel distortion, magnification falls off as you move outward, so every point lands slightly closer to the center than geometry says it should, and straight lines bulge away from the middle. Under pincushion, magnification climbs with distance, points land farther out than they should, and lines get pulled inward. Both errors are smooth, both grow toward the corners, and both are easy to undo because the error only ever points in one direction.

Mustache distortion does both jobs in one frame. Through the mid-field, the lens acts like a barrel lens. Somewhere out toward the corners the distortion curve reaches an extreme and turns back, and from there outward the same lens behaves like a pincushion one. A horizontal line near the top of the frame sags toward the center of the picture along most of its length and then curls back up, away from it, at both ends, which is where the name comes from. The turnover is the defining feature, not a change of sign. Zeiss publishes a wide-angle example whose radial distortion is negative across the whole frame, reaching -2.6% partway out and easing to -0.8% at the corner, and lines in that frame still switch from barrel-bent to pincushion-bent along their length. A curve that turns over hard enough also crosses zero, as the worked example below does, and then the lens has a neutral ring where points land exactly where a perfect lens would put them. That is a special case rather than the definition.

A square grid rendered through mustache distortion, in the case where the curve crosses zero. Through the mid-field the grid bows outward (barrel); at the dashed neutral ring the displacement passes through zero and points land exactly where a perfect lens would put them; outside that ring the grid pulls back inward (pincushion). 

Where a neutral ring does exist, points sitting on it are already in the right place, and any correction you apply to them makes them wrong. Either way, the correction has to follow a curve that turns over at the same radius the lens does, and a plain barrel or pincushion model has no turnover in it anywhere.

Two Terms, Two Different Signs

Radial distortion is normally written as a polynomial in the distance from the center of the frame. The first two terms carry almost all of the behavior you ever see.

The radial distortion polynomial. The undistorted radius is the distance a point should sit from the center of the frame, normalized so that 1.0 is the corner; the distorted radius is where the lens actually places it. The coefficient on the squared term and the coefficient on the fourth-power term are the lens's second- and fourth-order radial distortion constants, named here for the powers of the radius; written as a displacement rather than a ratio, the same two terms are the ones the optical literature calls third- and fifth-order. Negative values pull points toward the center and bow lines outward (barrel); positive values push points out and bow lines inward (pincushion). 

Near the center of the frame, the squared term shrinks far more slowly than the fourth-power term, so the second-order coefficient sets the behavior and the higher term contributes close to nothing. Out at the corner, both terms are at full strength, and a large enough fourth-order coefficient can swamp the second-order one entirely. Give the two coefficients the same sign, and you get a plain barrel or pincushion curve that steepens toward the edge. Give them opposite signs, and the sum crosses zero at a radius you can solve for in one line.

Where a mustache curve crosses back through zero and where it turns over, in terms of the second- and fourth-order radial coefficients. The first radius is the neutral ring, where the lens places points exactly right. The second is the radius of maximum barrel before the reversal. Both radii are real numbers only when the two coefficients carry opposite signs, and where those radii land decides what you see: a turnover inside the frame makes a straight line wave, while coefficients that push both radii out past the corner leave a lens that reads as plain barrel. 

Work a concrete case. Set the second-order coefficient to -0.05 and the fourth-order coefficient to +0.06, which is a mild amount of distortion by ultrawide standards. The displacement curve bottoms out at 1.04% barrel about 65% of the way to the corner, passes through zero at 91%, and arrives at the corner 1.0% into pincushion. Nothing in that lens ever misplaces a point by more than about 1%, and yet a straight line drawn across the top of the frame will visibly wave.

Relative radial displacement across the frame for a mustache lens and for a plain barrel lens scaled to the same worst-case error. The mustache curve reaches 1.04% barrel at 65% of the distance to the corner, passes through zero at 91%, and arrives at the corner at 1.0% pincushion. The barrel curve never changes sign anywhere in the frame. 

Why One Number Cannot Describe It

The manual Distortion slider in Camera Raw and Lightroom is one number. Photoshop's Lens Correction filter gives you one Remove Distortion value. Lensfun's poly3 model, the simplest of the four it supports, carries a single coefficient, and writes the corrected radius as k1 times the cube of the undistorted radius plus one minus k1 times the radius itself. What these three controls share is not a formula but a shortage of shape. One free parameter scales a fixed curve up and down; it cannot move where that curve bends, and none of these curves bends twice. A model whose curve never turns over cannot cancel one that does.

A single-coefficient fit fails in two distinct ways. Fit the coefficient by least squares across the whole field, and the correction splits the difference, leaving 1.71% of error at the corner. Force the fit to straighten the corner exactly, which is what most people do by eye because the corner is where a bent line is most obvious, and the corner comes out clean while 1.5% of error piles up around 71% of the way out.

What a one-coefficient correction leaves behind on the mustache lens from the previous chart. A least-squares fit across the field is off by 1.71% at the corner. A fit forced to straighten the corner exactly leaves 1.5% of error at 71% of the way out. Both residuals are larger than the 1.04% worst-case distortion the uncorrected lens produced. 

Correcting a mustache lens with a single slider does get you partway there, and that is the trap. On the curve above, the least-squares fit pulls the RMS error down from 0.65% to 0.57%, so by an averaged measure the frame improved. What it cannot do is remove the wave. The worst error in the frame went up, from 1.04% to 1.71%, and moved out to the corner where a bent line is easiest to see.

What Real Correction Profiles Actually Store

The models shipping inside raw converters are not that crude. A. E. Conrady published the first rigorous treatment of decentered lens systems in the Monthly Notices of the Royal Astronomical Society in 1919, and Duane Brown turned it into a practical calibration tool in "Decentering Distortion of Lenses," which ran in Photogrammetric Engineering in 1966. The result, generally called the Brown-Conrady model, is normally truncated to three radial coefficients and two tangential ones, and three radial coefficients are plenty to bend a curve twice.

Adobe ships it. Lightroom and Camera Raw correct geometry with the Adobe Camera Model, stored in Lens Correction Profile files, which for a rectilinear lens is Brown-Conrady in Adobe's own notation. Embedding a correction inside the file rather than looking it up in a profile is a separate mechanism, and it lives in the DNG specification. The WarpRectilinear opcode, introduced in DNG 1.3, has a radial term that is a sum of even powers of the radius up to the sixth, which carries the mapped radius itself through the seventh, and the specification requires the resulting warp function to be increasing, which forbids any correction that would fold one part of the field over another. DNG 1.6 added a second one. WarpRectilinear2 exists, in Adobe's words, to "support the correction of more complex (higher-order) distortion," and it carries radial terms in both odd and even powers out to the fourteenth, along with a division mode and a settable range of valid radii. A file can hold both, the newer opcode for readers that understand it and the older one behind it as a fallback. Lensfun's ptlens model, inherited from the database Tom Niemann built for PTLens, spreads three coefficients across a fourth-order polynomial, and its calibration guide tells you to reach for it when the simpler single-coefficient model "does not give good enough results."

Imatest is blunter about where the threshold sits. Its distortion documentation states that "arctan/tan and 3rd order models are inadequate for measuring wave (mustache) distortion. A minimum of a 5th order polynomial is required." That is the measurement side of the same problem. If your test software cannot represent the shape, it cannot report the wave faithfully. It will still return a number, because it fits whatever models it has and carries a residual error for each; Imatest prints the best fit, "the one with the least error," in bold. What goes missing is the shape, collapsed into a single best-fit figure with the wave left sitting in the residual.

Where the Wave Comes From

Ultrawides built for SLRs, and a good many built for mirrorless, are retrofocus designs, with a strongly divergent front group that pushes the back focal distance out past the focal length. On an SLR that distance is not optional, because the rear element has to clear a swinging mirror. Mirrorless bodies removed the requirement and the layout stayed anyway, for a different reason: a Distagon, Zeiss writes, is "also ideal for mirrorless system cameras thanks to their optimized ray path," holding correction "all the way to the corners of the image" with very low field curvature. It is still not the only way to build one. The largely symmetrical Biogon works on short-flange bodies and sidesteps the asymmetry entirely, which is exactly why Zeiss credits it with "exceptionally good distortion correction." Retrofocus is the common route all the same, and Zeiss notes that many retrofocus wide-angles carry this shape of distortion curve. That arrangement is deeply asymmetric, and asymmetric systems are the hard case for distortion correction, because the divergent front group introduces exactly the error the rest of the design has to walk back.

In classical aberration theory, Seidel distortion scales with the cube of the field height. The next order of the expansion adds terms that scale with the fifth power. Near the axis, the third-order term dominates by a wide margin. Far out in the field, the fifth-order term catches up, and if a designer sets the fifth-order coefficient opposite in sign to the third, the distortion curve reverses somewhere between the two. Aspheric surfaces give the designer fine control over exactly where each ray lands as a function of height on the element, which is precisely the freedom needed to shape a curve that turns over.

Some of this is deliberate. Image Engineering notes that certain systems, particularly the small modules in phones, are optimized to zero out distortion at the maximum image height, since a bent line at the corner is what a viewer notices first. Those systems, it says, "show the highest distortion level at lower distances from the optical center," and what comes out "is often a mixture of barrel and pincushion distortion and is described with the term wave distortion." Nulling the corner is already a mustache, because the curve has to turn back before it gets there. Push the null a little inside the corner and you get the stronger version, where the outermost ring bends the other way and the displacement itself changes sign. It is the same trade the corner-forced fit in the chart above makes, only executed in glass instead of software.

The commercial layer sits on top of that. Software correction is now an input to the optical design rather than a repair applied afterward. Fully correcting geometric distortion in glass tends to demand more elements, and more elements bring aberrations of their own. Distortion is also the aberration most amenable to a mathematical fix, because it moves information around rather than destroying it. Accept some barrel, spend the saved elements on sharpness and lateral color, and ship a smaller lens. The Sony 20-70mm going from -10.23% in raw to -1.65% in the corrected JPEG is that bargain in numbers.

Why Even a Good Profile Leaves Something Behind

Profile formats can describe a wave, so the residual you see in practice comes from somewhere else. Four things put it there.

Profiles are sampled, not continuous. Adobe profiles store separate measurements for combinations of focal length, focus distance, and aperture, then interpolate between them. A zoom's distortion curve does not merely grow and shrink across the range, it changes shape, and the turnover migrates as you zoom. Interpolating between a measurement at 20mm and one at 35mm gets the amplitude roughly right and the position of the turnover only approximately.

Distortion also moves with focus. A lens focused at two meters is not on the same curve as the same lens focused at infinity, and profiles carrying only one focus distance are extrapolating everywhere else. LensTip's headline figure for the Laowa 12mm f/2.8 Zero-D on full frame was -2.02%, and the same review recorded -2.39% under a different combination of focus distance and aperture.

Your copy is also not the copy that was measured. Decentering distortion, the subject of Brown's 1966 paper, is not radially symmetric at all. It comes from elements sitting slightly off axis after assembly, and no radial polynomial of any order can undo it, because a radial model assumes the error depends only on distance from the center. The tangential terms in Brown-Conrady approximate it, but they approximate the decentering of the sample the profile was built from, not the decentering of the lens in your hand.

The correction itself also costs something. Undoing 10% of barrel distortion means resampling the whole frame, stretching the corners hard and cropping away real angle of view. Undoing a mustache spreads that cost unevenly, because the correction expands some annular zones of the frame and compresses others, so per-pixel detail after correction varies with distance from the center instead of falling off smoothly. The size of the penalty still tracks the size of the warp, and a 1% wave asks far less of the pixels than 10% of barrel does. Reviewers of heavily corrected ultrawides regularly report softer corners after the profile runs, and that is not the profile failing. It is the profile working, and the pixels paying for it.

Buying glass that never produces the shape is the cleanest escape, and it is harder than the marketing suggests. Venus Optics named a whole line Zero-D for distortion control, and LensTip still measured the Laowa 12mm f/2.8 Zero-D at -0.72% on APS-C and -2.02% on full frame. Those are excellent numbers for a 12mm lens and they are not zero. The reviewer's verdict was that -2% would read as a sensible result on any other lens and reads as a shortfall on one sold on the promise. If you shoot architecture and interiors, where every vertical in the frame is a line a client will scrutinize, 2% of barrel means a correction pass on every frame you deliver. For a landscape shooter working with horizons and trees, it is invisible.

How to Find Out What Your Lens Is Doing

The test takes ten minutes. Find a large flat brick wall, or a tiled facade, or anything with a long straight repeating line. Square the camera to it carefully, since perspective convergence from a tilted camera will muddy everything you are trying to see. Shoot at the widest focal length and at your normal working aperture, then repeat at a middle focal length if you are testing a zoom.

Apply your usual profile in your usual converter and look at a mortar line running across the upper third of the frame. Lay a straight edge on it, digitally or literally. If the line crosses your straight edge somewhere between the middle and the corner, so the gap sits on one side of the edge through the middle of the frame and flips to the other side out near the end, you have residual mustache, and the amount you can see is what your profile is not catching. A gap that simply opens in the middle and closes at both ends is ordinary leftover barrel or pincushion, and one more nudge of the slider will take it out. If you then drag the manual Distortion slider and watch the middle straighten while the corners bow the other way, you have confirmed the whole argument on your own file in about four seconds.

What you do next depends on the tool. Hugin will fit three coefficients instead of one, optimizing a, b and c against straight-line control points you place yourself. PTLens applies stored three-coefficient calibrations. Lensfun-based converters apply whichever model its database holds for your particular lens, and that varies: ptlens carries three coefficients, poly5 two, the Adobe Camera Model three radial plus two tangential, and poly3 only one. The first three can follow a turnover. Poly3 cannot, so reaching for a Lensfun profile does not by itself guarantee a model that bends twice. Where the numbers came from matters as well, since a stored calibration describes the copy that got measured rather than the one in your hand. Lightroom will get you further with a good manufacturer profile than with the manual slider, and if the profile is close but not right, the remaining move is to correct the frame in a tool that can bend twice rather than to keep pulling on a control that can only bend once. Stitchers care more than anyone, because a distortion error that looks like nothing in a single frame turns into a visible mismatch at every seam, which is why panoramic stitching workflows start with lens correction rather than ending with it.

And if you are shopping rather than fixing, ask reviewers for the shape of the curve and not just the headline percentage. LensTip measured the manual-focus Samyang 14mm f/2.8 at -5.44% on APS-C and -7.21% on full frame, and it is the full frame result where the review flags a noticeable mustache, which is what you would expect if the reversal sits outside the radius an APS-C frame ever reaches. Mount that lens on a crop body, and you are dealing with a bow that runs in one direction, which a one-coefficient correction can chase. Mount it on full frame, and you have inherited a second problem the spec sheet never mentioned.

The Samyang 14mm f/2.8 IF ED UMC Aspherical, the lens in the LensTip figures above, with its fixed petal hood over a front element too convex to take a screw-in filter. The bulge is what lets the frame reach so far. The bend in the correction curve comes from somewhere else: the design countermeasures used to hold distortion down across a field that wide. Photo by Diliff, CC BY-SA 3.0. Source.

Lead image by Jacek Halicki, CC BY-SA 3.0. Source.

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Wow Wow 0
Sad Sad 0
Angry Angry 0

Comments (0)

User