Imaging defects of real camera systems

Traditional 3D rendering uses a perfect, idealized camera model that is easy to define in math-ematical terms and provides a reasonably fast and efficient rendering. Real world cameras, andsimilar optical imaging systems such as human eyes, exhibit quite a few inherent defects, someof which are actually essential to the perceived “realism” of an image. Too perfect images easilystand out as being synthetic. The following text is an attempt to briefly summarize the variousdefects and their causes, along with some examples on how they may be simulated using digitalmethods. Higher resolution versions of the example images can be found on the web at:http://www.itn.liu.se/~stegu/TNM057-2002/.

Depth of fieldA synthetic camera renders everything in view with perfect sharpness. A real camera has a finitedepth of field, so that only objects at a certain distance, in the focal plane, are rendered distinctly,and objects at other distances are progressively more blurred with increasing distance to the fo-cal plane. The depth of field is dependent on the focal length of the lens and the aperture. Tele-photo lenses generally have a smaller depth of field than wide angle lenses. A smaller numericalaperture (a larger aperture opening) lets more light through the lens but yields a smaller depthof field.

Depth of field is one of the most prominent and artistically important effects of a real camera,but unfortunately it is rather difficult to simulate. One option is to cheat by saving the depth in-formation for each pixel in the rendered scene, and blurring the image in a post-processing stepby a convolution filter that is chosen differently for different parts of the image depending onthe distance to the objects. This is a comparably cheap simulation, and it is quite often sufficient,but it does have its problems around edges between objects that are in focus and objects that arenot.

A common method for depth of field simulation in raytracing is to use multisampling. Severalcamera rays can be traced through each image pixel on the projection plane from slightly dif-ferent projection reference points. The resulting intensity for the pixel is calculated as the aver-age of the intensity value from each ray. All camera rays for one image pixel are traced so thatthey converge in a common projection plane, which becomes the focal plane of the rendering.The area over which the projection reference points for the different rays are spread is directlyequivalent to the aperture of the camera.

The equivalent of multisampling can be made for regular scanline-based rendering as well.Depth of field can be simulated by projecting the scene against the projection plane with severalslightly separated projection reference points, and averaging a fairly large number of images.

Figure 1: Depth of field (DOF). Left to right: infinite DOF, post effect DOF, multisample DOF

DefocusA real camera, particularly if it is of low quality, often has a visibly imperfect focus even in thefocal plane. A generally defocused image is quite easy to simulate by applying a local averagingfilter on the image. Blurring an image on a wholesale basis like this is a very common and sim-ple operation available in almost any image editing program. Formally, the blurring operationis a convolution with a filter kernel. Smaller kernels give little blur, and large kernels give heavyblur. For most blurring operations, a simple Gaussian filter kernel shape gives good results.

Noise and grainNoise in electronic image sensors, and its analog counterpart, the grain in photographic film, aredefects that are often clearly noticeable in real-world images, and may therefore need to be add-ed to synthetic images, particularly if they are to be blended with real images. Electronic noiseand film grain look quite different, but both are quite easily simulated as an image post effectby adding some randomness to the pixels of the perfect synthetic image.

Motion blurAnother common defect is motion blur, which occurs because a real camera has a shutter. Sometime is required for the available light to expose a photographic film or give a large enough sig-nal in an electronic image sensor, and any objects that move in the image during that time areblurred. A simple synthetic camera samples the scene at one single instance in time, and thusdoes not exhibit motion blur.

Motion blur is often simulated by averaging together several sampled images, distributed oversome simulated shutter time. A successful motion blur simulation done in this way requiresquite a few images, so motion blur often adds considerably to the rendering time. For raytracing,the multisampling mentioned above works well also for simulating motion blur. Motion blurand depth of field can be simulated together by distributing the ray samples for each pixel bothin time and space, thereby cutting down some on the total increase in rendering time. However,methods which incorporate explicit averaging of images over time are general but rather costlymethod. The effort can be reduced by performing the explicit averaging only in those areas ofthe image where objects move. Animation software packages have options for specifying whichobjects in a scene should be motion blurred. This is often called object motion blur, and it hasthe advantage of not adding anything to the rendering time for static parts of the scene.

Just like depth of field, motion blur may also be simulated by image post-processing methods.At rendering time, the renderer knows the position change of each object in the scene betweenone frame and the next. Velocity information can be stored for each object, or even for each pix-

Figure 2: Blur and grain. Left to right: sharp image, blurred image, simulated film grain

el, making it possible to blur moving parts of the image without having to spend any extra workon the non-moving parts. This method is often referred to as image motion blur. For many ap-plications it can be both faster and better than multisample motion blur.

Camera motion blur is a similar but more global effect. Even static scenes can be blurred froma camera motion. Blur from panning, tilting or rotating an otherwise stationary camera can besuccessfully simulated in a post-processing step as a local pixel averaging along the directionof the apparent motion in the image. Motion blur for camera trucking (changing the location ofthe camera in the scene) requires multisampling for an absolutely correct appearance, but thatis only rarely an issue. It can be simulated a lot easier and quite accurately using depth informa-tion for the scene. Even without depth information, cheap post-processing methods using linearor radial blurring can often be sufficient with proper care and thought.

Figure 3: Motion blur. Left to right: no motion blur, post effect blur, multisample blur

Figure 4: Post-effect camera motion blur.Top row: stationary camera, left-right pan, up-down tilt

Middle row: camera rotation, camera zoom

Glare, bleeding and soft focusGlare manifests itself as glowing halos around strong light sources, and a generally reducedcontrast for dark parts of scenes with strongly illuminated objects. The effect is due to imper-fections of the camera optics even at the focal plane. Most of the light is projected onto a smallvicinity of the ideal projection point, but some smaller part of it ends up in a larger vicinity ofthat point, sometimes even spread out over the entire image. Glass imperfections, dust, dirt andfingerprints can all have that effect, as can reflections within the camera housing. (That’s whythe inside of a camera is matte and black.) Glare can also arise from lateral light transport in thefilm or in the image sensor, or from electrical crosstalk between image sensor pixels, althoughin these cases it is often referred to as bleeding. Glare is not necessarily due to the camera. It canalso arise from atmospheric light scattering by dust particles.

Any type of glare can be easily simulated by a convolution operation with a large filter kernel,and adding a small part of the blurred result to the original image. Compared to defocusing,glare is a quite subtle secondary effect, but it is very visible around high intensity parts in theimage. If the dynamic range for the digital image intensity is restricted to 8 bits, the glare effecthas to be fudged by making a separate blurring of only the maximum intensity parts of the im-age. In rendering software, there is often a facility for specifying only some light sources or ob-jects which should have simulated glare around them, and this is often good enough.

Generally speaking, glare simulation is best performed on real radiance values for the imagepixels. If possible, truncation to 8 bit image pixels should not be done until after such simula-tions.

A strong and wide-spread glare is sometimes used for a dream-like or romantic effect called softfocus. Such strong glare is not only visible around light sources, but also around any reasonablylight object. Commercial soft focus optical filters exist for photography, but a similar effect canbe achieved very inexpensively by smearing some vaseline on part of the front lens. (The vase-line method is in common use by professional photographers.) As with glare, soft focus is easilysimulated by a convolution filter. Soft focus, too, is best simulated from real radiance pixel val-ues, even though an 8 bit input image can yield acceptable results, since the effect is strong andtherefore not as isolated to the high intensity parts of the image.

Figure 5: Multisample camera motion blur. Left: sidewards motion. Right: forwards motion.

StreaksA scratch somewhere on the lens will give a visible streak in the image. The streak emanatesfrom strong light sources and runs perpendicular to the scratch. A lens with many scratches onit will make a star-like pattern around light sources in the image. This can in fact be a nice ar-tistic effect, and there are optical star effect filters available. Streaks are easily simulated fromimages with real radiance values by means of convolution filters. If an 8 bit input image is used,the effect has to be isolated to only certain parts of the image, just like glare.

Lens flareLens flare is a special effect which easily gets cheesy if it is over-used, but in some cases it isdesirable. Lens flare is due to multiple internal reflections between the lens surfaces of the op-tics, and exhibits itself as halos, light spots and repeated images of the aperture shape when alight source is in view, or close to being in view. Differently from glare and streak effects, a lensflare consists of several distinct light shapes distributed along a straight line in the image. Theline runs through the high intensity light source and the center of the image. Because most reallenses have a circular field of view which is slightly larger than the diagonal of the film frame,lens flare might be present even if the light source is somewhat out of view. (The same holdstrue for glare and streaks.) To avoid unwanted lens flares in real cameras, a lens hood can beused to block out most parts of the field of view that fall outside the image, so that a lens flareis rarely present unless a strong light source is in direct view. Optics with lots of lens elementsgenerally give stronger and more complex flares than those with only a few lens surfaces. Sim-ulating a lens flare digitally in a simple manner is only a matter of blending a sequence of flareimages into the picture along a straight line. Simulating it in a physically correct way requiresdetailed knowledge of the construction and material properties of a real lens, but this wouldprobably never be an issue.

Figure 6: Top row: no special effect; glare around the light source.Bottom row: glare and streaks; glare, streaks and lens flare.

CoronasCoronas are similar to glare in that they appear around strong light sources, but they have a dis-tinct, ring-like appearance, and they can shift in the colors of the rainbow. Coronas are not al-ways a camera defect, but can also arise from light scattering or refraction by particles, icecrystals or water droplets in the atmosphere. A corona is rather easy to simulate using a post-processing step, either by using high dynamic range radiance values as input, or isolating theeffect to only some objects or light sources. It is not that visually important to get it exactly right.To simulate its exact appearance, a color sampling with significantly more than three channelsis required, so called hyperspectral color sampling.

VignettingA real camera has a higher intensity for the projected image in the middle than at the edges. Thisradial light falloff is called vignetting. It can sometimes be clearly visible, in particular for verycheap cameras. Vignetting occurs because the amount of light that is projected through the op-tics decreases with the angle of incidence against the optical axis, for several reasons. First,the apparent aperture size from an oblique angle decreases with . Second, the distancefrom the film plane to the aperture increases with towards the edges of the image, giv-ing an apparent size of the aperture as seen from the film plane that decreases with . Third,the solid angle subtended by some small area (e.g a sensor pixel) of the image sensor decreaseswith . The collective impact of these effects would mean that the image intensity shoulddecrease with the fourth power of the cosine of the angle of incidence, . This is often re-ferred to as the “cos4 law”. The “law” is actually only valid for a pinhole camera. It is at leastapproximately correct for a single, fairly flat lens with a small aperture, but modern high qualityoptics are designed to reduce vignetting, so the effect is not very strong. If desired, vignettingis a very simple effect to simulate by a post-processing step, where each pixel is attenuated witha factor depending on the distance to the image centre.

Figure 7: Perfect focus (left) and soft focus (right)

Figure 8: No vignetting (left) and strong vignetting (right)

θθcos

1 θcos⁄cos2θ

θcoscos4θ

Field distorsionReal camera lenses might project an image to the image plane somewhat differently from a per-spective projection. This is particularly true for large focal range zoom optics, very wide anglelenses or very cheap lenses. The most prominent distorsions are referred to as pincushion dis-torsion and barrel distorsion, where straight lines in the scene exhibit a curvature inwards oroutwards from the image centre. The distorsion is worse towards the image edges. Other fielddistorsions exist, but pincushion or barrel distorsion is the most prominent defect. If it shouldhappen to be a desired effect, simulating it digitally is a simple post-processing resampling step.

Color aberrationOptical materials have an index of refraction that varies at slightly with the wavelength of light.This makes it difficult (in fact impossible) to build lenses that have exactly the same imagingproperties over the entire visible range of wavelengths. Modern lens makers generally do a verygood job with compensating for this, but it is still a problem. Wavelength variations yield coloraberration, which manifests itself as a colored, rainbow-like edge ghosting towards the periph-ery of the image. It is doubtful whether anyone would like to simulate color aberration digitally,since it is a quite disturbing artifact, but similar results could be achieved by resampling the R,G and B color channels to simulate slightly different field distorsions for the different channels.To simulate color aberration in a physically correct manner, hyperspectral color sampling is re-quired.

Copyright Stefan Gustavson (stegu@itn.liu.se), 2002-01-26You may copy and use this document freely for any non-commercial, not-for-profit purpose.Commercial or for-profit use is not allowed.

Figure 9: Top row: ideal planar perspective projection, barrel distorsion.Bottom row: pincushion distorsion, color aberration (simulated from RGB components).