Image Processing: Critical to Hubble Discoveries

Lucas Divine & Chris LavinECE 533: Image Processing

12/12/03

Welcome to the Hubble

The Hubble Space Telescope was deployed into space in 1990, but its

conception took place in the 1940’s. The Hubble is a masterful telescope

designed with long term space observation in mind. It has three cameras, two

spectrographs, and many guidance sensors to capture high resolution images of

astronomical objects. It is 43.5 feet long, weighs 24.5 thousand pounds, and

orbits the earth every 97 minutes1. It was deployed into low-Earth orbit, which is

about 380 miles off the earth. This allows it to view space 10 times better than

any telescopes on earth. It also allows the telescope to see wavelengths that the

Earth’s atmosphere filters out. The Space Telescope Science Institute handles

most of the research and day to day operations of the Hubble. It is operated for

NASA and other Astronomical organizations. The images that the Hubble

captures require a great deal of image processing before they are useful. This

report will go over the various techniques used to create the incredible Hubble

images that the public has been seeing over the last decade.

The Hubble’s Instruments

The Hubble Space Telescope has a variety of instruments to perform all of

the necessary tasks that it needs to accomplish. Many of them allow for better

image processing back on earth. There are many components designed to aid

calibrating, focusing, and pointing the Hubble before any of the cameras can do

their job. Software packages developed specifically for these purposes allow the

control of the configuration hardware. Sensors allow for the reporting of the

hundreds of parameters that the other components need to know. The Hubble

is equipped with Fine Guidance Sensors (FGS) that are used for high speed

1

photometry, astrometry, and pointing of the telescope. They provide feedback

used to maneuver the telescope and perform celestial measurements.

The main interests of Image Processors are the cameras and other key

instruments that directly affect the Images that the Hubble Space Telescope is

able to take. They consist of the ACS, NICMOS, STIS, and WFPC22.

ACS: This is the Advanced Camera for Surveys. It is a third generation

Hubble Instrument that includes three electronic cameras, or channels, for

varying pictures. Each of the three channels (wide field, high resolution,

and solar blind) has a filter wheel or two that allow each channel to detect

large swaths of light across a huge spectrum of wavelengths. For

example, the ramp filters on the filter wheels allow narrow or medium band

imaging centered at an arbitrary wavelength. The ACS increases the

discovery efficiency of the Hubble by a factor of ten.

NICMOS: The Near Infrared Camera and Multi Object Spectrometer sees

the universe at near infrared wavelengths at a higher level of sensitively

and in sharper detail than any other telescope. The infrared and near-

infrared are the primary focus of the three cameras that make up the

NICMOS.

STIS: The Space Telescope Imaging Spectrograph provides spectra

images at ultraviolet and visible wavelengths, probing our solar system as

well as farther cosmological distances. Thus it acts like a prism to

separate light into its component colors. It is a two dimensional

spectrograph that blocks extraneous light and generates the spectra of

many locations simultaneously.

WFPC2: This is the current Wide Field Planetary Camera 2. There was a

version 1, but the technology has been upgraded, and a version 3 is

currently being developed. This camera is the workhorse behind many of

2

the most famous Hubble pictures. The WFPC2 has a four-camera design

that allows it to view more of the sky than a single camera would. A

system of mirrors divides the beam of incoming light into four separate

streams. This is the cause of sometimes ‘stair-step’ shape images. This

device can observe just about anything and has over 48 filters.

Raw Imaging Data from the Hubble

Raw data is taken off the cameras and stored in the Hubble Data Archive 3.

When a user requests data from the Hubble Data Archive, the raw files are then

calibrated by the On The Fly Reprocessing (OTFR) system. WFPC2, NICMOS,

and STIS data can all be retrieved with the OTFR system. Through OTFR,

Hubble archive users obtain data that can be reprocessed with the latest

calibration files, up to date headers, and the latest software. This allows the

system to only store uncalibrated data, which significantly reduces the storage

space.

Even after recalibration and transmission to earth, the images themselves

need a lot of work. Depending on what a specific scientist is looking for, they

perform a variety of techniques on the Hubble pictures to take a closer look at

what they are interested in.

Techniques for Manipulating Hubble Images

Most of the cosmos image detection depends on wavelengths. The

spectrum set up to detect is very important in detecting what a scientist might be

looking for or at. Creating a color spectrum and other spectra related processing

techniques take center stage in the handling of Hubble Space Telescope raw

images. Cosmic cleaning, image restoration, finding specifics in a picture, and

finding the ‘whole picture’ from a few images are all tasks that need to be

3

completed countless times on raw Hubble images. Almost any technique that

allows you to focus on what you would like from the images is acceptable, and

we cover some of the majority of techniques here.

Some effects are very specific to the Hubble Imaging, such as effects from

temperature and positioning. There are many very specific software programs

developed to address such issues. These programs are written by groups that

work with particular areas of the Hubble imaging. Most other effects can be

applied to the images via popular image processing software packages. These

are used by the majority of people for the techniques necessary for observing

what they wish.

Smoothing

Most Scientists use smoothing generally to reduce noise in images. This

is very important because noise in astronomical images can sometimes mask the

focus point of the experiment or investigation. The smoothing itself generally is

accomplished by an averaging or low pass filter in the spatial domain. Various

filtering happens in the majority of image processing techniques. The box,

weighted average, and median filters all could be used to varying degrees in the

smoothing process.

A specific example would be residual cosmic ray contaminations4. Cosmic

rays can dilute images with clutter that is not needed. By smoothing with a

median filter, almost all cosmic ray residuals can be removed from the pictures.

This is done because the pixels surrounding the residual are used in determining

the median to get rid of the ‘cosmic ray’ noise. At times, some other information

in the image may be lost as well. This might be faint stars or objects that the filter

thinks are noise. The loss of a specific important piece of information in a picture

causes the trial of a different smoothing filter.

4

Image Restoration

“The HST image restoration problem is aggravated by insufficient image

sampling, by a mixture of noise sources including spatially non-stationary, non-

Gaussian noise, and by the desire to quantitatively evaluate the restored data 5.”

Due to these many effects that can affect Hubble images, a number of image

restoration techniques have been developed and tried. Just about all work better

on the Hubble Images than images taken from earth because atmospheric

blurring changes continuously. Image Restoration is a large field, and by

studying astronomy, many scientists can narrow down the total types of noise

models that could be applied to the image. This allows them to develop their

own type of ‘toolbox’ of image restoration methods to first use on a raw Hubble

image.

In particular, frequency domain filtering, maximum-entropy methods, and

Wiener filtering are especially useful in astronomical image restoration. Most

spatial domain filters do not perform as well as frequency ones in detecting small

details and enhancing them. In an example from ‘Digital Image Processing,’6 the

Butterworth bandreject frequency domain filter is shown to apply to NASA

images quite well. These filters restore small details and textures to images.

Wiener filtering applies to the noise and degradation of images and is also used

largely in image restoration.

Another reliable and effective restoration method for Hubble images is the

Richardson-Lucy Iteration7. This method uses Poisson statistics as well as a

convolution of the image to reproduce the image. The process is then repeated

iteratively until the results converge to the maximum likelihood solution. Images

restored using the Richardson-Lucy iteration have good photometric linearity and

can be used for quantitative analysis. One downside is possible amplification of

noise in an image, so running a noise reduction filter in the beginning helps take

care of this problem.

5

Image restoration of Hubble images is also currently a large research field.

New algorithms continue to be developed in this area as fast or faster than in

other image processing areas.

Brightness and Contrast

The brightness of an object in space can generally tell you the distance it

is away. This makes brightness in Hubble images of chief concern to

mathematicians and scientists. Brightness is a subjective term, but in

astronomical calculations the word is sometimes used when referring to

luminance or radiance.

Contrast is used to balance out the light and darks to provide a greater

level of detail in images. One of the hardest parts in dealing with astronomical

images is the unique nature of light that comes from them. Most are extremely

faint and very low contrast.

These and color adjustments are often the most used when publicizing

images because they affect how aesthetically pleasing they look in books,

journals, and news articles. At some point, almost every Hubble image has the

brightness and contrast adjusted on them.

Area of InterestOften when researchers are trying to obtain details from the Hubble’s

results the preceding image processing techniques are simply not enough. The

following are several of the most used techniques to extract particular information

that astronomers are looking for.

Edge Detection:Many of the items that researchers want to look at often incredibly small in

comparison to the entire Hubble image. Often these areas of interest can be fifty

by fifty pixels or smaller. One of the tools astronomers use to help distinguish

detail, and in particular changing details is edge detection. Edge detection is

normally accomplished by using various masks to find particular kinds of edges,

6

there are numerous algorithms available to find the most important edges. One

resent example8 of edge detection is research on one of Saturn’s moons, Titan.

Scientists wanted to understand cloud movement on the moon. The images they

had of the moon were incredibly small so they used edge detection to draw out

the clouds and then compared changes over time. Otherwise they would never

have been able to see the detail necessary to track the movements.

Subtraction:When researchers want to find subtle changes in subsequent pictures

taken by the Hubble, they can use subtraction to extract just the changes. This

technique has been used to study the changes in Saturn’s cloud coverage.

Essentially the pixel values of two images are subtracted from each other. In

areas where there is no change you will only have black. In areas of change

though, you should see be able to see the movements. In the case of Saturn

researchers were able to follow a storm, which could only be seen in infrared;

from creation to the point to where it went to the other side of the planet.

Sharpening:Not surprisingly many images require sharpening before researchers can

interpret an area of detail. Recently a group was able to discover the first brown

dwarf (in between a planet and a star size wise) by using image sharpening. The

brown dwarfs are incredibly small in comparison to a star. To confirm the

existence of the brown dwarf9 researchers had to use a modified image

sharpening technique to detect the shadow created by the dwarf.

Many of the original Hubble images were very blurred, sharpening was

essential to get anything worthwhile out of the telescope. Dealing with the early

Hubble images greatly increased our knowledge in sharpening blurred images.

Granted the blur pattern was already known, but it still taught researchers a great

deal.

7

Erosion and Dilation: When you take an image of space you are most likely going to get a lot of

stars in the picture. Often researchers aren’t interested in the thousands of tiny

points of light in these images. They want to study one or two larger stars, that

the Hubble was most likely focused on. The scientist can isolate these important

stars by using erosion. Depending on the size of the area of interest a filter size

is chosen and applied to the entire image. Only the stars that are large enough

in the image will remain. Then the researcher simply applies the same filter with

Dilation to return the detail to the remaining stars. Many of the images that are

shown to the public have this process done to them.

MEM and MLM:Although the image processes that are almost universally applied to

Hubble images normally provide for better pictures they can often take important

details away. It has been discovered that smoothing function that is used on

Hubble images has the tendency to alter the brightness levels of nearby pixels.

These changes are minor, but can still significantly affect results. To counter act

the affects deconvolution of the image is performed using Maximum Entropy

Method or Maximum Likelihood Method.

Building a Complete ImageMost of the Hubble Images the public sees come in nice rectangular form.

But on occasion NASA releases images with odd staircase like empty spaces in

the top right corner of the image. These images are the result of the WFPC2

camera which is actually four cameras10. Three of the four cameras take the

standard images people are use to seeing. The top right camera though actual

magnifies the area it is looking at to provide more detail. Often a scientist will

8

only be interested in the picture taken by just one camera, but when they want an

image of the entire area these four images have to be placed together. Image

processing is used to first reduce the size of magnifying camera’s image so it

matches the scale of the other images. Then computers are used to overlay the

four images. In the following example you can get a feel for what is done.

Adding in ColorThe often beautiful images that we see on the news from the Hubble don't

start out with all of those amazing colors. The Hubble actually only takes gray

scale images, so typically the images that the public is presented with are

created from several pictures.

Single Image ColorIt is possible to add color to just a single gray scale image, you just assign

a particular color for each of the 256 gray values. The example below shows the

crab nebula, to draw out the detail scientists in what is as much art as science

9

choose a color spectrum and imposed it over the original brightness intensity

image. We created the color image below by using a simple matlab program.

Natural ColorThe Hubble is capable of seeing the majority of the light spectrum, but

rarely does it take a picture of the entire spectrum, using filters the Hubble

captures just a small piece of that spectrum. The images that we get are actually

just depictions of the intensity of the filtered light. For most images astronomers

first decide what kind of image they want to get. The most obvious is try to get a

natural looking image, for example taking a picture of Mars (as seen below). This

image was created by taking three separate images; one of red light, one of

green light, and finally one of blue light. These three intensity images where

combined to form an RGB image. This technique uses the fact that RGB images

are actually stored as three separate intensity images representing the three

primary colors.

Representative Color

10

But often scientists are much more interested in details that would be hard

to see if one took a natural image. For one thing, there are many phenomena in

our universe can't even be seen by the human eye. For example take this image

of Saturn11, seen next to an actual image of the planet. Saturn does not look this

to the naked eye. This image was compiled by taking three slices of the infrared

spectrum and interpreting them as the three components of an RGB image. The

colors allow researches to better understand the chemical makeup of the planet.

This representative method is often used do to the incredibly limited scope of

human vision.

Enhanced ColorThe other most frequently used technique is Enhancing Color. To bring

out details astronomers often select just certain parts of the color spectrum that

represent fine details they want to see. One of the most famous Hubble images,

the Eagle Nebula11 is created in this manner. Researches were interested in

particular atoms which give off light in very small wavelength ranges. Red shows

emission from singly-ionized sulfur atoms. Green shows emission from hydrogen.

Blue shows light emitted by doubly- ionized oxygen atoms.

11

Using all of these techniques researchers are capable of coming up with

some of the most amazing images imaginable. However each image has to be

processed separately making the procedure very slow. This is the main reason

NASA only releases a handful of new images each month. It just takes too long

to prepare images for the public.

ConclusionMany of the Hubble image processing techniques are specialized versions

of general techniques in the field. And visa versa, some actually were first

developed for the Hubble Space Telescope images and are now applied within

many disciplines of image processing.

Since the Hubble’s first launch in 1990, amazing and momentous

discoveries have taken place and some of the most incredible astronomical

images have been captured. But capturing the image is only part of the process.

Image processing must be done in a variety of forms to help humans ‘make

sense’ of the image. Image processing is critical to Hubble discoveries because

it allows for the key information extraction within the image. These images, and

the information within, have helped scientists learn about the universe, but are

12

also applied in grade schools, helping grade school students learn about the

solar system. The world relies on the captured images, as well as the scientists

and researchers to make them useful. Developing and inventing new techniques

in image processing will play a large role in the future of astronomical research.

Sources1’Facts & Figures’ - http://hubble.stsci.edu/reference_desk/facts_.and._figures/

2’HST Instruments’ - http://www.stsci.edu/hst/HST_overview/instruments

3’What does raw data look like?’ - http://www.exploratorium.edu/origins/hubble/ideas/picture/picture2.html

4’The Hubble Helix’ - http://archive.stsci.edu/hst/helix/reductions.html

5‘Hubble Space Telescope image restoration in its fourth year’ - http://www.iop.org/EJ/abstract/0266-5611/11/4/003

6Digital Image Processing by Gonzalez and Woods

7’Astronomical Image Restoration’ - http://www.iis.ee.ic.ac.uk/%7Efrank/surp00/article1/spss98/astro.html

8Titan Image Processing: http://www.cv.nrao.edu/adass/adassVI/wun.html

9photo of first confirmed 'brown dwarf' http://www.chron.com/content/interactive/space/missions/sts-103/hubble/archive/951130.html

10Drizzling Dithered WFPC2 Images - http://icarus.stsci.edu/~stefano/newcal97/pdf/mutchlerm.pdf

13

11The Meaning of Color - http://hubblesite.org/sci.d.tech/behind_the_pictures/meaning_of_color/index.shtmOthers:

‘On-the-Fly Reprocessing of HST Data‘ - http://archive.stsci.edu/hst/otfr/

‘STIS’ - http://www.ball.com/aerospace/stis.html

‘Directory of Software for image analysis’ - http://dmoz.org/Science/Astronomy/Software/Image_Processing_and_Data_Analysis/

‘Nonlinear Image Recovery with Half-Quadratic Regularization’ - http://citeseer.nj.nec.com/geman95nonlinear.html

‘Hubble Space Telescope’ - http://www.stsci.edu/hst/

14