Forgery manipulation detection: challenges and...
Transcript of Forgery manipulation detection: challenges and...
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Forgery manipulation detection:challenges and trends
Institute of ComputingUniversity of Campinas (Unicamp)
CEP 13084-851, Campinas, SP - Brazil
Siome [email protected]
Anderson [email protected]
1
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Forgery scenario
Digital Forensics Analysis
5
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Introduction
‣ Terminology
‣ Historical aspects
‣ Techniques
‣ Opportunities
Summary
6
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Introduction
‣ What is Digital Image Forensics?
‣ Motivation
• Crime judgement
• Proof destruction
• Creation/forgery of events
8
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Two ways of life by Oscar Rejland, 1857.
Historical aspects
10
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Historical aspects
Stalin with (original) and without (doctored) Nikolai Yezhov.
11
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Historical aspects
Israeli attack on Lebanon. Adnan Hajj photographer
darkened and dramatized the event.
12
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Historical aspects
US soldier “guides” an Iraqi with his child.
Photograph and forgery by Brian Walski.
13
One of the most impressive news photos of 2006
Liu Weiqiang of the Daqing Evening News.
Historical aspects
Recent discoveries
Rome protest, 2004.
D. Sacchi, F. Agnoli, E. Loftus. Applied Cognitive Psychology, vol. 21, n. 8, 249-273, 2007.
Recent discoveriesCredits to Stwart Franklin, 1989
D. Sacchi, F. Agnoli, E. Loftus. Applied Cognitive Psychology, vol. 21, n. 8, 249-273, 2007.
Beijing, 1989.
Science frauds.
Recent discoveries
(a) Erasing (b) Removing(c) Replicating
Top right: healing. Bottom: texture maps
H. Farid. Exposing Digital Forgeries in Scientific Images. ACM Multimedia and Security Workshop, 2006.
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Techniques
‣ Composition
‣ Retouching
‣ Sharpening
‣ Computer generation
21
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Is this image an “original” image or was it created by means of composition (copy/paste)?
‣ Does this image represent a trully scene/event or was it digitally tampered to deceive the viewer?
Important questions
22
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ What is the processing history of this image?
‣ What parts of the image has undergone any kind of processing and up to what extent?
‣ Was the image acquired by a source manufactured by vendor X or Y?
Important questions
23
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Passive blind image analysis
‣ No watermarking needed at all
24
Community efforts
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Source identification
‣ Computer generated images identification
‣ Forgery detection
Three branches
25
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Forgery detection branches
‣ Approaches based on variations of some image features
‣ Approaches based on image features inconsistencies
‣ Approaches based on image acquisition process inconsistencies
26
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
General camera pipelineLight Lens System
Exposure, focusing and image stabilization
Filters
Infre-red, anti-aliasing... for max. visible quality
Imaging sensors
CCD, CMOS...
Color Filter Arrays (CFA)...
Mosaicing
• Demosaicing• White point
correction, • Image sharpening• Aperture correction• Gamma correction• Compression...
DIP
Resulting photograph
27
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Ng et al. studied the effects of image splicing on magnitude and phase characteristics of the normalized bispectrum (bicoherence)
Normalized bispectrum is the Fourier transform of the third moment of a signal
‣ ~62% and high cost implementation
Variations in image features
28T. T. Ng, S. Chang, and Q. Sun. Blind detection of photomontage using higher order statistics. ISCAS, 2004
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Variations in image features
‣ Avcibas et al. proposed to use IQMs
‣ The approach calculates deviations between the image under analysis and its estimated original version (obtained through denoising)
I. Avcibas, S. Bayram, N. Memon, B. Sankur, M. Ramkumar. A classifier design for detecting image manipulations. IEEE ICIP, 2004. 29
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Variations in image features
‣ Analysis for controlled manipulations (brighteness, scaling, blurring, sharpening, rotation)
‣ ~74% accuracy
‣ Bayram et al. proposed the inclusion of BSMs and wavelets coeficient analysis
‣ ~90% accuracy
S. Bayram, I. Avcibas, N. Memon, B. Sankur. Image manipulation detection. JEI, vol. 15, no. 4, 2006. 30
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Farid et al. and also Fridrich et al. in independent projects have analyzed JPEG recompression artifacts
‣ They found that recompression of an (already compressed) image at a different quality factor distorts the smoothness of DCT coefficient histograms
Image features inconsistencies
31
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Essentially, the commonest image tampering approach involves splicing of images with different compression levels (therefore able to be detected)
‣ However, double compression is not a proof of tampering
Image features inconsistencies
32
Histogram of a normally distributed signal
Histogram of a single quantized signal (Step 2)
Histogram of a double quantized signal (Steps 3 followed by 2)
Histogram of a normally distributed signal
Histogram of a single quantized signal (Step 3)
Histogram of a double quantized signal (Steps 2 followed by 3)
Image features inconsistencies
‣What are the problems with the JPEG double quantization detection?
• Cropping
Image features inconsistencies
‣What are the problems with the JPEG double quantization detection?
• Cropping
• High-quality compression followed by significantly lower quality compression
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Popescu and Farid (2005), proposed a method for detecting traces of resampling
‣ The principle is that upsampling (interpolation) introduces periodic inter-coefficients correlations
‣ Resampling at arbitrary rates require combinations of up-sampling and down-sampling operations
Image features inconsistencies
36
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Image features inconsistencies
Mosaicing
Demosaicing
37
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ They used an EM algorithm to estimate the distribution parameters
‣ ~100% accuracy for RAW images
‣ Accuracy drops down under JPEG compression
Image features inconsistencies
38
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Bilinear
Bi-cubic
Smooth Hue
No CFAinterpolation
Image Prob. map p |F(p)|
Estimated interpolation coefficients from 100 images CFA interpolated with eight different algorithms.
39
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
OriginalTampered
Tampered Original
40
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
41
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣What are the problems with color filter array analysis?
41
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣What are the problems with color filter array analysis?
• Compression
41
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣What are the problems with color filter array analysis?
• Compression
• You can discover the demosaicing algorithm and apply it after the modifications
41
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Image features inconsistencies
‣ Repetition of image parts is a common form of forgery
‣ This kind of tampering can be easily detected with exaustive search and analysis of correlation parts
‣ Exaustive search-based methods are not computationally practical
42
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Image features inconsistencies
‣ Fridrich et al. (2003) proposed a faster (nlogn) and more accurate method
‣ The methods obtains DCT coefficients from overlapping sliding windows
‣ The resulting coefficients are disposed row-wise in a matrix and lexicographically sorted
J. Fridrich, D. Soukal, and J. Lukas. Detection of copy-move forgery in digital images. DFRWS, 2003. 43
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
‣ Correlated blocks under a specified threshold are tagged as duplicated
‣ A similar approach was proposed by Popescu and Farid (2005) that usesPCA coefficients instead of DCT
Image features inconsistencies
A. Popescu and H. Farid. Exposing Digital Forgeries by Detecting Duplicated Image Regions.TR 2004-515, 2004. 44
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Image features inconsistencies
A. Popescu and H. Farid. Exposing Digital Forgeries by Detecting Duplicated Image Regions.TR 2004-515, 2004.
Original Doctored Duplication maps
45
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
A. Swaminathan, M. Wu, and K. Liu. Image tampering identification using blind deconvolution. ICIP, 2006.
‣ Swaminathan et al. used inconsistencies in color filter array interpolation
‣ After estimating the CFA pattern and the interpolation filter, the demosaiced image is reconstructed and compared to the image itself
46
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
J. Lukas, J. Fridrich, and M. Goljan. Detecting digital image forgeries using sensor pattern noise. Proc. of SPIE, 2006.
‣ Lukas et al. (2006) proposed to use inconsistencies in the sensor pattern noise extracted from an image
‣ The noise patterns obtained from various regions are correlated with the corresponding regions in the camera’s reference pattern and a decision is made
47
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
48
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣What is the main problem with sensor pattern noise based approaches?
48
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣What is the main problem with sensor pattern noise based approaches?
• Flatfielding. It analyzes the two main image noise components (FPN, PRNU)
48
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣What is the main problem with sensor pattern noise based approaches?
• Flatfielding. It analyzes the two main image noise components (FPN, PRNU)
• FPN estimated with dark frame
48
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣What is the main problem with sensor pattern noise based approaches?
• Flatfielding. It analyzes the two main image noise components (FPN, PRNU)
• FPN estimated with dark frame
• PRNU needs L images with homogeneously illuminated scene
48
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
M. Jonhson and H. Farid. Exposing Digital Forgeries by Detecting Inconsistencies in Lighting. ACM Multimedia and Security Workshop, 2005
‣ Johnson and Farid (2005) analyzed the inconsistencies in light direction
‣ To estimate the light directions, the authors assume
• Surface is Lambertian (reflects light isotropically)
• Patches have a constant reflectance value (as opposite to the entire surface)
• It is illuminated by a point light source infinitelly far away
49
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
‣ Common solutions to estimate point light sources require knowledge of the 3-D surface normals and, at least, four distinct points of the surface with the same reflectance
‣ That would require more than one image or a known object in the scene (e.g., a sphere)
‣ For forensic applications, such solutions are not practical
50
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
P. Nillius and J. Eklundh. Automatic estimation of the projected light source direction. CVPR, 2001
‣ Instead, the authors used an approach proposed by Nillius and Eklundh (2001) to calculate a single directional light source from only one image
‣ That makes sense for outdoor images
‣ Authors also present an extension for more than one point light source (indoor images)
51
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Inconsistencies in the acquisition
Diffuse non-directional light
Directional light
~123º
~86º
~98º
~98º
~93º
52
WVU, Anchorage - 2008 .:.Rocha & Goldenstein, Forgery manipulation detection
Challenges
‣ Performance evaluation and benchmarking
‣ Current approaches mainly present proofs of concept
‣ Proper data sets need to be designed and shared
‣ Robustness issues -- attacks barely studied in the literature
53