IQA 暑期班 2017 -...
Transcript of IQA 暑期班 2017 -...
![Page 2: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/2.jpg)
Image Quality Assessment via Deep Learning 2
Contents
• Background
• Image Quality Assessment (IQA)
• Photo Quality Assessment (PQA)
• Biometric Quality Assessment (BQA)
• Discussions
![Page 3: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/3.jpg)
Image Quality Assessment via Deep Learning 3
Contents
• Background
• Image Quality Assessment (IQA)
• Photo Quality Assessment (PQA)
• Biometric Quality Assessment (BQA)
• Discussions
![Page 4: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/4.jpg)
Image Quality Assessment via Deep Learning 4
Background
• Dramatically increasing amount of imagesn Devices: computers, mobile phones, cameras, monitors, …
n Applications: medias, websites, IM clients, …
![Page 5: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/5.jpg)
Image Quality Assessment via Deep Learning 5
Background
• Extensively existed distortionsn Processing: acquisition, compression, transmission,
reconstruction, …
n Distortions: blurring, JPEG compression, Gaussian noise, …
![Page 6: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/6.jpg)
Image Quality Assessment via Deep Learning 6
What is Quality?
• Various aspectsn Athletics: perceived beauty or image appeal
n Fidelity: deviation from the undistorted version
n Intelligibility: discriminability of image content
[Keelan,HandbookofImageQuality,2002][JanssenandBlommaert,JIST,1997]
[Janssen,Proc.IEEE,2001][Halonen,etal.,Proc.SPIE,2011]
Athletics FidelityUndistortedDistorted
Intelligibility
![Page 7: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/7.jpg)
Image Quality Assessment via Deep Learning 7
Background
• Demands for IQA metrics
Evaluation, control, and improve the perceptual quality of multimedia content, quality of service (QoS), image processing systems, …
[Z. Wang, IEEE SPM’11]
Diagram of IQA-based feedback optimization
method.
ImageProcessingSystem
UpdatingAlgorithm
IQAEvaluation
Input Output
![Page 8: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/8.jpg)
Image Quality Assessment via Deep Learning 8
Background
• Common Subjective IQA Method
1005 Excellent
4 Good
3 Fair
2 Poor
1 Bad
80
60
20
0
40
Quality Scale/Score
Categorical Continuous
Test image k Test image k+1
T1 = 10s Reference image
T2 = 3s Mid-gray image
T3 = 10s Test image
T4 = 10s Mid-gray image
[ITUR BT.500-13., 2012]
![Page 9: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/9.jpg)
Image Quality Assessment via Deep Learning 9
Contents
• Background
• Image Quality Assessment (IQA)
• Photo Quality Assessment (PQA)
• Biometric Quality Assessment (BQA)
• Discussions
![Page 10: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/10.jpg)
Image Quality Assessment via Deep Learning 10
Image Quality Assessment (IQA)
• Traditional Objective Methodsn Mean Squared Error (MSE)
n Peak Signal-to-Noise Ratio
n Advantages
• Mathematically conventional
• Effective for white noise
n Disadvantage
• Lack of consideration of human visual property
∑∑= =
−=m
i
n
jjiRjiI
mnMSE
1 1
2),(),(1
⎟⎟⎠
⎞⎜⎜⎝
⎛⋅=
MSEIPSNR2max
10log10
[Wang&Bovik,IEEESPM09]
Brightened(MSE=309) JPEG(MSE=309)
Gaussiannoise(MSE=309) Gaussianblur(MSE=309)
![Page 11: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/11.jpg)
Image Quality Assessment via Deep Learning 11
Image Quality Assessment (IQA)
• How to Construct Objective IQA Metrics?
KnowledgeaboutTheHVS
PerceptualModeling
VisualPhysology andPhychophysics
Knowledgemapexpressing theelementsrequiredtoconstructsuccessful IQAmodels.
[Wang&Bovik,SPM’11]
KnowledgeaboutImageSource
Deterministic Natural Scenestatistics (NSS)
KnowledgeaboutImageDistortions
ArtifactDetection
DistortionModelling
Model-basedImageQualityAssessment
DualProblems
![Page 12: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/12.jpg)
Image Quality Assessment via Deep Learning 12
Image Quality Assessment (IQA)
• Classification of Objective IQA
Reduced-reference(RR)IQAmethods
No-reference(NR)IQAmethods
Full-reference(FR)IQAmethods
RR-IQAmethods
NR-IQAmethods
FR-IQAmethods
Referenceimage
Distortedimage
SubjectiveQualityScores
SupplementaryInformation
Distortedimage
SubjectiveQualityScores
DistortedimageReferenceimage
SubjectiveQualityScores
![Page 13: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/13.jpg)
Image Quality Assessment via Deep Learning 13
Image Quality Assessment (IQA)
• FR-IQA: Structural Similarity Index (SSIM)
Referenceimage!
Distortedimage"
LuminanceMeasure:Meanintensity
#$ =1'( )*
+
*,-
ContrastMeasure:Standarddeviationofintensity
.$ =1
'−1( )* −#$ 0+
*,-
-0
StructureMeasure:Normalizedimage
)−#$ /.$
2 !, " = 2#5#6+ 8-#50 +#60 + 8-
9 !, " = 2.5.6 +80.50 +.60 +80
: !, " = 2.56 +8;.5.6 +8;
<<)= !," =2 !," > ? 9 !," @ ? : !," A
[Z.Wang,etal. TIP’04]
n Advantages: efficient and effective
n Extensions: CBM [X. Gao et al., LNAI’05], IW-SSIM[W. Zhou and Q. Li, TIP’11], RR-SSIM [A. Rehman
and Z. Wang, TIP’12], etc.
![Page 14: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/14.jpg)
Image Quality Assessment via Deep Learning 14
Image Quality Assessment (IQA)
• FR-IQA: Deep Similarity (DeepSim) [F.Gao,etal. NEUCOM’17]
![Page 15: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/15.jpg)
Image Quality Assessment via Deep Learning 15
Image Quality Assessment (IQA)
• RR-IQAn Application: wireless communication scenario
n Framework of RR-IQA System
Distortion Channel
Feature Extraction
Ancillary Channel
Feature Extraction
RR Quality Analysis
Sender Side Receiver Side
original image
quality meature
RR features
distorted image
![Page 16: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/16.jpg)
Image Quality Assessment via Deep Learning 16
Image Quality Assessment (IQA)
• NR-IQAn Convolutional Neural Networks for BIQA (Kang et al. CVPR’14)
50 feature mapsInput patch
32
32 77
minpooling
maxpooling
convolutional 26
26
50
50800
ReLUs ReLUs
linear regressio
noutput(score)
800
a) The input is locally normalized 32�32 image patches;
b) The convolutional layer produces 50 feature maps each of size 26�26, followed by a pooling operation that reduces each feature map to one max and one min;
c) The last layer is a simple linear regression with a one dimensional output that gives the score.
![Page 17: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/17.jpg)
Image Quality Assessment via Deep Learning 17
Image Quality Assessment (IQA)
• Performance Evaluationn Color Image Database TID2013
• 25 reference color images (512*368 pixels);
• 120 distorted versions of each reference image;
• 24 types of distortions, 125 images of each type;
• Totally 3000 images;
• MOS of each image is available.
[N. Ponomarenko, et al. EUVIP’13]
![Page 18: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/18.jpg)
Image Quality Assessment via Deep Learning 18
Image Quality Assessment (IQA)
• Performance Evaluationn Scatter plots
![Page 19: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/19.jpg)
Image Quality Assessment via Deep Learning 19
Image Quality Assessment (IQA)
• Performance Evaluationn Pearson linear correlation coefficient (PLCC),
n Spearman’s rank- order correlation coefficient (SRCC), and
n Kendall rank-order correlation coefficient (KRCC)
![Page 20: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/20.jpg)
Image Quality Assessment via Deep Learning 20
Image Quality Assessment (IQA)
• Applications: IQA Guided CNNs Training
![Page 21: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/21.jpg)
Image Quality Assessment via Deep Learning 21
Image Quality Assessment (IQA)
• Applications: GAN
![Page 22: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/22.jpg)
Image Quality Assessment via Deep Learning 22
Image Quality Assessment (IQA)
• Applications: Compression Artifact Removal
Galteri, Leonardo, et al. “Deep Generative Adversarial Compression Artifact Removal." arXiv preprint arXiv:1704.02518 (2017).
![Page 23: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/23.jpg)
Image Quality Assessment via Deep Learning 23
Contents
• Background
• Visual Quality Assessment (VQA)
• Photo Quality Assessment (PQA)
• Biometric Quality Assessment (BQA)
• Discussions
![Page 24: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/24.jpg)
Image Quality Assessment via Deep Learning 24
Photo Quality Assessment (PQA)
• What is Photo Aesthetic Quality?
Beautiful / Professional / Good Ugly / Amateurish / Bad
[Marchesotti et al. 2015]
![Page 25: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/25.jpg)
Image Quality Assessment via Deep Learning 25
Photo Quality Assessment (PQA)
• Challengesn Aesthetic Principles
n Subjective PQA Mechanism
n Personalized
n Correlated with various image attributes (e.g. emotion, scene, semantic, style)
![Page 26: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/26.jpg)
Image Quality Assessment via Deep Learning 26
Photo Quality Assessment (PQA)
• Research Groups
Prof. Tang Xiaoou, SeanFellow IEEE
Multimedia Laboratory
Xinmei TianAssociate Professor
Xin Lu
![Page 27: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/27.jpg)
Image Quality Assessment via Deep Learning 27
Photo Quality Assessment (PQA)
• AVA database (250k images)n Ratings, Average ratings, Semantics, Styles
![Page 28: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/28.jpg)
Image Quality Assessment via Deep Learning 28
Photo Quality Assessment (PQA)
• Crowd Opinion [Marchesotti et al. 2015]
![Page 29: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/29.jpg)
Image Quality Assessment via Deep Learning 29
Photo Quality Assessment (PQA)
• Crowd Opinion [Marchesotti et al. 2015]
![Page 30: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/30.jpg)
Image Quality Assessment via Deep Learning 30
Photo Quality Assessment (PQA)
• Crowd Opinion [Marchesotti et al. 2015]
![Page 31: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/31.jpg)
Image Quality Assessment via Deep Learning 31
Photo Quality Assessment (PQA)
• Style
low depth-of-field, single salientobject and rule-of-thirds
[Marchesotti et al. 2015]
black and white
![Page 32: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/32.jpg)
Image Quality Assessment via Deep Learning 32
PQA via Deep Learning
• Methodology
Y. Deng, C.-L. Chen, and X. Tang, “Image aesthetic assessment: An experimental survey,” 2016.
PQA via CNNs
Single-column CNNs
Multi-column CNNs
Multi-task CNNs
![Page 33: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/33.jpg)
Image Quality Assessment via Deep Learning 33
PQA via Deep Learning
• Single-column CNNsn CNNs trained on ImageNet
n Transferred CNNs
• Pretrain in the PQA task
• Fine-tuning in the binary classification task of photo aesthetic (good or bad)
Y. Deng, C.-L. Chen, and X. Tang, “Image aesthetic assessment: An experimental survey,” 2016.
![Page 34: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/34.jpg)
Image Quality Assessment via Deep Learning 34
PQA via Deep Learning
• Multi-column CNNsn global features + local features
n global features + attribute features
Y. Deng, C.-L. Chen, and X. Tang, “Image aesthetic assessment: An experimental survey,” 2016.
![Page 35: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/35.jpg)
Image Quality Assessment via Deep Learning 35
PQA via Deep Learning
• Multi-task CNNsn Correlated with various image attributes (e.g.
emotion, scene, semantic, style)
• Task 1: Aesthetic prediction
• Task 2: Semantic prediction
• Task n: …
Y. Deng, C.-L. Chen, and X. Tang, “Image aesthetic assessment: An experimental survey,” 2016.
![Page 36: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/36.jpg)
Image Quality Assessment via Deep Learning 36
PQA via Deep Learning
• Multi-task Multi-column CNNs Y. Deng, C.-L. Chen, and X. Tang, “Image aesthetic assessment: An experimental survey,” 2016.
![Page 37: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/37.jpg)
Image Quality Assessment via Deep Learning 37
PQA via Deep Learning
• A-lampn Adaptive Layout-Aware Multi-Patch
![Page 38: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/38.jpg)
Image Quality Assessment via Deep Learning 38
PQA via Deep Learning
• A-lamp
Pipeline of attribute-graphs construction. (a) Salient objects (labeled by yellow bounding boxes) are first detected by a trained CNN, and regarded as local nodes. The dashed green bounding box denote the overall scene, which served as a global node. (b) Local and global attributes are extracted from these nodes to capture the object topology and the image layout. (c) Attribute-graphs are constructed and (d) concatenated into an aggregation layer.
![Page 39: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/39.jpg)
Image Quality Assessment via Deep Learning 39
PQA via Deep Learning
• Performance
Y. Deng, C.-L. Chen, and X. Tang, “Image aesthetic assessment: An experimental survey,” 2016.
![Page 40: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/40.jpg)
Image Quality Assessment via Deep Learning 40
PQA via Deep Learning
• Applications
n Enhancement
n Retargeting
n Retrieval
n Mobile APPs
![Page 41: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/41.jpg)
Image Quality Assessment via Deep Learning 41
Image Enhancement
• EnhanceGAN
• Deng, Yubin, Chen Change Loy, and Xiaoou Tang. "Aesthetic-Driven Image Enhancement by Adversarial Learning." arXiv preprint arXiv:1707.05251(2017).
![Page 42: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/42.jpg)
Image Quality Assessment via Deep Learning 42
Image Enhancement
• Creatism
• Fang, Hui, and Meng Zhang. "Creatism: A deep-learning photographer capable of creating professional work." arXiv preprint arXiv:1707.03491(2017).
A panorama (a) is cropped into (b), with saturation and HDR strength enhanced in (c), and with dramatic mask applied in (d).
![Page 43: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/43.jpg)
Image Quality Assessment via Deep Learning 43
Contents
• Background
• Image Quality Assessment (IQA)
• Photo Quality Assessment (PQA)
• Biometric Quality Assessment (BQA)
• Discussions
![Page 44: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/44.jpg)
Image Quality Assessment via Deep Learning 44
Biometric Quality Assessment (BQA)
• Definitionn Quality of a biometric
sample is a measure of its efficiency in aiding recognition of an individual, ideally, irrespective of the recognition system in use.
Sample images of varying quality.
(a) Fingerprint, (b) iris (from WVU multimodal database), and (c) face (from SCface and CAS-PEAL face databases)
![Page 45: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/45.jpg)
Image Quality Assessment via Deep Learning 45
Biometric Quality Assessment (BQA)
• Pipeline of a typical biometric system.
![Page 46: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/46.jpg)
Image Quality Assessment via Deep Learning 46
Biometric Quality Assessment (BQA)
• Face
![Page 47: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/47.jpg)
Image Quality Assessment via Deep Learning 47
Biometric Quality Assessment (BQA)
• Face: ID
![Page 48: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/48.jpg)
Image Quality Assessment via Deep Learning 48
Biometric Quality Assessment (BQA)
• Four primary image features for BQA
![Page 49: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/49.jpg)
Image Quality Assessment via Deep Learning 49
Biometric Quality Assessment (BQA)
• BQA via LightCNN
![Page 50: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/50.jpg)
Image Quality Assessment via Deep Learning 50
Biometric Quality Assessment (BQA)
• Face recognition/identification/verification
Outline of the DeepFace architecture.
![Page 51: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/51.jpg)
Image Quality Assessment via Deep Learning 51
Biometric Quality Assessment (BQA)
![Page 52: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/52.jpg)
Image Quality Assessment via Deep Learning 52
Biometric Quality Assessment (BQA)
• Applications: Utilizing BQA for context switching
![Page 53: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/53.jpg)
Image Quality Assessment via Deep Learning 53
Contents
• Background
• Image Quality Assessment (IQA)
• Photo Quality Assessment (PQA)
• Biometric Quality Assessment (BQA)
• Discussions
![Page 54: IQA 暑期班 2017 - 杭州电子科技大学mil.hdu.edu.cn/resources/IQA_PQA_BQA_applications_FeiGao.pdf · Image Quality Assessment via Deep Learning 44 Biometric Quality Assessment](https://reader033.fdocuments.net/reader033/viewer/2022042108/5e87c0680c0ad700281d09a9/html5/thumbnails/54.jpg)