4 3 Kettler End to End
-
Upload
vikram-bhuskute -
Category
Documents
-
view
231 -
download
1
description
Transcript of 4 3 Kettler End to End
End-to-End Speech Quality Testingin a Complex Transmission Scenario
F. Kettler*, H.W. Gierlich*,J. Berger**, H. Klaus**, I. Kliche**, K.-D. Michael**,
T. Scheerbarth**, R. Scholl***, J.-L. Freisse****
* HEAD acoustics GmbH, Herzogenrath,** T-Nova Dt. Telekom Innovationsgesellschaft mbH Berkom,
*** European Telecommu. Standards Institute ETSI (now Siemens AG)**** European Telecommunications Standards Institute ETSI
Overview
n Introduction 1st ETSI Speech Quality Test Event, Oct. 2000
n Test Setup for End-to-end Speech Quality Testing
n Test Methods, Parameters and Results
n One-way Speech Quality
n Echo and Double Talk Performance
n Background Noise Transmission Quality
n Summary
Introduction
IP networkdesigned for ‘pure’ data
transmission
• delay is high, unpredictable and time-variant• influence on echo and conversational quality• speech coders• specific kind of disturbances (packet loss, jitter...)
+ differentiation between terminal and network is nolonger possible!
Telephone networkreal time transmission
properties
ETSI Speech Quality Test Event
Organized by ETSI, Sophia Antipolis, France, Oct. 2000Sponsored by Texas Instruments, Alcatel, T-Nova and HEAD acousticsTest labs T-Nova and HEAD acoustics
n Evaluation of end-to-end (mouth to ear) speech qualityusing real equipment in a controlled VoIP environment
n One exclusive testing day for each manufactures
n Objective tests and subjective test (offline)
n Detailed test report and an anonymous reportcontaining all results
n “Goals”: identification of system improvements andcurrent status of achievable VoIP quality
Overview
n Introduction 1st ETSI Speech Quality Test Event, Oct. 2000
n Test Setup for End-to-end Speech Quality Testing
n Test Methods, Parameters and Results
n One-way Speech Quality
n Echo and Double Talk Performance
n Background Noise Transmission Quality
n Summary
Typical Signal Processing
Decoder
Buffer,Packe-tization
Buffer,PLC
H(f )
- CoderNLPNR
+
CN
VAD
EC
IP
IP
loudspeaker
jitter buffers,PLC
decodercomfortnoise
microphone
EC
levelswitching
coderVAD
noisereduction
delay,jitter,packetloss
Speech Quality Parameters
speechquality
talkingsituation
listeningsituation
conversationalsituation
… from the user’s perspective
Listening Situation
• voice activity detection• speech coders• packet loss• packet loss concealment• ...
IP
terminal A terminal B
Talking Situation
• round trip delay• echo level and echo characteristic• implementation of speech echo cancellers• quality of background noise transmission
IP
terminal A terminal B
...echo?
Conversational Situation
• propagation delay - dynamics impairments• double talk detection (EC implementation)• echo during double talk• level variations during double talk
IP
terminal A terminal B
Test Set-up
„Acoustical –acoustical“test set-up
with IP terminalsmounted to theHATS (Head AndTorso Simulators)
NISTNet
PacketMONITOR
Packet LossDELAY
Output
HEAD acousticsTestsystem CAS
Input
IP IP
IP-Phone
PCIP-Terminal
Test Set-up
„Acoustical –electrical”test set-up
with one IP phoneand one gateway(accessed througha 4-wire ISDN line)
IP
NISTNet
PacketMONITOR
T1 / E1 / BRI
Packet LossDELAY
Output
HEAD acousticsTestsystem CAS
Input
GatewayIP
ISDN DSS1
ALCATEL PABX 4400
IP
IP Terminal(Phone or PC)
Overview
n Introduction 1st ETSI Speech Quality Test Event, Oct. 2000
n Test Setup for End-to-end Speech Quality Testing
n Test Methods, Parameters and Results
n One-way Speech Quality
n Echo and Double Talk Performance
n Background Noise Transmission Quality
n Summary
Listening Situation
Analysis methods: perceptual speech quality models
IP
terminal A terminal B
feeding recording and analysis
Instrumental Measures
Realizations
� PESQ (P.862) Perceptual Evaluation of Speech Quality� TOSQA Telecommunications Objective Speech
Quality Assessment� TOSQA 2001 Extension to acoustical measurements
Typical Processing Steps (Schematic):
Adaptation Hearing Model
Comparison,Reference,Reduction,Signal Value
Results ofListening Tests!
S indexQ
SpeechSignal
Test Object Hearing Modelprocessed
signal
reference signal
1st ETSI SQTE - Test Conditions
IP
terminal A terminal B
c1: clear conditionc2: 1%packet lossc3: 2%packet lossc4: 3%Packet lossc5: 5%packet lossc6: 1%packet loss + 20 ms jitter
1st ETSI SQTE - Results
C1 C3 C5C2 C4 C6 G.711 G.729
MNRU
18 30C1 C3 C5C2 C4 C6 C1 C3 C5C2 C4 C6
241206
direkt
References
1
2
3
4
5
G.711
PL=20...30ms
VAD on, PLC on/off
PL=30ms
VAD on, PLCon/off
G.729
PL=10...30ms
VAD on, PLC on/off
G.711 reference
[email protected] ref.
G.729 ref.
TMOS
Analysis of packet loss and PLC
Occurrence of packet loss(20 ms packet length)
Packet loss concealment(typical implementation)
Detailed analysis of PLC implementations
Cross correlation analysisvs. time
Relative Approach analysisvs. time and frequency
Overview
n Introduction 1st ETSI Speech Quality Test Event, Oct. 2000
n Test Setup for End-to-end Speech Quality Testing
n Test Methods, Parameters and Results
n One-way Speech Quality
n Echo and Double Talk Performance
n Background Noise Transmission Quality
n Summary
Analysis methods: echo measurements based onComposite Source Signals (CSS) under single anddouble talk conditions
IP
terminal A terminal B
...echo?
feeding, recording
and analysis
Talking Situation
Noise part
Voiced part
Pause
• Signal descriptionin ITU-T P.501
• Used e.g. in G.168(Echo Cancellers)
Composite Source Signal
Sequence length 700 ms
SLR JLR JLR RLR7dB 0dB 0dB 3dB
TELR a(Echo) a(Echo) TELR
RLR JLR JLR SLR3dB 0dB 0dB 7dB
OLR = 10dB
Requirement on echo lossdepends on transmission
delay
Echo under Single Talk Conditions
Analysis methods: double talk measurements basedon two de-correlated Composite Source Signals
IP
terminal A terminal B
feeding feeding, recording
and analysis
Conversational Situation
• Signal description in ITU-T P.501• Analysis methods in ITU-T P.502
Combination of two CSS
Typical double talksequence (speech)English, male and femalevoices
Combination of CSSincl. single and doubletalk periods
Comparison to Real Speech
measured signalnearly perfectlymatches originaltest signal
green: measured signal
red: original test signal
yellow: overlap
Double Talk Evaluation: Result #1
green: measured signal
red: original test signal
yellow: overlap
Double Talk Evaluation: Result #2
short term echocomponentsduring pausesand jitter
echo jitter
green: measured signal
red: original test signal
yellow: overlap
Double Talk Evaluation: Result #3
strong echocomponents(occurred onlyduring doubletalk!)
strong echo
green: measured signal
red: original test signal
yellow: overlap
Double Talk Evaluation: Result #4
clipping atbeginning andend of mostbursts andcomfort noiseduring pauses
clipping
comfort noise
Overview
n Introduction 1st ETSI Speech Quality Test Event, Oct. 2000
n Test Setup for End-to-end Speech Quality Testing
n Test Methods, Parameters and Results
n One-way Speech Quality
n Echo and Double Talk Performance
n Background Noise Transmission Quality
n Summary
Test Signal for Noise Transmission
Noise signal(Hoth spectrum)with increasinglevel vs. time
Signal descriptionin ITU-T P.800
Comparison of Test Results
1
2
3
4
Comparison ofthree typicalimplementations
(1) red: original test signal
(2) yellow: VAD threshold
(3) blue: AGC gain
(4) magenta: comfortnoise injection
Overview
n Introduction 1st ETSI Speech Quality Test Event, Oct. 2000
n Test Setup for End-to-end Speech Quality Testing
n Test Methods, Parameters and Results
n One-way Speech Quality
n Echo and Double Talk Performance
n Background Noise Transmission Quality
n Summary
n Identification of improvements and current status ofachievable VoIP quality - “success”
n Combination of various test procedures!
n Overall quality rating given by the participants“excellent”
n A short report can be found under etsi.org/plugtests
n ETSI strongly recommended to continue the processof end-to-end speech quality tests
n 2nd SQTE planned for April 2002
Summary