Supplementary Material: Learning to Fuse Proposals from ... · Supplementary Material: Learning to...

Supplementary Material:Learning to Fuse Proposals from Multiple

Scanline Optimizations in Semi-Global Matching

Johannes L. Schonberger 1,2 Sudipta N. Sinha 1 Marc Pollefeys 1,2

1 Microsoft 2 Department of Computer Science, ETH Zurich

In this supplementary material, we show additional results for all pixels inTables 1 and 2 in Section 5.2 in the main paper. Furthermore, we provide adetailed list of training/test scenes and the per-dataset performance for theablation studies in Table 1 in Section 5.2 in the main paper. In addition, weshow further results of our proposed raw and filtered SGM-Forest output incomparison to baseline SGM. For more visualizations, we refer the reader to theMiddlebury 20141, KITTI 20152, and ETH3D3 benchmark websites.

1 http://vision.middlebury.edu/stereo/eval3/2 http://www.cvlibs.net/datasets/kitti/eval scene flow.php?benchmark=stereo3 https://www.eth3d.net/low res two view

2 J.L. Schonberger, S.N. Sinha, M. Pollefeys

Adirondack ArtL

Disparity

Error

SGM SGM-Forest(raw) SGM-Forest(filt.) SGM SGM-Forest(raw) SGM-Forest(filt.)

Jadeplant Motorcycle

Disparity

Error


Piano PianoL

Disparity

Error


Fig. 1. Qualitative Middlebury results for SGM and SGM-Forest. Absolute error mapsclipped to [0px, 8px]. Refer to main paper for MotorcycleE, Pipes, Shelves scenes.

Learning to Fuse Proposals from Multiple Scanline Optimizations in SGM 3

Playroom Playtable

Disparity

Error


PlaytableP Recycle

Disparity

Error


Teddy Vintage

Disparity

Error


Fig. 2. Qualitative Middlebury results for SGM and SGM-Forest. Absolute error mapsclipped to [0px, 8px]. Refer to main paper for MotorcycleE, Pipes, Shelves scenes.


Table 1. Additional results for Table 1 in Section 5.2 in the main paper. Validationperformance for non-occluded (top) and all (bottom) pixels on the Middlebury 2014training set (15 half resolution pairs). Rows 1–5 show results for SGM baselines. Rows6–14 report ablation studies for SGM-Forest. Bottom three rows show results for thebest SGM-Forest setting, trained on different datasets. Letters M, K, and E refer toMiddlebury 2005–06, KITTI, and ETH3D, respectively. The matching cost is alwaysMC-CNN-acrt. Runtimes exclude matching cost and timed on same CPU.

Method

Left

Vie

wScanlines

Rig

ht

Vie

wScanlines

Filterin

g

Train

ing

Dataset

bad

0.5

px

[%]

bad

1px

[%]

bad

2px

[%]

bad

4px

[%]

Tim

e[s]

non-occluded

SGM all – 50.85 23.04 8.89 5.16 3.0SGM – mind Lr(p, d) all – 52.18 25.45 11.81 7.79 3.1SGM – mind medianr Lr(p, d) all – 63.25 31.81 9.90 8.24 3.2SGM-SVM all M 48.68 21.88 8.57 5.09 323.7SGM-MLP all M 47.77 21.83 8.53 5.08 21.0

SGM-Forest

horiz+vert M 47.36 21.30 8.49 4.93 5.7top-down M 47.45 21.20 8.38 4.94 5.8bottom-up M 47.65 21.54 8.54 4.98 5.8

all M 46.67 20.85 8.40 4.89 6.1all • M 46.49 20.81 8.23 4.72 6.3all • • E 46.80 20.32 8.17 4.79 8.2all • • K 46.48 20.45 8.09 4.81 8.2all • • M 46.08 19.99 7.78 4.41 8.2

all

SGM all – 65.58 36.08 20.66 16.24 3.0SGM – mind Lr(p, d) all – 66.79 38.35 23.32 18.36 3.1SGM – mind medianr Lr(p, d) all – 67.53 39.75 23.34 18.12 3.2SGM-SVM all M 60.89 32.59 20.31 16.16 323.7SGM-MLP all M 60.49 32.61 20.25 16.14 21.0

SGM-Forest

horiz+vert M 61.09 32.69 18.02 13.19 5.7top-down M 61.31 32.85 18.31 13.37 5.8bottom-up M 61.38 32.91 18.42 13.43 5.8

all M 60.28 32.15 17.90 13.14 6.1all • M 60.18 32.08 17.69 12.91 6.3all • • E 59.89 30.69 16.78 11.67 8.2all • • K 59.70 30.61 16.72 11.67 8.2all • • M 59.20 30.58 16.57 11.62 8.2


Table 2. Additional results for Table 2 in Section 5.2 in the main paper. This tableshows the validation performance for non-occluded (top) and all (bottom) using 3-foldcross-validation for different matching costs and datasets at different error thresholds.Our method (SGM-F.) outperforms baseline SGM in all settings.

Middlebury 2014 KITTI 2015 ETH3D 2017

Datacost Method 0.5px 1px 2px 4px 0.5px 1px 2px 4px 0.5px 1px 2px 4px

non-occluded

NCCSGM 54.15 28.59 15.23 10.14 59.70 32.28 13.09 6.17 30.94 14.78 8.62 5.67SGM-F. 50.06 25.29 12.55 8.08 51.61 24.74 9.22 4.17 21.14 10.28 5.59 3.67

MC-CNN-fastSGM 51.22 23.49 10.58 6.85 57.53 29.82 11.28 4.80 24.70 8.56 4.14 2.57SGM-F. 48.73 22.24 9.55 5.91 50.25 22.98 7.88 3.28 16.31 6.08 3.04 1.94

MC-CNN-acrt SGM 50.85 23.04 8.89 5.16 56.27 26.90 7.41 3.00 37.46 14.44 7.17 4.72SGM-F. 46.08 19.99 7.78 4.41 46.16 18.82 5.76 2.56 26.26 11.05 6.56 4.71

all

NCCSGM 69.23 42.36 27.96 22.25 60.59 33.79 15.06 8.34 32.52 16.71 10.66 7.69SGM-F. 64.00 37.22 22.85 17.09 52.39 25.80 10.11 4.69 22.48 11.26 6.36 4.35

MC-CNN-fastSGM 65.82 36.22 21.98 17.47 58.48 31.39 13.30 7.02 26.34 10.50 6.13 4.52SGM-F. 62.04 32.96 18.22 13.16 51.03 24.05 8.73 3.78 17.62 7.17 3.66 2.51

MC-CNN-acrt SGM 65.58 36.08 20.66 16.24 57.24 28.55 9.54 5.26 39.03 16.34 9.14 6.67SGM-F. 59.20 30.58 16.57 11.62 46.88 19.77 6.51 2.97 27.40 11.89 7.30 5.52


Table 3. This table shows the per-dataset performance for the ablation study in Table1 in the main paper. SGM here corresponds to row 1, SGM-Forest (raw) to row 10,and SGM-Forest (filt.) to row 13 in Table 1 in the main paper. Note that the numbersin Table 1 in the main paper are averaged using the Middlebury benchmark weighting(half weight for scenes PianoL, Playtable, Playroom, Shelves, and Vintage)

all non-occluded

Dataset Method 0.5px 1px 2px 4px 0.5px 1px 2px 4px

AdirondackSGM 61.97 32.25 13.02 8.04 58.96 26.92 6.47 1.68SGM-Forest (raw) 44.39 20.41 9.48 5.43 40.75 15.65 4.92 1.50SGM-Forest (filt.) 41.59 17.78 7.09 3.31 38.23 13.77 3.98 1.06

ArtLSGM 69.06 29.55 24.66 22.73 60.64 11.49 5.99 4.21SGM-Forest (raw) 60.83 29.57 20.62 16.99 52.57 15.43 5.97 3.30SGM-Forest (filt.) 60.27 29.39 20.61 15.88 52.97 16.73 7.33 3.55

JadeplantSGM 70.71 47.98 33.15 28.02 62.57 33.55 15.09 9.30SGM-Forest (raw) 57.24 39.74 31.68 26.57 45.53 23.84 14.62 9.23SGM-Forest (filt.) 56.53 38.72 30.39 24.74 45.80 23.94 14.60 8.81

MotorcycleSGM 60.53 31.23 14.00 11.33 55.97 23.40 4.62 2.37SGM-Forest (raw) 44.17 18.21 10.04 7.84 39.23 11.45 3.66 2.22SGM-Forest (filt.) 42.91 17.24 8.84 6.39 38.61 11.58 3.74 2.18

MotorcycleESGM 61.71 32.16 14.46 11.46 57.35 24.55 5.24 2.49SGM-Forest (raw) 44.29 18.53 10.14 7.94 39.46 12.06 3.97 2.34SGM-Forest (filt.) 42.95 16.96 8.78 6.38 39.06 11.68 3.94 2.25

PianoSGM 65.09 38.72 19.91 15.01 62.19 33.68 13.54 8.60SGM-Forest (raw) 48.49 26.54 17.31 12.23 44.59 21.42 12.58 8.35SGM-Forest (filt.) 47.89 26.40 16.57 11.27 43.94 21.29 11.92 7.64

PianoLSGM 68.87 44.65 26.00 20.46 66.27 40.06 20.03 14.32SGM-Forest (raw) 57.86 35.96 24.04 18.37 54.76 31.51 19.43 14.34SGM-Forest (filt.) 57.32 35.04 22.59 16.94 54.15 30.49 17.99 13.10

PipesSGM 62.90 35.34 19.33 17.24 55.95 23.45 5.01 3.14SGM-Forest (raw) 43.13 22.79 16.22 14.04 33.22 10.26 4.08 2.70SGM-Forest (filt.) 41.89 21.60 15.48 13.41 32.26 9.62 4.04 2.71

PlayroomSGM 69.82 46.35 25.60 18.44 65.04 37.88 14.07 6.22SGM-Forest (raw) 59.01 37.03 23.58 16.85 52.65 27.57 12.92 6.48SGM-Forest (filt.) 59.00 36.89 23.04 16.00 52.73 27.58 12.51 5.88

PlaytableSGM 65.50 36.95 21.32 15.16 61.53 29.72 12.60 6.10SGM-Forest (raw) 51.42 26.71 16.49 10.59 47.61 21.13 10.74 5.14SGM-Forest (filt.) 51.69 26.37 15.20 9.02 48.12 21.43 10.36 4.78

PlaytablePSGM 64.64 34.79 20.38 14.54 60.62 27.39 11.63 5.51SGM-Forest (raw) 49.97 24.67 15.42 9.91 46.35 19.26 9.92 4.73SGM-Forest (filt.) 50.41 24.50 14.46 8.59 47.00 19.61 9.70 4.41

RecycleSGM 63.64 34.19 15.66 10.63 60.88 29.36 9.75 4.84SGM-Forest (raw) 51.81 24.32 12.41 7.83 48.79 19.94 8.42 4.83SGM-Forest (filt.) 50.07 22.46 11.03 6.40 47.05 18.37 7.88 4.54

ShelvesSGM 75.40 55.59 39.45 31.38 73.85 52.80 35.77 27.39SGM-Forest (raw) 67.95 49.58 36.46 27.76 66.27 47.16 33.92 25.25SGM-Forest (filt.) 66.95 48.12 34.44 24.80 65.24 45.73 32.20 23.04

TeddySGM 64.62 20.14 13.62 11.94 60.55 11.06 3.91 2.23SGM-Forest (raw) 52.76 20.92 12.29 8.52 47.59 12.64 4.12 1.90SGM-Forest (filt.) 51.07 20.42 12.45 8.62 45.83 12.15 4.32 2.23

VintageSGM 70.22 45.74 27.83 18.62 67.58 40.94 21.65 11.96SGM-Forest (raw) 61.13 42.89 30.69 23.23 57.81 38.25 25.57 18.41SGM-Forest (filt.) 60.99 42.42 29.96 22.13 57.66 37.76 24.80 17.24


Table 4. Middlebury 2014 train/test split for the ablation study in Table 1 in Section5.2 in the main paper.

Train Test

Middlebury2005/Books Middlebury2014/Adirondack

Middlebury2005/Dolls Middlebury2014/ArtL

Middlebury2005/Laundry Middlebury2014/Jadeplant

Middlebury2005/Moebius Middlebury2014/Motorcycle

Middlebury2005/Reindeer Middlebury2014/MotorcycleE

Middlebury2006/Aloe Middlebury2014/Piano

Middlebury2006/Baby1 Middlebury2014/PianoL

Middlebury2006/Baby2 Middlebury2014/Pipes

Middlebury2006/Baby3 Middlebury2014/Playroom

Middlebury2006/Bowling1 Middlebury2014/Playtable

Middlebury2006/Bowling2 Middlebury2014/PlaytableP

Middlebury2006/Cloth1 Middlebury2014/Recycle

Middlebury2006/Cloth2 Middlebury2014/Shelves

Middlebury2006/Cloth3 Middlebury2014/Teddy

Middlebury2006/Cloth4 Middlebury2014/Vintage

Middlebury2006/Flowerpots

Middlebury2006/Lampshade1

Middlebury2006/Lampshade2

Middlebury2006/Midd1

Middlebury2006/Midd2

Middlebury2006/Monopoly

Middlebury2006/Plastic

Middlebury2006/Rocks1

Middlebury2006/Rocks2

Middlebury2006/Wood1

Middlebury2006/Wood2

Supplementary Material: Learning to Fuse Proposals from ... · Supplementary Material: Learning to...

Documents

Transcript of Supplementary Material: Learning to Fuse Proposals from ... · Supplementary Material: Learning to...