Stair Captions and Stair ActionsNew Datasets for Deep Learning

Akikazu TakeuchiPrincipal Research ScientistStair Lab.

n

http://flowers.stair.center

n

n

n

n

http://flowers.stair.center

n

n

n

28

80

10

2017/03/12 2017

Stair Captions Stair Actions

16caption dataset

80

dataset

100,000

Stair CaptionsDetails

MS-COCO (Microsoft Common Objects in Context) 2014

n

• MS-COCO 2014

n

•

2017/03/12 2017

2017/03/12 2017

1

NLP2017

STAIR Captions:

2017/03/12 2017

Andrej Karpathy NeuralTalk2 Stair Captions

Available now!

Stair Captions

n http://captions.stair.center

2017/03/12 2017

Stair ActionsDetails

n

• 100n

• 1000 5

2017/03/12 2017

100

2017/03/12 2017

62,000

Stair Actions

n http://actions.stair.center

n

Stair Actions

“Stair Actions” is based on results obtained from a project commissioned by the New Energy and Industrial TechnologyDevelopment Organization (NEDO).

2017/03/12 2017

Stair Captions2DCNN + RNN

2017/03/12 2017

Donahue et al., “Long-term Recurrent Convolutional Networks for Visual Recognition and Description,” CVPR2015.

Visual Input

Visual FeaturesCNN

Sequence LearningLSTM

Predictions

UCF101 HMDB51 Ours[Donahue et al., CVPR 2015] 71.12 ― ―

[Feichtenhofer et al., NIPS 2016] 93.46 66.41 ―

86.04 52.87 45.6

n

n

n 2012 ImageNet 100

n

n

n

2017/03/12 2017

FAQ

n

n x

n →

n

n

n CC0

2017/03/12 2017

������ �"-2(0/�������#)54$���

��� "�!��

'0%,+5*"35&5�� "�$���0.1$��

���������

� � � � � � �

CreditWeb

n mokha

n

n

n

n

n Scaleout

n

Stair Actions

“Stair Actions” is based on results obtained from a project commissioned by the New Energy and Industrial TechnologyDevelopment Organization (NEDO).

2017/03/12 2017