Post on 21-Jan-2018
Just Dance 4
Ok, once more
Variation from (Victoria Zhou)
1. Step forward towards corner 2, into croisé derrière à terre, arms demi-seconde
2. Plié in 5th change direction to corner 13. Relevé derrière, arms in 4th en avant
palms down4. Passé and change direction to corner 2
place foot on pointe in 5th, arms move to a low kissing gesture
5. Retiré and back to 5th on pointe with the front foot, back foot and front foot arms move to demi-seconde
6. Repeat to the right, and repeat all.7. Posé coupé effacé towards corner 1,
arms in 4th en avant8. Posé coupé arms in 4th palms down9. Step and posé retiré croisé, place back
foot in 5th on pointe, port de bras with the left arm from 5th en haut to 5th enavant, end with the arms forward in a low line, cross the wrists.
One more time
What representation for dance?for humans
and machines Sci-Fi Dan
cing G
IF by C
olin
Raff
While for music we have good (machine readable) representations, we lack these for dance.
Why do we need knowledge representation for dance?
Three reasons
Archival and retrieval
Analysis: Digital humanities
Supporting creativity
Towards a choreography assistant tool
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
Existing representations and tools
Labanotation
LabanXML and Laban Editor
LED Labanotation editor http://donhe.topcities.com/pubs/led.htmlNakamura & Hachimura (2006)
Benesh
Dance Forms
Cecchetti system7 elementary movements: [plie (bend), etandre(stretch), releve (rise), sauter (jump), glisse(glide), tourne (turn), elancer (dart)]Positions: 1st, 2nd, 3rd,.. (left right croise)Facing position (1…8)Position in spaceDirection of movement (de cote, dessous, dessus, en avant, en arriere, devant, derriere)Combinations (100+) pas-de-chat, pas-de-bourre, piroutte
Ballet languages/systems
Based on interview with Marije Koning
XML Dance Grammar
Balakrishnan Ramadoss and Kannan Rajkumar. Modeling the Dance Video Semantics using Regular Tree Automata Fundamenta Informaticae 86 (2008) 175–189 175 IOS Press
Interactive Dance Choreography Assistance
Victor de BoerJosien Jansen Ana-Liza Tjon-A-Pauw
Frank Nack
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
To what extent can choreographers be supported by
semi-automatic dance analysis and the generation of new
creative elements in choreographies?
Method
Questionnaire: How do choreographers work (withtechnologies)
Tool: Proof of concept digital choreo assistant
Evaluation: Test application toand different strategies
Questionnaire
54 Dutch choreographers
Online questionnaire
Personal choreography archiving
0
2
4
6
8
10
12
14
16
18
writtendance
notation
digital dancenotation
videotaping other memory,without
problems
memory,forget things
Preferred Notations
0
5
10
15
20
25
30
35
Notations Laban & Benesh
0
5
10
15
20
25
30
Never heard ofit
Cannot workwith it
Other Know Laban Can write both
Interest in support in the creative process
Originality, Creativity and Emotion are most important aspects
One very negative sub-group> Afraid to lose humanity
One positive towardscreative assistance
Two sub-groups:
Tool Requirementsbased on MoSCoW method
• A dancer must be able to add their dance style to the tool• A dancer must be able to add their existing choreography to the tool• The tool must be able to give new suggestions for variations of the
choreography• The suggestions must be based on different strategies • The dancer must be able to see the whole choreography at any
moment in time (written)• The communication of the tool are written dance terms• The tool must be “easy to use”, which means getting suggestions may
take no longer than 2 minutes• The tool does have simplified body movements (legs, feet, arms, hands
and head)
Proof-of-concept mobile app
3 different dance styles
Ballet (including 78 steps)
Modern dance (including 57 steps)
Street dance (including 31 steps)
Dancepiration – a tool for choreography assistance
4 rule-based strategies for creating variations on existing choreographies
1. Random step replaced by random other step
2. Random step replaced by ontology-based other step
3. Random steps replaced by multiple strategies
4. Specific step replaced by ontology-based steps
Ontology-based variation for the 3 dance styles.
El Raheb, et al. BalOnSe: Ballet Ontology for Annotating and Searching Video performances. In Proceedings of the 3rd International Symposium on Movement and Computing (p. 5). ACM, 2016
Evaluation
Evaluation
6 choreography students
Random-based versus Ontology-based
Each dance style is tested 3 times with both strategies per person
Rate original choreography and each variation (10pt scale)
Rate on 5pt Likert scale: Correctness, Creativity, Helpfulness, Meaningfulness
Results
Respondents are positive about the tool
…prefer to choose a specific step to change themselves
… consider creativity in this tool very high (avg 4.2/5)
Correctness is important to improve, it influences other factors the most
Ontology-based variant outperforms random variations
Score OriginalRandomOntology-Based Difference SigAverage grade 6.17 5.50 6.35 +0.85 **Correctness 2.89 3.37 +0.48 *Creativity 3.19 3.37 +0.18
Helpfulness 2.59 3.00 +0.41
Meaningfulness 2.70 2.96 +0.26
** = statistically sign
ificant at α
=0.0
5 (t-test/an
ova)
Style matters* = statistically sign
ificant at α
=0.1
0 (t-test/an
ova)
** = statistically sign
ificant at α
=0.0
5 (t-test/an
ova)
Element Style Random Ontology-Based DifferenceCorrectness Ballet 2.89 2.56 -0.33
Streetdance 2.78 3.56 +0.78 *Modern 3.00 4.00 +1.00 **
Creativity Ballet 3.44 3.56 +0.12Streetdance 2.78 3.11 +0.33Modern 3.11 3.44 +0.33
Helpfulness Ballet 2.67 2.67 0.00Streetdance 2.44 2.89 +0.45Modern 2.89 3.44 +0.55
Meaningfulness Ballet 2.89 2.78 -0.11Streetdance 2.33 2.67 +0.34Modern 2.89 3.44 +0.55
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
Which presentation methods are considered most effective for an interactive dance choreography assistant tool?
Experiment: Comparing 4 choreography presentation methods
1: Textual descriptions
3: 3D animations
DanceForms 2 (http://charactermotion.com/products/danceforms/
4: auditory instructions
Experiment• 7 choreographers• 2 new(!) styles
– Hip-hop and dancehall
• Simple choreography + pre-generated variations
• Large projection screen in practice space• 4 presentation variations (random)
Overall assessment
Dancehall vs Hip-hop
0123456789
10
Textual 2Danimations
3Danimations
Auditory
Sco
re
0123456789
10
Textual 2Danimations
3Danimations
Auditory
Sco
re
0
1
2
3
4
5
6
7
8
9
10
Overall assessment Stimulation ofcreativity
Understandability (Un-)disruptiveness
Textual 2D an. 3D an. Auditory
3D animations are the best
A significant sub-group of choreographers is interested in and enthusiastic about automatic choreography support
Needs to be able to understand ‘dance language’
Knowledge representation matters
Style matters
Presentation styles matter -> 3D + dance language
Sensing
Representation
+Reasoning
Presentation
generation
• Motion detection
• Floor sensors
• Move recognition
• Dance movement representation
• Dance choreography representation
• Use of background knowledge
• Pattern detection
• Choreography generation
• Visual presentation
• 3-D animation
• Auditory presentation
Sensing
data
Choreography
variationPresentation
Choreography
Next level: Representation and Reasoning
Multi-tiered semantic model
Low-level image features
Atomic movements (Labanotation?)
Compound movements (100+ movements)
Emotional content, Socio-cultural layers etc.
Machine Learning for classification and pattern detection
Generative module (automatic choreographer)
Thank you
v.de.boer@vu.nl @victordeboer http://victordeboer.com
Sensing
• Motion capture– Marker-based
– Marker-less
• Joint rotations, limb positions etc.– unintuitive
• Backup: Video annotation
img: news.stanford.edu