Synthesizing 3d worlds
-
Upload
baskar-rethinasabapathi -
Category
Design
-
view
114 -
download
0
description
Transcript of Synthesizing 3d worlds
Synthesizing 3D Worlds
- Baskar Rethinasabapathi
Why do we need this?
- laborious task; saves designer time- Building an avg scene takes upto 20 minutes
Why do we need this?
- laborious task; saves designer time- Building an avg scene takes upto 20 minutes
- constant need for variety
Why do we need this?
- laborious task; saves designer time- Building an avg scene takes upto 20 minutes
- constant need for variety
Systems find it difficult
- High dimensional ; complex to model- Object properties differ
- Constraints based- Interactive and dependent
Existing approaches
- Procedural modeling - Should have a structural grammar
- Component based- Not suited for scenes
- Evolutionary- Repetitive; less creative
- Probabilistic- To be discussed
Potential application
– Creating indefinite virtual worlds
No man’s sky – Procedural
http://youtu.be/ZVl1Hmth3HE?t=28s
Problem domains addressed
System 1 - Example-based Synthesis of 3D Object Arrangements
- focus on local arrangements
System 2 - Synthesizing open worlds with constraints using locally annealed reversible jump MCMC
- focus on open world arrangements
Paper Source
1. http://dl.acm.org/citation.cfm?id=2366154
2. http://dl.acm.org/citation.cfm?id=2185552
High-Level Challenges
- output should be highly plausible/agreeable
- should generate a very large variety
- Users should not be involved much
General approaches
1. Example based
2. Constraints based
Dataset
- Objects created - from Google 3D warehouse
- by users and annotated
Goal of System 1
- Users should judge synthesized scenes to be highly plausible compared to hand created ones
- At least one out of every three synthesized results should be usable
System design 1
Scene -> Contextual + User example -> Learneddatabase Categories Input scenes Mixed Model
Probabilistic Mixed Model =
Occurrence model + Arrangement model
Goal of System 2
- Should be more applicable on open world layouts
- Optimal object arrangements which is diverse enough
System design 2
Scene database ->Extract system of constraints -> Generate Factor graphs
Repeated constraints in Scene -> novel MCMC method -> new scenes
System 1 - Algorithm
- Build a bipartite matching graph between objects in image
- Edge cost proportional to = same basic category+ smaller distance
- Max weight = Scene’s static support + Size of Objects + Edge cost
Clustering from the database
- Randomly choose anchor object – initialize a basic cluster
- Align with = neighborhood similarity + geometric features
- Merge clusters
Interesting sub problems
System 1
- Finding contextual neighborhood- How do we find a cookie on a plate in image A is interchangeable with bread
on image B?- Look for forks and knives around
- Similar in type and arrangement
- Finding hierarchical static support- Cookie should be on a plate, not on the bed
- Filling gaps in user examples- Construct a Bayesian network from database
System 2 - AlgorithmSystem 2
- Defining constraints (π1)- numPlates = randInt(1,10)
- Size_Table = Uniform(15,25)
- Position_Plates = uniform(-10,10),uniform(-10,10)
- Size_Plate = uniform(0.1,2)
- Building factor graphs- 70% of the table should be occupied (π2)
- softEq(0.7, total_area(plates)/ area(table))
- Plates should be inside the table (π3)- For all i, softEq(0.0, area_outside_table(plate_i))
- Total constraints model = π1* π2* π3
System 2 - AlgorithmSystem 2
- Markov Chain- Next state is dependent only on present state
- Monte Carlo Markov Chain- The system of Markov chains attains an equilibrium eventually
- Reversible Jump MCMC- Should accommodate trans-dimensional constraints
- Locally annealed Reversible Jump MCMC- Do not accept/reject new constraints immediately
Demo
Analogy with other papers
Bricolage - example based learning
Discussion
- How usable are the scenes generated from both systems?
- Why can’t System 1 be scaled to open world layouts?Or, vice versa. Can you apply System 2 for local arrangements?
Discussion
- Consider the Coffee Shop Layout. If I want all kinds of fabric in the scene to be changed to leather, how do I achieve?
- Which of the two systems is more suitable for this problem?
Results
- Yes. Goals achieved
- output should be highly plausible/agreeable- Should generate a very large variety- Users should not be involved much
- Users agreed with synthesized results of System 1
- No evaluation in System 2- Focus on optimization compared to other greedy algorithms
Critique