Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2....
Transcript of Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2....
![Page 1: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/1.jpg)
Differentiable Cloth Simulation for Inverse Problems
Junbang Liang
1
![Page 2: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/2.jpg)
Content● Motivation
● Related Work
● Our Method○ Simulation pipeline
○ Gradient Computation
● Results
2
![Page 3: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/3.jpg)
Motivation● Differentiable Physics Simulation as a Network Layer
○ Physical property estimation
○ Control of physical systems
3
![Page 4: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/4.jpg)
Motivation● Differentiable Physics Simulation as a Network Layer
○ Physical property estimation○ Control of physical systems
Yang et al. (2017) Demo of our differentiable simulation 4
![Page 5: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/5.jpg)
Content● Motivation
● Related Work
● Our Method○ Simulation pipeline
○ Gradient Computation
● Results
5
![Page 6: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/6.jpg)
Related Work● Differentiable rigid body simulation
○ Formulation not scalable to high dimensionality
Degrave et al. 2019
Belbute-Peres et al. 2019
6
![Page 7: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/7.jpg)
Related Work● Learning-based physics [Li et al. 2018]
○ Unable to guarantee physical correctness
7
![Page 8: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/8.jpg)
Our Contributions● Dynamic collision handling to reduce dimensionality
● Gradient computation of collision response using implicit differentiation
● Optimized backpropagation using QR decomposition
8
![Page 9: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/9.jpg)
Content● Motivation
● Related Work
● Our Method○ Simulation pipeline
○ Gradient Computation
● Results
9
![Page 10: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/10.jpg)
Introduction to Simulation● Partial differential equation (PDE) of Newton’s law:
● Solve satisfying , where
● Discretization to ordinary differential equations (ODE):
● Solve satisfying , where
10
![Page 11: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/11.jpg)
Introduction to Simulation● Partial differential equation (PDE) of Newton’s law:
● Solve satisfying , where
● Discretization to ordinary differential equations (ODE):
● Solve satisfying , where
11
![Page 12: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/12.jpg)
Point Cloud Simulation Flow1.
2.○ S
○ Newton’s method
3.
4.
12
![Page 13: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/13.jpg)
Cloth Simulation Flow1.
2.○ S
○ Newton’s method
3.
4.
5.
13
![Page 14: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/14.jpg)
Collision Response●
14
![Page 15: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/15.jpg)
Collision Response●
15
![Page 16: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/16.jpg)
Mesh Simulation Flow: Backpropagation
1.
2.○ S
○ Newton’s method
3.
4.
5.
Gradient computation available?
Handled by auto-differentiation
Handled by auto-differentiation
Handled by auto-differentiation
16
![Page 17: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/17.jpg)
Mesh Simulation Flow: Backpropagation
1.
2.○ S
○ Newton’s method
3.
4.
5.
Using implicit differentiation!
Gradient computation available?
17
![Page 18: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/18.jpg)
Implicit Differentiation: Linear Solve● Formulation:
● Input: and . Output:
● Back propagation: use to compute and ○ : the loss function.
18
![Page 19: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/19.jpg)
Implicit Differentiation: Linear Solve● Back propagation: use to compute and , where is the loss
function.
● Implicit differentiation form:
● Solution:
where is computed from , and is the solution of .
19
![Page 20: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/20.jpg)
Mesh Simulation Flow: Backpropagation
1.
2.○ S
○ Newton’s method
3.
4.
5. Using implicit differentiation!
Gradient computation available?
20
![Page 21: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/21.jpg)
Gradients of Collision?
21
![Page 22: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/22.jpg)
Collision Handling
● Objective formulation: Quadratic Programming
22
![Page 23: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/23.jpg)
Gradients of Collision Response● Karush-Kuhn-Tucker (KKT) condition:
● Implicit differentiation:
23
![Page 24: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/24.jpg)
Gradients of Collision Response● Solution:
● where dz and dλ is provided by the linear equation:
24
![Page 25: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/25.jpg)
Acceleration of Gradient Computation●
● Linear system of n+m○ n: DOFs in the impacts○ m: number of constraints/impacts
● Insight: Optimized point moves along the tangential direction w.r.t. constraint gradient
25
![Page 26: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/26.jpg)
Acceleration of Gradient Computation●
● Linear system of n+m○ n: DOFs in the impacts○ m: number of constraints/impacts
● Insight: Optimized point moves along the tangential direction w.r.t. constraint gradient
26
![Page 27: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/27.jpg)
Acceleration of Gradient Computation● Explicit solution of the linear equation:
where Q and R is obtained from:
● Theoretical speedup: O((n+m)³) → O(nm²)○ n: number of vertices○ m: number of constraints
27
![Page 28: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/28.jpg)
Content● Motivation
● Related Work
● Our Method○ Simulation pipeline
○ Gradient Computation
● Results
28
![Page 29: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/29.jpg)
Experimental Results● Ablation study
○ Backpropagation speedup
● Applications○ Material estimation
○ Motion control
29
![Page 30: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/30.jpg)
Ablation Study● Speed improvement in backpropagation● Scene setting: a large piece of cloth crumpled inside a pyramid
30
![Page 31: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/31.jpg)
Results● Speed improvement in backpropagation● Scene setting: a large piece of cloth crumpled inside a pyramid
The runtime performance of gradient computation is significantly improved by up to two orders of magnitude.
31
![Page 32: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/32.jpg)
Material Estimation● Scene setting: A piece of cloth hanging under gravity and a constant wind
force.
32
![Page 33: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/33.jpg)
Results● Application: Material estimation● Scene setting: A piece of cloth hanging under gravity and a constant wind
force.
Our method achieves the fastest speed and the smallest overall error.
33
![Page 35: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/35.jpg)
Motion Control● Scene setting: A piece of cloth being lifted and dropped to a basket.
35
![Page 36: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/36.jpg)
Results● Application: Motion control● Scene setting: A piece of cloth being lifted and dropped to a basket.
Our method achieves the best performance with a much smaller number of simulations.
36
![Page 38: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/38.jpg)
Conclusion● Differentiable simulation
○ Applicable to optimization tasks
○ Embedded in neural networks for learning and control
● Fast backpropagation for collision response
38
![Page 39: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/39.jpg)
Future Work● Optimization of the computation graph
○ Vectorization
○ PyTorch3D/DiffTaichi
● Integrate with other materials○ Rigid body, deformable body, articulated body, etc
39
![Page 40: Differentiable Cloth Simulation for Inverse Problems · Mesh Simulation Flow: Backpropagation 1. 2. S Newton’s method 3. 4. 5. Gradient computation available? Handled by auto-differentiation](https://reader033.fdocuments.net/reader033/viewer/2022061000/60af61c070665c72d51c8da8/html5/thumbnails/40.jpg)
Q&A
40