INFOMR Project Dafne van Kuppevelt ● Vikram Doshi ● Seçkin Savaşçı Development Review.
-
Upload
willa-stephens -
Category
Documents
-
view
220 -
download
0
Transcript of INFOMR Project Dafne van Kuppevelt ● Vikram Doshi ● Seçkin Savaşçı Development Review.
INFOMR Project
Dafne van Kuppevelt ● Vikram Doshi ● Seçkin Savaşçı
Development Review
Heat Kernel Signature
Before Development
• C# develop faster use .NET framework’s abilities our experience on C# and other similar languages(java)
• x86 architecture• Visual Studio IDE• Google Code for project hosting
Easy to start Complete package for project
management Support git, svn & mercurial http://code.google.com/p/infomr-group-x/
Initial Parsing
Getting Laplacian Matrices form Off Files
• L = D – A
Storage Problem
3 -1 0 -1 -1
-1 3 -1 0 -1
0 -1 3 -1 -1
-1 0 -1 3 -1
-1 -1 -1 -1 4
1
0 1
1 0 1
1 1 1 1
450002 * 32 = 7.54 GB 45000*44999/2*1= 120 MB
~64X Compression Computational Overhead
Initial Parsing
Storage Problem( continued)
Human Readable Matrix File Serialization to Files
Able to use in different language implementations
Language Specific
Storage Inefficient(string conversion,whitespaces, line feeds)
Storage Efficient
Own parsing method Deserialize and use!
Time Problem
Human Readable - Integer Serialization – Bitwise
~2 hours ~2 mins
Larger files cannot be parsed All can be parsed
Initial Parsing
Getting Eigen Values & Vectors from Laplacian Matrices
• We tried to implement our own eigen decomposer FAILED
• We started to search for a suitable library Our needs were:
- Structure for storing sparse symmetric matrices - Eigen decomposer method that is specialized for sparse symmetric matrices partially FAILED
We switched to trial & error for finding a good library, our goal was now speed and goodmemory usage.
Initial Parsing
library Spec Status Time Memory
Math.NET C# x86 Seq LAPACK xGPL 20+ mins Average
DotNumerics C# x86 Seq LAPACK port xGPL 2 mins Average
CenterSpace C# x86 Seq LAPACK $1295 30+ mins Good
Eigen C++ x86 Seq UNIQUE xGPL 2 hours Good
Experiment results with 2000 vertex model :
DotNumerics Structure for storing Symmetric matrices Eigen Decomposer for Symmetric matrices
~40 largest models are out of project scope due to time and memory problems
Initial Parsing
Time and Memory Problems
• Largest model will take more than 250 hours to parse• For the largest model we must have ~16 GB memory space because DotNumerics use Double precision for storing values
Overview of running on current architecture (x86): 2GB process dedicated memory /largeaddressaware 3 GB We cannot make injections due to LAPACK calls
It is impossible to reach beyond 3 GB address space on x86, theoretically 4 GB
Curious Cat?
Initial Parsing
PrototypeFrom To
Arch x86 x64
Virtual address space 3 GB 8 TB
Virtual Memory 4 GB 20 GB
Language C# C++
Library DotNumerics Armadillo(xGPL, tweaked for x64)
Base library LAPACK Intel MKL Parallel ( $129)
Other Library - Boost
Parallel , Used CPUs No, 1 Yes, 2
Optimization for Intel CPU No Yes
Other Memory Improvements - Memory Mapped File
Result : Eigen Decomposition time for 2000 vertex model? Guess?
Initial Parsing
Model Vertex Count Time Memory
2000 10 seconds Superb
Largest one (45000) 2 days ( Due to my 3GB main memory and memory mapped files, Disk I/O times become significant ; average 10 times slower for the best access)
Very bad because of memory mapping
Prototype Results
We didn’t change to develop on this prototype, because :
• Parsing the largest one is still infeasible• Not all of us have x64 architecture