- 1. Niklas Nebel Sun Microsystems PERFORMANCE IMPROVEMENTS IN
CALC
2. Agenda
- Introduction and context 3. Local optimizations 4. Handling
sheets separately 5. DataPilot performance 6. Load & save
outlook
7. Introduction and Context 8. Performance work in all of
OOo
- Performance project
- Big improvements from 3.0 to 3.2
- Start-up: Cold start of Writer 20% faster 9. Writer load
performance
- Comparable with MS Word 2007
- Impress load performance
- Comparable with MS PowerPoint 2007
- Calc performance
- Load and save: Up to twice as fast 10. Recalculation: Up to 20
times faster (extreme case)
11. Local Optimizations 12. API Usage When Saving Text Cells
- Filter uses getFormula API method 13. Single quote character
added if text can be parsed as a number 14. Unnecessary parsing
step 15. Can take up to 17% of CPU time
16. Querying the Document Null Date
- Internal representation: Days since the null date 17. File
format: XML Schema dates ( ISO 8601) 18. Utility method for
conversion
- Queries the null date from the document 19. Several UNO
calls
- Querying once is enough 20. 10% of CPU time if only date cells
are used
21. Collecting Formatted Cell Ranges
- Collect cell ranges with equal cell formats
- For generating automatic styles 22. Keep a list of ranges for
each set of formats 23. Try to join adjacent ranges
- Formats are kept and iterated column-wise
- Can use this information when trying to join
- Prevents pathological cases
24. Formula Optimizations
- String handling when formuas are parsed
- Functions, references, names are case-insensitive 25.
Operators, separators, parentheses are not 26. Reduce case
conversion calls
- Sorting of values for MEDIAN etc.
- Not necessary to completely sort the array 27. Use
std::nth_element STL method instead 28. Faster calculation after
loading
29. Formula Recalculation (1)
- Detection of duplicate notifications
- When a cell range is modified 30. Parameter range can contain
several changed cells 31. Notify each range only once
- Also useful for single-cell change
- Parameter range can contain several changed results 32. Extreme
case: Issue 95967 20x faster
33. Handling Sheets Separately 34. Updating Row Heights
- Optimal row height depends on local conditions
- Core structures need concrete height values
- Positioning of shapes: Whole file
- File format: relative to cell position 35. Internally: absolute
positions
- Screen output: Only single sheet
- Update row heights
- After loading: Visible sheet and sheets with shapes 36. Others
as needed (display, printing, )
37. Updating Row Heights: Comments
- Cell comments (formerly: notes) are shapes 38. Often used in
large sheets
- Create shape only when comment is shown
- Saves time if there are many hidden comments 39. Row heights
can be updated later
40. Updating Row Heights: Results
- No effect for single sheet 41. Little improvement for text and
numbers 42. 30% CPU time with date cells on many sheets 43. Formula
results don't have to be calculated
44. Partial Saving
- Don't generate XML elements for whole file 45. Copy unchanged
parts on stream level 46. Could copy from temporary storage
- Storage layer creates copy of the unpacked file
- Access the original file
- Cost
- File access: Read the compressed file 47. CPU: Uncompress
48. Experiment: Incremental Saving
- Generate XML elements only for changed cells
- Proof of concept: Only single-cell changes
- No additional information kept after loading 49. Minimal
parsing to find affected cells in stream
- Takes extra time 50. Less if affected cells near start of
file
- Results (compared to 3.0):
- 40 70% improvement in CPU time 51. 30 50% improvement in total
time
52. Sheet-Wise Saving
- Handle sheets instead of individual cells 53. Fewer sheets than
cells
- Additional information can be kept in memory
- Easier to find modified sheets than modified cells 54. One
obvious limitation:
- Only useful with several sheets
55. Finding Modified Sheets
- Few code changes for most types of changes
- Formula notification for cell contents 56. Formula calculation
for changed results 57. Cell format changes 58. Column widths or
row heights 59. Handled separately: Print ranges, etc.
- Currently no handling of drawing layer changes
- All sheets are considered modified
60. Automatic Styles
- Direct formats are collected in automatic styles
- Referenced by name
- Generated name (ce1 etc.)
- One list for the whole document 61. Have to be created with the
same names again
- Implemented for cell contents (incl. comments)
- Keep a mapping of names to cell/text positions 62. Collect
styles for unchanged sheets first 63. Include in existing duplicate
detection for other sheets
- Sheets with shapes always saved normally
64. Putting the Parts Together
- When loading a file
- Compatibility checks: Namespaces, encoding 65. Keep stream
positions and style information
- Steps to save a spreadsheet document
- meta.xml, styles.xml, embedded objects: as usual 66.
content.xml
- Generate common content and modified sheets 67. For each sheet:
Generate or copy stream portion
- For Save and Save As update stream positions
68. Results
- Influencing factors
- Unchanged sheets 69. Type of sheet content 70. CPU time / file
access
- Example
- Text, numbers, dates 71. 16 sheets
- Single sheet modified
- Twice as fast 72. On top of other changes
73. Formula Recalculation (2)
- Sheet area is divided into slots
- 16 columns by 128 rows 74. Range dependency registered in all
affected slots 75. Needs attention when row limit is changed
- Change: Use hash_set instead of set
- Faster modification of dependency structures 76. Loading
time
- Change: Separate structures per sheet
- Faster recalculation if several sheets are used
77. DataPilot Performance 78. DataPilot Memory Usage
- Issue 55266: Several fields with many items 79. Fix now under
way from IBM Symphony team
- Don't allocate results for all child items 80. New cache
table
- CWS datapilotperf
- Planned for OOo 3.3 81. Combination of large fields no longer a
limitation
82. Load & Save Outlook 83. DOM Usage
- Prototype by Christian Lippka for Impress
- Use fast SAX to fill a compact DOM representation 84. Import
from DOM, possibly parallel to parsing
- Results for Impress
- Only 2% improvement for typical presentation 85. Filling DOM
tree uses 2% of CPU time 86. Not worth the effort
- Calc may be different
- Larger number of XML elements 87. But: Memory usage twice the
XML stream size
88. Further Separation of Sheets
- Load only the visible sheet
- Load other sheets as needed, or in background 89. Parse XML
fragment from stream, or use DOM 90. Formulas, charts may depend on
changed cells
- Dependencies must be known before saving
- Parse formulas only as needed
- Per sheet or individually 91. Already a separate step (but for
all formulas)
- Handle several sheets in parallel
- More fine-grained locking needed
92. Q & A 93. PERFORMANCE IMPROVEMENTS IN CALC Niklas Nebel
[email_address]