Life with big Firebird databases
-
Upload
nataly-polyanskaya -
Category
Technology
-
view
1.497 -
download
1
description
Transcript of Life with big Firebird databases
![Page 2: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/2.jpg)
2
• Tools and consulting• Platinum Sponsor of Firebird Project• Founded in 2002: 12 years of
Firebird and InterBase recoveries and consulting
• Based in Moscow, Russia• Firebase is our Brazil partner (since
2006)
![Page 3: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/3.jpg)
3Big databases
Series of performance tests with databases 9Gb... 30Gb... 1.7Tb
http://ib-aid.com/en/articles/firebird-performance-degradation-tests-myths-and-truth/
![Page 4: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/4.jpg)
4Non-optimized performance results
![Page 5: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/5.jpg)
5
Optimized test results
![Page 6: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/6.jpg)
6
What we did
• SuperServer → Classic or SuperClassic• Tuned Firebird config parameters
Optimized config are available here:http://ib-aid.com/en/optimized-firebird-configuration/
![Page 7: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/7.jpg)
7The difference between big and small databases
• Small– Optimize your SQLs– In case of any trouble
do backup/restore (gbak)
• Big– Where is the problem?– Why it happens?– Backup/restore takes 1
days... Will it solve the problem or it will be waste of time?
![Page 8: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/8.jpg)
8
Maintenance tasks
• Backups• Performance monitoring
– Transactions– SQLs and plans– Database structure
![Page 9: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/9.jpg)
9
Backups
• Usual backup: new backup rewrites old backup
Firebird database
New backup
Old backup
![Page 10: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/10.jpg)
10
Wrong!• If backup fails, there will be no good old
backup, no new backup, and corrupted database
Firebird database
New backup
Old backup
![Page 11: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/11.jpg)
11Correct schema
Firebird database
New backup
Old backupFinished backup
Database space for backups:(Number_of_stored_backups+1)*Size
![Page 12: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/12.jpg)
12
Gbak vs Nbackup
• NBACKUP+ fast- does not check data- not reliable till 2.5.2- delta problems
• Gbak+ checks data- slow (and speed degrades)- test restore needed
![Page 13: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/13.jpg)
13
Big databases backup schema
Production server Backup server
Nbackup gbak
Requirements● delta monitoring● health monitoring
Requirements:● gbak backup ● test restore
![Page 14: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/14.jpg)
14
Big databases backup schema
Production server Backup server
Nbackup gbak
FBDataGuard 1 FBDataGuard 2
![Page 15: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/15.jpg)
15
FBDataGuard
• Backups (in correct way)• Health checks of the database• Performance parameters monitoring
– Transactions– Users, Indices, database statistics
• Corruption monitoring• Email alerts
![Page 16: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/16.jpg)
16
Health monitoring
• Critical metadata check and backup– Can be used for recovery with FirstAID
• Database limitations checks– Transactions limits, format limits, space limits
• Errors– Firebird.log
![Page 17: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/17.jpg)
17
Indices maintenance
• Indices health– Idea of statistics – Recalculation of indices statistics
• Manual• Automatic
![Page 18: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/18.jpg)
18
Demo
![Page 19: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/19.jpg)
19
Performance monitoring
![Page 20: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/20.jpg)
20
Tasks and Tools
• SQL queries and plans: FBScanner and FBPerfMon
• Transactions: FBDataGuard (alerts, database statistics) and IBTM (visualization)
• Database structure analysys: IBAnalyst
![Page 21: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/21.jpg)
21Recommended setup for
monitoring
Firebird Production server
Users
FBScanner remote server
Administrator has:1) FBScanner Analyzer2) MON$Logger3) TraceAPI4) IBTM5) IBAnalyst
MON$
TraceAPI
FBDataGuard& Trace API
![Page 22: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/22.jpg)
22Examples of problem resolution
Long running active transaction — prevent performance problem1.Administrator got email from FBDataGuard about
transaction gap2.IBTM shows long running active transaction3.MON$ logger identified source of transaction, and
admin disconnected client
![Page 23: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/23.jpg)
23
Transactions monitoring
• Transaction markers (NEXT, OIT, OST, OAT) dynamics is good indicator of problems– Long running active transactions (stuck OAT)– Forced Rollback (stuck OIT)– Sweep and autosweep success
![Page 24: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/24.jpg)
24
![Page 25: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/25.jpg)
25MON$
![Page 26: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/26.jpg)
26Examples of problem resolutionBad performance — fix performance problem– User reported problem with performance in application A– Quick check with MON$logger does not show the
problem– Admin analysed logs from FBScanner (permanent
monitoring) and found bad query, reported to developer, with SQL plan and fresh gstat statistics from FBDataGuard
– Developer checked gstat and plan, found that new index is needed
– Admin applied index, performance is good again
![Page 27: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/27.jpg)
27
SQL Queries and Plans
• For all Firebird versions — FBScanner– Works as a proxy– Connection string changes required– Can be installed on remote server
• For Firebird 2.5 — FBPerfMon– Mon$ support– TraceAPI support
![Page 28: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/28.jpg)
28Trace
![Page 29: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/29.jpg)
29
FBScanner Log
![Page 30: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/30.jpg)
30Database structure
![Page 31: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/31.jpg)
31
Conclusion
• To sleep well and live long you need:– Relialble backups– Continious monitoring and alerts– Quick identification of problem– Full details how to fix it
![Page 32: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/32.jpg)
32
Backups
Health monitoring
Indices maintenance
Transactions dynamics
SQL Queries & plans
Database structure analysys
Database
Typical tasks
![Page 33: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/33.jpg)
33
FBDataGuard
IBTM
FBScanner & FBPerfMon
IBAnalyst
![Page 34: Life with big Firebird databases](https://reader034.fdocuments.net/reader034/viewer/2022050801/5495b17ab47959794d8b4e43/html5/thumbnails/34.jpg)
34
Only at FDD 2014
IBSurgeon Optimization Pack 5-in-1 (FBDataGuard+FBScanner+FBPerfMon +IBTM +IBAnalyst)
• EUR 299 R$400 per server• EUR 1290 R$850 Unlimited Subscription 1 year• EUR 349 R$400 FirstAID 50 database