DIY Java Profiler
description
Transcript of DIY Java Profiler
![Page 2: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/2.jpg)
JVM diagnostic interfaces
• JMX
• JVMTI – native API only
• Attach API
Ad hoc instrumentation
and more
• Perf counters
• Heap dump
![Page 3: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/3.jpg)
MBeans: threading
CPU usage per thread (user / sys)
Memory allocation per thread
Block / wait times Should be enabled
Stack traces
Invaluable
![Page 4: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/4.jpg)
SJK: ttop
2014-10-01T19:27:22.825+0400 Process summary
process cpu=101.80%
application cpu=100.50% (user=86.21% sys=14.29%)
other: cpu=1.30%
GC cpu=0.00% (young=0.00%, old=0.00%)
heap allocation rate 123mb/s
[000037] user=83.66% sys=14.02% alloc= 121mb/s - Proxy:ExtendTcpProxyService1:TcpAcceptor:TcpProcessor
[000075] user= 0.97% sys= 0.08% alloc= 411kb/s - RMI TCP Connection(35)-10.139.200.51
[000029] user= 0.61% sys=-0.00% alloc= 697kb/s - Invocation:Management
[000073] user= 0.49% sys=-0.01% alloc= 343kb/s - RMI TCP Connection(33)-10.128.46.114
[000023] user= 0.24% sys=-0.01% alloc= 10kb/s - PacketPublisher
[000022] user= 0.00% sys= 0.10% alloc= 11kb/s - PacketReceiver
[000072] user= 0.00% sys= 0.07% alloc= 22kb/s - RMI TCP Connection(31)-10.139.207.76
[000056] user= 0.00% sys= 0.05% alloc= 20kb/s - RMI TCP Connection(25)-10.139.207.76
[000026] user= 0.12% sys=-0.07% alloc= 2217b/s - Cluster|Member(Id=18, Timestamp=2014-10-01 15:58:3 ...
[000076] user= 0.00% sys= 0.04% alloc= 6657b/s - JMX server connection timeout 76
[000021] user= 0.00% sys= 0.03% alloc= 526b/s - PacketListener1P
[000034] user= 0.00% sys= 0.02% alloc= 1537b/s - Proxy:ExtendTcpProxyService1
[000049] user= 0.00% sys= 0.02% alloc= 6011b/s - JMX server connection timeout 49
[000032] user= 0.00% sys= 0.01% alloc= 0b/s - DistributedCache
https://github.com/aragozin/jvm-tools/blob/master/sjk-core/COMMANDS.md#ttop-command
Available via PerfCounters
![Page 5: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/5.jpg)
MBeans: memory
• Memory geometry information
• Collection count
• Last collection details
for each collector
![Page 6: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/6.jpg)
SJK: GC
[GC: Copy#1806 time: 7ms interval: 332ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-325397.59kb/s] Tenured Gen: 162185k+14k->162199k[max:477888k,rate:42.22kb/s] Survivor Space: 235k-13k->222k[max:23872k,rate:-41.93kb/s]]
[GC: Copy#1807 time: 8ms interval: 338ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-319621.30kb/s] Tenured Gen: 162199k+219k->162418k[max:477888k,rate:648.30kb/s] Survivor Space: 222k-217k->4k[max:23872k,rate:-644.90kb/s]]
[GC: Copy#1808 time: 7ms interval: 321ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-336548.29kb/s] Tenured Gen: 162418k+0k->162418k[max:477888k,rate:0.00kb/s] Survivor Space: 4k-2k->1k[max:23872k,rate:-7.64kb/s]]
[GC: Copy#1809 time: 7ms interval: 321ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-336548.29kb/s] Tenured Gen: 162418k+0k->162418k[max:477888k,rate:0.00kb/s] Survivor Space: 1k+0k->1k[max:23872k,rate:0.24kb/s]]
[GC: Copy#1810 time: 4ms interval: 700ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-154331.43kb/s] Tenured Gen: 162418k+0k->162418k[max:477888k,rate:0.00kb/s] Survivor Space: 1k+288k->290k[max:23872k,rate:412.00kb/s]]
[GC: Copy#1811 time: 5ms interval: 311ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-347369.77kb/s] Tenured Gen: 162418k+0k->162418k[max:477888k,rate:0.00kb/s] Survivor Space: 290k-155k->135k[max:23872k,rate:-498.52kb/s]]
[GC: Copy#1812 time: 3ms interval: 340ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-317741.18kb/s] Tenured Gen: 162418k+0k->162418k[max:477888k,rate:0.00kb/s] Survivor Space: 135k-2k->132k[max:23872k,rate:-6.14kb/s]]
[GC: Copy#1813 time: 6ms interval: 325ms mem: Eden Space: 108032k-108032k->0k[max:191168k,rate:-332406.15kb/s] Tenured Gen: 162418k+0k->162418k[max:477888k,rate:0.00kb/s] Survivor Space: 132k+0k->133k[max:23872k,rate:0.65kb/s]]
Total
Copy[ collections: 28 | avg: 0.0065 secs | total: 0.2 secs ]
MarkSweepCompact[ collections: 0 | avg: NaN secs | total: 0.0 secs ]
https://github.com/aragozin/jvm-tools/blob/master/sjk-core/COMMANDS.md#gc-command
![Page 7: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/7.jpg)
JVM Attach API
• List JVM processes
• Attach to JVM by PID
• Send control commands
heap dump / histogram
stack dump
• Inspect system properties and VM options
• Launch instrumentation agents
https://github.com/gridkit/jvm-attach
![Page 8: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/8.jpg)
SJK: hh --dead
Dead object histogram
Similar to jmap –histo
Invoke jmap –histo two time
all heap objects
live heap object
calculates difference
Can show top N rows
https://github.com/aragozin/jvm-tools/blob/master/sjk-core/COMMANDS.md#hh-command
![Page 9: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/9.jpg)
SJK: hh --dead
1: 19117456 2038375696 [C
2: 9543865 441272568 [Ljava.lang.Object;
3: 13519356 432619392 java.util.HashMap$Entry
4: 12558262 301398288 java.lang.String
5: 7193066 287722640 org.hibernate.engine.spi.CollectionKey
6: 619253 160678888 [I
7: 4710497 113051928 org.jboss.seam.international.Messages$1$1
8: 571327 100876880 [Ljava.util.HashMap$Entry;
9: 1436183 57447320 org.hibernate.event.spi.FlushEntityEvent
10: 1661932 53181824 java.util.Stack
11: 209899 52047904 [B
12: 1624200 51974400 org.hibernate.engine.internal.Cascade
13: 929354 44608992 java.util.HashMap
14: 1812762 43506288 org.hibernate.i.u.c.IdentityMap$IdentityMapEntry
15: 850157 34006280 java.util.TreeMap$Entry
16: 1044636 25071264 java.util.ArrayList
17: 1340986 23423328 [Ljava.lang.Class;
18: 710973 22751136 java.io.ObjectStreamClass$WeakClassKey
19: 885164 21243936 org.hibernate.event.internal.WrapVisitor
20: 885126 21243024 org.hibernate.event.internal.FlushVisitor
...
Total 95197823 4793878008
https://github.com/aragozin/jvm-tools/blob/master/sjk-core/COMMANDS.md#hh-command
![Page 10: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/10.jpg)
SJK: jps
JDK’s jps on steroid
Uses attach API
Lists VMs
Filtering by JVM system properties
Prints property values
Prints effective –XX options
https://github.com/aragozin/jvm-tools/blob/master/sjk-core/COMMANDS.md#jps-command
![Page 11: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/11.jpg)
SJK: jps
My favorite command
> sjk jps -pd PID MAIN duser.dir XMaxHeapSize
90543 sjk-0.3.1-SNAPSHOT.jar /var/vas_sdk_test_server -XX:MaxHeapSize=32126271488
5315 WrapperSimpleApp /var/vas_sdk_test_server/vas-sdk-test-13030 -XX:MaxHeapSize=4294967296
11094 WrapperSimpleApp /var/vas_sdk_test_server/vas-sdk-test-13020 -XX:MaxHeapSize=4294967296
993 Main /var/gedoms-uat/private/rtdb_1 -XX:MaxHeapSize=12884901888
56603 AxiomApplication /var/gedoms-uat/private/gedoms_1 -XX:MaxHeapSize=2147483648
24046 WrapperSimpleApp /var/sonar/sonar-3.6.2/bin/linux-x86-64 -XX:MaxHeapSize=536870912
https://github.com/aragozin/jvm-tools/blob/master/sjk-core/COMMANDS.md#jps-command
![Page 12: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/12.jpg)
Perf counters
Based on shared memory
safe for target JVM
Flat data model
misc JVM counters
true GC CPU usage data
you can add own counter programmatically
![Page 13: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/13.jpg)
Stack Trace Sampling
Capture
• Dump stack traces via local connection
• Store in highly compressed dump
Analysis
• Frame frequency
• Conditional frame frequency
• Traces classification histogram
![Page 14: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/14.jpg)
Stack Trace Sampling
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Base
Other
DefaultServlet.doGet
LifeCycleImpl (render)
LifeCycleImpl (execute)
Business logic
Seam interceptor (lock contention)
Seam interceptor (inject/outject)
Resource bundle (getObject)
Resource bundle (missing)
Facelet compile
Hibernate (rest)
Hibernate (autoFlush)
JDBC
![Page 15: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/15.jpg)
Working with heap dumps
Java API to traverse heap dump object graphAvailable at https://github.com/aragozin/jvm-tools/tree/master/hprof-heap
Based on NetBeans profiler library
No temporary files used
Fixed generic method signatures
Improved performance
Useful for In-place processing of large heap dumps
Write domain specific heap usage reports
![Page 16: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/16.jpg)
SJK Summary
Visit https://github.com/aragozin/jvm-tools
Single executable JAR
Command line interface
Exploits JMX / Attach API / PerfCounters
Simple sampling profiler included
Extensible commands
Write commands for your own application
![Page 17: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/17.jpg)
BTrace
Visit https://kenai.com/projects/btrace
Instrumentation profiling
Inject code snippets written in Java
CLI or Java API to use
Extendible
![Page 18: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/18.jpg)
BTrace
@OnMethod(clazz = "org.jboss.seam.Component",
method = "/(inject)/")
void entryByMethod2(@ProbeClassName String className, @ProbeMethodName String methodName,
@Self Object component) {
if (component != null) {
Field nameField = field(classOf(component), "name", true);
if (nameField != null) {
String name = (String)get(nameField, component);
Profiling.recordEntry(bench,
concat("org.jboss.seam.Component.", concat(methodName, concat(":", name))));
}
}
}
@OnMethod(clazz = "org.jboss.seam.Component",
method = "/(inject)/",
location = @Location(value = Kind.RETURN))
void exitByMthd2(@ProbeClassName String className, @ProbeMethodName String methodName,
@Self Object component, @Duration long duration) {
if (component != null) {
Field nameField = field(classOf(component), "name", true);
if (nameField != null) {
String name = (String)get(nameField, component);
Profiling.recordExit(bench,
concat("org.jboss.seam.Component.", concat(methodName, concat(":", name))), duration);
}
}
}
![Page 19: DIY Java Profiler](https://reader034.fdocuments.net/reader034/viewer/2022052508/559444f31a28abf32f8b47f1/html5/thumbnails/19.jpg)
Thank you
Alexey [email protected]
http://blog.ragozin.info- my articleshttp://aragozin.timepad.ru- IT community events in Moscow