Performance & JVM & GC 1 of 36
Performance & JVM & GC Tips & Tricks & Troubleshoot
Table of Content --------------------------------------------------
Performance 3
Performance Optimization Architecture 3
Memory Leak Types 4
Top 10 Most Common Java Performance Problems 5
DataBase 5
Persistence Configuration 5
Caching 5
Pool Connections 5
Memory Optimization 5
Garbage Collector 5
Concurrency 5
Address Memory Leaks 5
J2EE Application Performance Optimization 6
Overview 6
Identify Bottlenecks 6
Use Performance Monitors Error! Bookmark not defined.
Application Server Tuning 8
Database Tuning 8
Code optimization 8
10 Application Performance Tuning Tips (Architect perspective) 8
Define the requirements (benchmarking) 8
Measure don’t guess 9
Automate 9
Only Optimize if Needed 9
Learn to Parallelize 9
Learn to Scale 9
Cache It 9
Right Abstraction 9
Top 10 Causes of Java EE Enterprise Performance Problems 10
Lack of proper capacity planning 10
Performance & JVM & GC 2 of 36
Inadequate Java EE middleware environment specifications 12
Too many or poor integration with external systems 13
Lack of proper database SQL tuning & capacity planning 13
Application specific performance problems 13
Java EE middleware tuning problems 13
Insufficient proactive monitoring 13
Saturated hardware on common infrastructure 14
Network latency problems 15
Solving common Java EE performance problems 17
Out-of-memory errors 17
Java Memory problems 17
Problem Groups 17
Memory Leaks 17
Unnecessarily high memory usage 17
Inefficient object creation 17
Inefficient garbage collector behaviour 17
HTTP Session as Cache 18
ThreadLocal Memory Leak 19
Large Temporary Objects 20
Bad Garbage Collector Configuration 20
ClassLoader Leaks 20
Conclusion 21
Common Performance problems 21
Commonly seen reasons behind performance issues 22
Memory management algorithms/management in java 22
Explain types of references in Java? 23
Talk about garbage collector for various references! 27
Explain garbage collection on Remote Objects or Distributed Garbage collection. 28
Does OutOfMemoryError and StackOverFlowError cause JVM crash? 29
Different OutOfMemory errors!! 30
Why does the JVM crash with a core dump or a Dr.Watson error? 31
Memory Leaks Examples 31
Auto Boxing 31
JVM & Garbage Collection 34
What flags can I use to tune the JVM and GC? 34
Performance & JVM & GC 3 of 36
Performance & JVM & GC Tips & Tricks & Troubleshoot
--------------------------------------------------
Performance
Performance Optimization Architecture
____________________________________________________________________________
Performance & JVM & GC 4 of 36
Memory Leak Types
____________________________________________________________________________
Performance & JVM & GC 5 of 36
Top 10 Most Common Java Performance Problems
● DataBase ○ Connection Pools
○ Indexes & Defragmentation
○ Views
○ Concurrency Controls (Optimistic & Pessimistic)
○ DB optimization techniques i.e. Tablespaces in Oracle
● Persistence Configuration ○ Lazy Loading (Right Fetch Types)
○ Enable Hibernate statistics & Improve Slow queries
○ Second Level Cache
○ Avoid N + 1 problems
○ Prefer Batch operations
● Caching ○ ORM Caches
○ Application Cache like Oracle Coherence
● Pool Connections ○ Database
○ Thread pool
○ EJB pools
● Memory Optimization
● Garbage Collector ○ Type of GC
● Concurrency
● Address Memory Leaks
____________________________________________________________________________
Performance & JVM & GC 6 of 36
J2EE Application Performance Optimization
● Overview
○ Set Goal
○ Identify Problem Area
○ Follow Methodical & Focussed Path
● Identify Bottlenecks
○ Use load runners & stress test tools
● Use Performance Monitors
Performance & JVM & GC 7 of 36
○ Perfmon tools in windows OS
Performance & JVM & GC 8 of 36
● Application Server Tuning
○ Memory
○ GC tuning
○ Connection & Thread & EJB pools
○ Pre compiled JSP objects
○ NO hot deployment
● Database Tuning
○ SQL query optimization
○ Short transactions
○ Avoid select *
○ Avoid distinct
○ Avoid String operations like concat etc
○ Use Indices whenever possible
● Code optimization
○ Avoid sync blocks
○ Logging less or async log in production
○ Instead of create & Destroy, try to use object pool etc
○ Less overhead on HTTP session
○ Use HTTP forward instead of sendredirect (involves browser roundtrip)
____________________________________________________________________________
10 Application Performance Tuning Tips (Architect perspective)
● Define the requirements (benchmarking)
Performance requirements should be defined SMART
(https://en.wikipedia.org/wiki/SMART_criteria) – like “95% of the Login requests should respond
in less than 2 seconds measured on the web server“.
Performance & JVM & GC 9 of 36
● Measure don’t guess
This tip is easy: You need a tool to measure and optimize your code. For Java this can be
included tools like jstat or jvisualvm. I prefer a good profiler like JProfiler and an APM solution
like AppDynamics for production analysis.
● Automate
Continuous Delivery pipeline which includes profiling and performance analysis of acceptance
and load tests and a performance report for each build/release – including comparison of
performance metrics for two builds.
● Only Optimize if Needed
● Learn to Parallelize
○ Scale Up
● Learn to Scale
○ Scale Out
○ Clusters
○ CDN (Functional Scalability)
● Cache It
○ Distributed Cache like MemCached or Oracle’s Coherence
○ CDN (Functional Scalability)
● Right Abstraction
This is also a simple rule: Don’t be too abstract in your application design.
E.g. don’t create a layer above your database that abstracts away all special features of
Oracle/DB2/MSSQL/……this will make your application slow, as you are not using the features
you have paid for. Yes, I know you did this to have less effort to replace the database if
needed…but believe me, this will not happen in the next 10 years and if it happens you will
have other problems…
____________________________________________________________________________
Performance & JVM & GC 10 of 36
Top 10 Causes of Java EE Enterprise Performance Problems
● Lack of proper capacity planning
Capacity planning can be defined as a comprehensive and evolutive process measuring and
predicting current and future required IT environment capacity. A proper implemented capacity
planning process will not only ensure and keep track of current IT production capacity and
stability but also ensure that new projects can be deployed with minimal risk in the existing
production environment. Such exercise can also conclude that extra capacity (hardware,
middleware, JVM, tuning, etc.) is required prior to project deployment.
In my experience, this is often the most common "process" problem that can lead to short- and
long- term performance problems. The following are some examples.
Performance & JVM & GC 11 of 36
Performance & JVM & GC 12 of 36
● Inadequate Java EE middleware environment specifications
The second most common cause of performance problems I have observed for Java EE
enterprise systems is an inadequate Java EE middleware environment and / or infrastructure.
Not making proper decisions at the beginning of new platform can result in major stability
problems and increased costs for your client in the long term. For that reason, it is important to
spend enough time brainstorming on required Java EE middleware specifications. This exercise
should be combined with an initial capacity planning iteration since the business processes,
expected traffic, and application(s) footprint will ultimately dictate the initial IT environment
capacity requirements.
Now, find below typical examples of problems I have observed in my past experience:
● Deployment of too many Java EE applications in a single 32-bit JVM
● Deployment of too many Java EE applications in a single middleware domain
● Lack of proper vertical scaling and under-utilized hardware (e.g., traffic driven by one or
just a few JVM processes)
● Excessive vertical scaling and over-utilized hardware (e.g., too many JVM processes vs
available CPU cores and RAM)
● Lack of environment redundancy and fail-over capabilities
● Trying to leverage a single middleware and / or JVM for many large Java EE applications
can be quite attractive from a cost perspective. However, this can result in an operation
nightmare and severe performance problems such as excessive JVM garbage collection
and many domino effect scenarios (e.g., Stuck Threads) causing high business impact
Performance & JVM & GC 13 of 36
(e.g., App A causing App B, App C, and App D to go down because a full JVM restart is
often required to resolve problems).
● Too many or poor integration with external systems
○ Like Rest, JDBC, EJB calls - use appropriate connection pool mechanism, adequate
number of calls etc
○ Timeout configuration
● Lack of proper database SQL tuning & capacity planning
● Application specific performance problems
○ Correct Collections & Algorithms
○ Object Scopes
○ Mutables
○ Singletons
○ Avoid Sync blocks
○ Lack Of Data Caching
○ Excessive Logging
● Java EE middleware tuning problems
○ Configure Thread pools
○ Connection Pools
○ EJB Pools
○ Thin Sessions
○ JDBC tuning
○ JMS tuning
● Insufficient proactive monitoring
Lack of monitoring is not actually "causing" performance problems, but it can prevent you from
understanding the Java EE platform capacity and health situation. Eventually, the environment
can reach a breakpoint, which may expose several gaps and problems (JVM memory leak, etc.).
Performance & JVM & GC 14 of 36
From my experience, it is much harder to stabilize an environment after months or years of
operation as opposed to having proper monitoring, tools, and processes implemented from day
one.
That being said, it is never too late to improve an existing environment. Monitoring can be
implemented fairly easily. My recommendations follow.
● Review your current Java EE environment monitoring capabilities and identify
improvement opportunities.
● Your monitoring solution should cover the end-to-end environment as much as possible;
including proactive alerts.
● The monitoring solution should be aligned with your capacity planning process
discussed in our first section.
● Saturated hardware on common infrastructure
Another common source of performance problems is hardware saturation. This problem is often
observed when too many Java EE middleware environments along with its JVM processes are
deployed on existing hardware. Too many JVM processes vs. availability of physical CPU cores
can be a real problem killing your application performance. Again, your capacity planning
process should also take care of hardware capacity as your client business is growing.
My primary recommendation is to look at hardware virtualization. Such an approach is quite
common these days and has quite a few benefits such as reduced physical servers, data center
size, dedicated physical resources per virtual host, fast implementation, and reduced costs for
your client. Dedicated physical resources per virtual host is quite important since the last thing
you want is one Java EE container bringing down all others due to excessive CPU utilization.
Performance & JVM & GC 15 of 36
● Network latency problems
Our last source of performance problems is the network. Major network problems can happen
from time to time such as router, switch, and DNS server failures. However, the more common
problems observed are typically due to regular or intermittent latency when working on a highly
distributed IT environment. The diagram below highlights an example of network latency gaps
between two geographic regions of a Weblogic cluster communicating with an Oracle database
server located in one geographic region only.
Performance & JVM & GC 16 of 36
Intermittent or regular latency problems can definitely trigger some major performance
problems and affect your Java EE application in different ways.
● Applications using database queries with large datasets are fully exposed to network latency
due to high number of fetch iterations (back and forward across network).
● Applications dealing with large data payloads (such as large XML data) from external
systems are also exposed to network latency that can trigger intermittent high-response
time when sending and receiving responses.
● Java EE container replication process (clustering) can be affected and put at risk its failover
capabilities (e.g., multicast or unicast packet losses).
Tuning strategies such as JDBC row data "prefetch", XML data compression, and data
caching can help mitigate network latency. But such latency problems should be reviewed
closely when first designing the network topology of a new IT environment.
____________________________________________________________________________
Performance & JVM & GC 17 of 36
Solving common Java EE performance problems
● Out-of-memory errors
One of the most common problems that plagues enterprise applications is the dreaded
OutOfMemoryError. The error is typically followed by one of the following:
● An application server crash
● Degraded performance
● A seemingly endless loop of repeated garbage collections that nearly halts processing
and usually leads to an application server crash
Regardless of the symptoms, you will most likely need to reboot the application server before
performance returns to normal.
____________________________________________________________________________
Java Memory problems
Problem Groups
● Memory Leaks
in Java are created by referencing objects that are no longer used. This easily happens when multiple
references to objects exist and developer forget to clear them, when the object is no longer needed.
● Unnecessarily high memory usage
aused by implementations consuming too much memory. This is very often a problem in web
applications where a large amount of state information is managed for “user comfort”. When the
number of active users increases, memory limits are reached very fast. Unbound or inefficiently
configured caches are another source of constant high memory usage.
● Inefficient object creation
easily results in a performance problem when user load increases, as the garbage collector must
constantly clean up the heap. This leads to unnecessarily high CPU consumption by the garbage
collector. As the CPU is blocked by garbage collection, application response times increases often
already under moderate load. This behaviour is also referred to as GC trashing.
● Inefficient garbage collector behavior
is caused by missing or wrong configuration of the garbage collector. The garbage collector will
take care that object are cleaned up. How and when this should happen must however be
configured by the programmer or system architect. Very often people simply “forget” to
Performance & JVM & GC 18 of 36
properly configure and tune the garbage collector. I was involved in a number of performance
workshops where a “simple” parameter change resulted in a performance improvement of up
to 25 percent.
In most cases memory problems affect not only performance but also scalability. The higher the
amount of consumed memory per request, user or session the less parallel transactions can be
executed. In some cases memory problems also affect availability. When the JVM runs out of memory or
it is close to memory limits it will quit with an OutOfMemory error. This is when management enters
your office and you know you are in serious trouble.
Memory problems are often difficult to resolve for two reasons: In some case analysis will get complex
and difficult – especially if you are missing the right methodology to resolve them. Secondly their
foundation is often in the architecture of the application. Simple code changes will not help to resolve
them.
In order to make life easier I present a couple of memory antipatterns which are often found in real
world world applications. Those patterns should help to be able to already avoid memory problems
during development.
HTTP Session as Cache
This antipattern refers to the misuse of the HTTPSession object as a data cache. The session object
serves as means to store information which should “survive” a single HTTP request. This is also referred
to a as conversational state. Meaning data is stored over a couple of requests until it is finally processed.
This approach can be found in any non-trivial web application. Web applications have no other means
than storing this information on the server. Well, some information can be put into the cookie, but this
has a number of other implications.
It is important to keep as few data as possible and as short as possible. It can easily happen that the
session contains megabytes of object data. This immediately results in high heap usage and memory
shortages. At the same time the number of parallel users is very limited. The JVM will respond to an
increasing number of users with an OutOfMemoryError. Large user sessions have other performance
penalties as well. In case of session replication in clusters increased serialization and communication
effort will result in additional performance and scalability problems.
In some projects the answer to this kind of problems is increasing the amount of memory and switching
to 64bit JVMs. They cannot resisit the temptation of just increasing heap size up to several gigabytes.
However this is often only hiding symptoms than providing a cure to the real problem. This “solution” is
only temporal and also introduces a new problem. Bigger and bigger heaps make it more difficult to find
“real” memory problems. Memory dumps for very large heaps (greater 6 gigabytes) cannot be
processed by most available analysis tools. We at dynaTrace invested a lot of R&D effort to be able to
Performance & JVM & GC 19 of 36
efficiently analyze large memory dumps. As this problem is gaining more and more importance a new
JSR specification is also addressing it.
Session caching problems often arise because the application architecture has not been clearly defined.
During development data is simply put into the session as it is comfortable. This very often happens in
an “add and forget” manner, as nobody ensures that this data is removed when no longer needed.
Normally unneeded session data should be handled by the session timeout. In enterprise applications
which are constantly under heavy use the session timeout, this will not work. Additionally very often
very high session timeouts are used – up to 24 hours – to provide additional “comfort” to users so that
they do not have to login again.
A practical example is putting selection choices from list, which have to be fetched from the database, in
the session. The intention is to avoid unnecessary database queries. (Smells like premature optimization
– doesn’t it). This results in several kilobytes being put into the session object for every single user.
While it is reasonable to cache this information the user session is definitely the wrong place for it.
Another example is abusing the Hibernate session for managing conversational state. The Hibernate
session object is simply put into the HTTP session to be have fast access to data. This however results in
much more data to be stored as necessary and the memory consumption per users rises significantly.
In modern AJAX applications conversational state can also be managed at the client side. Ideally this
leads to a stateless or nearly stateless server application which also scales significantly better.
ThreadLocal Memory Leak
ThreadLocal variables are used in Java to bind variables to a specific thread. This means every thread
gets it’s own single instance. This approach is used to handle status information within a thread. An
example would be user credentials. The lifecycle of a ThreadLocal variable is however related to the
lifecycle of the thread. ThreadLocal variables are cleaned up when the thread is terminated and
removed by the garbage collector – if not explicitly removed by the programmer.
Forgotten ThreadLocal variables can especially in application servers easily result in memory problems.
Application servers uses ThreadPools in avoid constant creation and destruction of threads. An
HTTPServletRequest for example gets a free thread assigned at runtime, which is passed back to the
ThreadPool after execution. If the application logic uses ThreadLocal variables and forget to explicitly
remove them, the memory will not be freed up.
Depending on the pool size – in production systems this can be several hundred threads – and the size
of the objects reference by the ThreadLocal variable this can lead to problems. A pool of 200 threads
and a ThreadLocal size of 5MB will in the worst case lead to 1 GB of unnecessarily occupied memory.
Performance & JVM & GC 20 of 36
This will immediately result in high GC activity leading to bad response times and potentially to an
OutOfMemoryError.
A practical example was a bug in jbossws version 1.2.0 which was fixed in version 1.2.1 – “DOMUtils
doesn’t clear thread locals”. The problem was a ThreadLocal variable which referenced a parsed
document having a size of 14 MB.
Large Temporary Objects
Large temporary objects can in the worst case also lead to OutOfMemoryErrors or at least to high GC
activity. This will for example happen if very big documents (XML, PDF, images, …) have to be read and
processed. In a specific case the application was not responsive for a couple of minutes or performance
was so limited that it was not practically usable. The root cause was the garbage collection going crazy.
Bad Garbage Collector Configuration
In the scenarios presented so far the problem was caused by the application code. In a lot of cause the
root cause however is wrong – or missing – configuration of the garbage collector. I frequently see
people trusting the default settings of their application servers and believing these application server
guys know best what is ideal for their application. The configuration of the heap however strongly
depends on the application and the actual usage scenario. Depending on the scenario parameters have
to adopted to get a well performing application. An application processing a high number of short
lasting requests has to be configured completely different than a batch application, which is execution
long lasting tasks. The actual configuration additionally also depends from the used JVM. What works
fine for Sun JVM might be a nightmare for IBM (or at least not ideal).
Misconfigured garbage collectors are often not immediately identified as the root cause of a
performance problem (unless you monitor Garbage Collector acitvity anyway). Often the visual
manifestation of problems are bad response times. Understand the relation of garbage collector activity
to response times is not obivous. If garbage collector times cannot be correlated to response times,
people find themselves hunting a very complex performance problem. Response times and execution
time metric problems will manifest across the applications – at different places without an obvious
pattern behind this phenomenon.
ClassLoader Leaks
When talking about memory leaks most people primarily think about objects on the heap. Besides
objects, classes and constants are also managed on the heap. Depending on the JVM they are put into
specific areas of the heap. The Sun JVM for example uses the so called permanent generation or
PermGen. Classes very often are put on the heap several times. Simply because they have been loaded
by different classloaders. The memory occupation of loaded classes can be up to several hundred MB in
modern enterprise applications.
Performance & JVM & GC 21 of 36
Key is to avoid unecessarily increasing the size of classes. A good sample is the definition of large
amount of String constants – for example in GUI applications. Here all texts are often stored in
constants. While the approach of using constants for Strings is in principle a good design approach, the
memory consumption should not be neglected. In a real world case all constants where define in one
class per language in an internationalized application. A not obviously visibile coding error resulted in all
of this classed being loaded. The result was a JVM crash with an OutOfMemoryError in the PermGen of
the application.
Application servers suffer additional problems with classloader leaks. These leaks are causes as
classloaders cannot be garbage collected because an object of one of the classes of this classloader is
still alive. As a result memory occupied by these classes will not be freed up. While this problem today is
handled well by J EE application server, it seems to appear more often in OSGI-based application
environments.
Conclusion
Memory problems in Java applications are manifold und easily lead to performance and scalability
problems. Especially in J EE applications with a high number of parallel users memory management must
be a central part of the application architecture.
While the garbage collector takes care that unreferenced objects are clean up, the developer still is
responsible for proper memory management. In addition to application design memory management is
a central part of application configuration.
____________________________________________________________________________
Common Performance problems
● Java engine hangs during startup.
● Java engine takes a long time to return a response. For example, search operations take long
time to respond in portal.
● Time-out error for portal and other java applications.
● Long running or frequent garbage collection activity (Full GCs).
● Heavy paging activity.
● High CPU usage.
● Connectivity between servernode and message server fails (missed broadcast, delay while
getting response from message server/servernode. All -33X exit codes)
● System or applications threads maxed out.
● System slows down and crashes with out of memory (exitcode 666). <std_serverN.out> contains
java.lang.OutOfMemoryError or Out of memory. System will write heap dumps if the
parameters as per Sap note 1004255 are maintained.
Performance & JVM & GC 22 of 36
● Unable to login to java applications or login takes a lot of time.
● Intermittent servernode restarts.
● Portal or other java application pages remain blank after login.
____________________________________________________________________________
Commonly seen reasons behind performance issues
● Insufficient heap, permanent or physical memory.
● Insufficient number of application and/or system threads
● Too many logs are written due to log severity settings.
● Message server connectivity issues - problem with message server timeout settings.
● Java threads being blocked by certain applications.
● Slow response from LDAP server.
● Incorrect JVM parameters settings.
● Outdated JVM used.
● High CPU consumption by Java processes.
● Outdated database statistics lead to slow portal performance.
● Network issues.
● Certain applications consuming majority of heap.
● Inconsistent support pack levels leading system to crash.
● Timed out java services.
● Dispatcher node hangs (valid for releases lower that Netweaver 7.1).
● Communication error between java and SCS instances..
● All system threads are in use.
● All application threads are in use.
____________________________________________________________________________
Memory management algorithms/management in java
In java, memory is managed via garbage collector. Few techniques for memory management are:
1. Reference Counting: A count of references to each object is maintained. When garbage collector runs,
it deletes objects with zero reference count.
Drawback: Circular references are maintained in memory.
2. Tracing collectors/Copy Collector/Stop and copy collector: Start from a root object and keep a track of
all references which have direct/indirect reference to the root object. Then all the live objects are
moved to another heap, taking care of references properly.
Performance & JVM & GC 23 of 36
Drawback: At each point of time, you will have 2 heaps thus consuming twice the memory.
3. Mark sweep collectors/Stop and work collector: Similar to tracing collector except that instead of
copying the references to the new heap, they are swept out of memory, after a list of live and dead
objects is known.
Mark and sweep is a stop-the-world garbage collection technique; that is all application threads stop
until garbage collection completes or until a higher-priority thread interrupts the garbage collector. If
the garbage collector is interrupted it must restart which can lead to application thrashing with little
apparent result.
____________________________________________________________________________
Explain types of references in Java?
There are actually four different degrees of reference strength: strong, soft, weak, and phantom, in
order from strongest to weakest:
Strong Reference: By default.
Weak references: A weak reference is a reference that isn't strong enough to force an object to remain
in memory. Weak references allow you to leverage the garbage collector's ability to determine
reachability for you, so you don't have to do it yourself. You create a weak reference like this:
WeakReference<Widget> weakWidget = new WeakReference<Widget>(widget);
weakWidget.get() // get the actual Widget
Of course the weak reference isn't strong enough to prevent garbage collection, so you may find (if
there are no strong references to the widget) that weakWidget.get() suddenly starts returning null.
Soft references: A soft reference is exactly like a weak reference, except that it is less eager to throw
away the object to which it refers. An object which is only weakly reachable (the strongest references to
it are WeakReferences) will be discarded at the next garbage collection cycle, but an object which is
softly reachable will generally stick around for a while. Soft References aren't required to behave any
differently than WeakReferences, but in practice softly reachable objects are generally retained as long
as memory is in plentiful supply. This makes them an excellent foundation for a cache, since you can let
the garbage collector worry about both how reachable the objects are and how badly it needs the
memory they are consuming.
Phantom references
Performance & JVM & GC 24 of 36
A phantom reference is quite different than either SoftReference or WeakReference. Its grip on its
object is so tenuous that you can't even retrieve the object -- its get() method always returns null. The
only use for such a reference is keeping track of when it gets enqueued into a ReferenceQueue, as at
that point you know the object to which it pointed is dead. How is that different from WeakReference,
though?
The difference is in exactly when the enqueuing happens. WeakReferences are enqueued as soon as the
object to which they point becomes weakly reachable. This is before finalization or garbage collection
has actually happened. In case of Weak Reference, object could even be "resurrected" by an finalize()
method, but the WeakReference would remain dead. PhantomReferences are enqueued only when the
object is physically removed from memory, and the get() method always returns null specifically to
prevent you from being able to "resurrect" an almost-dead object.
Use of Phantom Reference:
1. They allow you to determine exactly when an object was removed from memory. They are in fact the
only way to determine that. This isn't generally that useful, but might come in handy in certain very
specific circumstances like manipulating large images: if you know for sure that an image should be
garbage collected, you can wait until it actually is before attempting to load the next image, and
therefore make the dreaded OutOfMemoryError less likely.
2. PhantomReferences avoid a fundamental problem with finalization – resurrection. With
PhantomReference, resurrection is impossible. When a PhantomReference is enqueued, there is
absolutely no way to get a pointer to the now-dead object.Arguably, the finalize() method should never
have been provided in the first place. PhantomReferences are definitely safer and more efficient to use,
and eliminating finalize() would have made parts of the VM considerably simpler. But, they're also more
work to implement, so I confess to still using finalize() most of the time. The good news is that at least
you have a choice.
____________________________________________________________________________
Phantom Reference Notes
The object is strongly reachable as long as there's a reference to it; either in a static field, an instance
field of another strongly reachable object, or a local variable that is currently within scope.
When there are no more strong references to the object, it becomes softly reachable. Softly reachable
objects stay softly reachable until the garbage collector determines that memory is running out, and
they need to make room for other objects. The garbage collector then "clears" all the soft references
(meaning, the soft reference will no longer refer to the object it used to refer to), and enqueues the
references.
Performance & JVM & GC 25 of 36
After that, the object becomes weakly reachable. The garbage collector will immediately clear all weak
references to the object, and enqueue them. The object is then eligible for finalization.
Now, the garbage collection will run the finalize() method on the object, if it hasn't already done so.
After the finalize() method is done, the object is finalized, and should** become phantom reachable. So
an object that's phantom reachable only has phantom references to it. Phantom reachable objects are
removed from memory; the space they take up is reclaimed by the garbage collector.
Phantom references are a special case. They *always* return null when asked about their referent. This
is to prevent people from getting strong references to the object after it's been finalized. Secondly,
phantom references aren't automatically cleared by the garbage collector before they are enqueued.
You need to clear them manually after you've retrieved them from the reference queue, and performed
cleanup operations.
Once all the phantom references to the object have been cleared or finalized, the object is no longer
reachable. The object is unreachable.
I'll give some examples of use cases for these references. Soft references are only cleared when the JVM
runs out of memory (this actually isn't a guarantee, the garbage collector may clear the references even
if there's plenty of space). Therefore, they are great for building caches. They keep an object in memory
as long as there's space, but they don't prevent the object from being collected when space runs out.
Weak references are great when you want to store extra information about an object, but that
information is only useful for as long as the object exists. You can then have a weak reference to the
object, which also strongly refers to the extra information. When the object has no strong references to
it anymore, it becomes weakly reachable, is automatically cleared, and the weak reference is enqueued.
The thread that deals with the reference queue can then clean up the extra information when it
retrieves the weak reference from the queue.
Finally, Phantom references are good for two things only. One is determining that the object is *really*
dead and removed from memory. The second thing is performing cleanup. If you need to clean up a
bunch of other objects when the first object is dead, you can do so, and then clear the phantom
reference. This is exactly the same as just performing the finalize() method, except it's safer, because the
finalize() method can perform cleanup before the object is physically removed from memory.
**An object that has been finalized *should* become phantom reachable. However, this is not always
true. The finalize() method can resurrect a dead object by storing a reference to it in a variable, thereby
making it strongly reachable once again. This is why cleanup in the finalize() method is dangerous. It's
possible to clean up other objects, but then resurrecting the original object. When you perform
operations on the object, the program may fail catastrophically.
Performance & JVM & GC 26 of 36
Example
package org.karanki.play.gc; import java.lang.ref.PhantomReference; import java.lang.ref.ReferenceQueue; public class TestPhantomRefQueue { public static void main(String[] args) throws InterruptedException { Object obj = new Object(); final ReferenceQueue queue = new ReferenceQueue(); PhantomReference pRef = new PhantomReference(obj, queue); obj = null; new Thread(new Runnable() { public void run() { try { System.out.println("Awaiting for GC"); // This will block till it is GCd PhantomReference pRef = (PhantomReference) queue.remove(); System.out.println("Referenced GC'd"); } catch (InterruptedException e) { e.printStackTrace(); } } }).start(); // Wait for 2nd thread to start
Performance & JVM & GC 27 of 36
Thread.sleep(2000); System.out.println("Invoking GC"); System.gc(); } }
____________________________________________________________________________
Talk about garbage collector for various references!
Talk about garbage collector for various references.
Ans. If an element is determined to be eligible for processing, GC must determine if it is eligible for
collection. The first criterion here is simple. Is the referent marked? If it is marked, the reference object
is not eligible for collection and GC moves onto the next element of the list. However, if the referent is
not marked and is eligible for collection, the process differs for each reference type.
Soft references are collected if their referent has not been marked for the previous 32 garbage
collection cycles. You adjust the frequency of collection with the -Xsoftrefthreshold option. If there is a
shortage of available storage, all soft references are cleared. All soft references are guaranteed to have
been cleared before the OutOfMemoryError is thrown.
Weak and phantom references are always collected if their referent is not marked.
Performance & JVM & GC 28 of 36
____________________________________________________________________________
Explain garbage collection on Remote Objects or Distributed Garbage
collection.
In a distributed system, just as in the local system, it is desirable to automatically delete those remote
objects that are no longer referenced by any client. This frees the programmer from needing to keep
track of the remote objects' clients so that it can terminate appropriately. RMI uses a reference-counting
garbage collection algorithm for the same.
To accomplish reference-counting garbage collection, the RMI runtime keeps track of all live references
within each Java virtual machine. When a live reference enters a Java virtual machine for first time, it
sends a "referenced" message to the server for the object. Going forward, whenever a live reference
enters JVM, its reference count is incremented and is decremented as soon as it leaves the JVM. When
the last reference has been discarded, an unreferenced message is sent to the server. Many subtleties
exist in the protocol; most of these are related to maintaining the ordering of referenced and
unreferenced messages in order to ensure that the object is not prematurely collected.
When a remote object is not referenced by any client, the RMI runtime refers to it using a weak
reference. The weak reference allows the Java virtual machine's garbage collector to discard the object
if no other local references to the object exist. As long as a local reference to a remote object exists, it
cannot be garbage-collected and it can be passed in remote calls or returned to clients. Remote objects
Performance & JVM & GC 29 of 36
are only collected when no more references, either local or remote, still exist. The distributed garbage
collection algorithm interacts with the local Java virtual machine's garbage collector in the usual ways by
holding normal or weak references to objects.
In addition to the reference counting mechanism, a live client reference has a lease with a specified
time. When the client is done with the reference and allows the remote stub to go out of scope, or when
the lease on the object expires, the reference layer on the host automatically deletes the record of the
remote reference and notifies the client's reference layer that this remote reference has expired. The
lease time is controlled by the system property java.rmi.dgc.leaseValue. The value is in milliseconds and
defaults to 10 minutes. The concept of expirable leases, as opposed to strict on/off references, is used
to deal with situations where a client-side failure or a network failure keeps the client from notifying the
server that it is done with its reference to an object.
A remote object needing unreferenced notification must implement the java.rmi.server.Unreferenced
interface. When those references no longer exist, the unreferenced method will be invoked.
____________________________________________________________________________
Does OutOfMemoryError and StackOverFlowError cause JVM crash?
Any problem in PURE Java code throws a Java exception or error. Java exceptions or errors will NOT
cause a core dump (on UNIX systems) or a Dr.Watson error (on WIN32systems). Any serious Java
problem will result in an OutOfMemoryError thrown by the JVM with the stack trace and consequently
JVM will exit. An OutOfMemoryError (not jvm crash) can be thrown due to one of the following 4
reasons:
1. JVM may have a memory leak due to a bug in its internal heap management implementation. But this
is highly unlikely because JVMs are well tested for this.
2. The application may not have enough heap memory allocated for its running. You can allocate more
JVM heap size (with –Xmx parameter to the JVM) or decrease the amount of memory your application
takes to overcome this. You can increase heap size as below:
java -Xms1024M -Xmx1024M
Care should be taken not to make the –Xmx value too large because it can slow down your application.
3. Another not so prevalent cause is the running out of a memory area called the “perm” which sits next
to the heap. All the binary code of currently running classes is archived in the “perm” area. The ‘perm’
area is important if your application or any of the third party jar files you use dynamically generate
classes.
For example: “perm” space is consumed when XSLT templates are dynamically compiled into classes,
J2EE application servers, JasperReports, JAXB etc use Java reflection to dynamically generate classes
and/or large amount of classes in your application. To increase perm space:
Performance & JVM & GC 30 of 36
java -XX:PermSize=256M -XX:MaxPermSize=256M
4. The fourth and the most common reason is that you may have a memory leak in your application.
____________________________________________________________________________
Different OutOfMemory errors!!
Let’s have a look at the Sun HotSpot JVM and its concrete implementation of OutOfMemoryError errors.
1. In the heap we get an OutOfMemoryError, if the garbage collector cannot reclaim enough memory
for a new object. In such situation the Sun HotSpot JVM shows this error message:
java.lang.OutOfMemoryError: Java heap space
2. An alternative for this is as below, it occurs when application tries to create an array on the heap
that is bigger than the total heap size.
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
3. If there is not enough memory in the method area for creating a new class, the Sun HotSpot
implementation gets an error in the permanent generation:
java.lang.OutOfMemoryError: PermGen space
4. OutOfMemory errors in thread exclusive memory areas occur less frequently and are identified by
the following error messages in the Sun HotSpot JVM:
java.lang.OutOfMemoryError: unable to create new native thread
This occurs if there are too many threads in the JVM and there is not enough memory left to create a
new thread. I’ve seen this because the memory limits of a process have been reached (especially in
32bit operating systems, e.g. on Windows 32bit it is 2GB) or the maximum number of file handles for
the user that executes the java process has been reached.
5. It indicates that a memory allocation error on a native stack (JNI method call) has occured.
java.lang.OutOfMemoryError: <reason> <stacktrace> (Native method)
6. It is also interesting that a memory allocation error on the JVM stack (too many frames on the stack)
does not throw an Java OutOfMemory error but as the JVM specification mandates.
java.lang.StackOverflowError
7. The last variant of the OutOfMemoryError is out of swap space. This error is thrown if there is not
enough memory left on the operating system level – which is normally true if other processes are using
all of the available memory or the swap space is configured too small.
java.lang.OutOfMemoryError: request <size> bytes for <reason>.
Performance & JVM & GC 31 of 36
____________________________________________________________________________
Why does the JVM crash with a core dump or a Dr.Watson error?
Both the core dump on UNIX operating system and Dr.Watson error on WIN32 systems mean the same
thing. If you define a crash as an unhandled problem (i.e. no Java Exception or Error); then this cannot
be done from within Java. The JVM is a process like any other and when a process crashes a core dump
is created. A core dump is a memory map of a running process. This can happen due to one of the
following reasons:
1. Using JNI (Java Native Interface) code containing a fatal bug in it. Typical crashes in native code
happen by dereferencing pointers to wrong memory areas (like Nullpointer) or illegal opcodes.
For ex: using Oracle OCI drivers, which are written partially in native code or JDBC-ODBC bridge drivers,
which are written in non Java code. Using 100% pure Java drivers (communicates directly with the
database instead of through client software utilizing the JNI) instead of native drivers can solve this
problem.
2. The OS on which your JVM is running might require a patch or service pack.
3. The JVM implementation may have a bug in translating system resources like threads, file handles,
sockets etc from the platform neutral Java byte code into platform specific operations. If this JVM’s
translated native code performs an illegal operation then the operating system will instantly kill the
process and mostly will generate a core dump file.
The core dump files are generated by the operating system in response to certain signals. The JVM can
also intercept certain signals like SIGQUIT which is kill -3 <pid> from the operating system and it
responds to this signal by printing out a Java stack trace and then continue to run. On the other hand
signals like SIGSTOP (kill -23 <pid>) and SIGKILL (kill -9 <pid>) will cause the JVM process to stop or die.
The JVM argument "java –Xsqnopause" will indicate JVM not to pause on SIGQUIT signal from OS.
4. On Linux/Unix, it is easy to crash JVM crash by sending it a Signal to the running process.
____________________________________________________________________________
Memory Leaks Examples
Auto Boxing
package org.karanki.play.performance;
Performance & JVM & GC 32 of 36
public class Adder { public long addIncremental(long l) { long sum=0L; sum =sum+l; return sum; } public static void main(String[] args) { Adder adder = new Adder(); for(long i = 0; i < 10000000; i++) { adder.addIncremental(i); System.out.println(i + " being added..."); try { Thread.sleep(50); } catch (InterruptedException e) { e.printStackTrace(); } } } }
The Long wrapper creates Long object every time when sum = sum + 1 is performed. Hence using
primitives is much better.
____________________________________________________________________________
Performance & JVM & GC 33 of 36
Performance & JVM & GC 34 of 36
JVM & Garbage Collection
What flags can I use to tune the JVM and GC?
-XX:-UseConcMarkSweepGC: Use the CMS collector for the old gen.
-XX:-UseParallelGC: Use Parallel GC for New Gen
-XX:-UseParallelOldGC: Use Parallel GC for Old and New Gen.
-XX:-HeapDumpOnOutOfMemoryError: Create a thread dump when the application runs out of
memory. Very useful for diagnostics.
-XX:-PrintGCDetails: Log out details of Garbage Collection.
-Xms512m: Sets the initial heap size to 512m
-Xmx1024m: Sets the maximum heap size to 1024m
-XX:NewSize and -XX:MaxNewSize: Specifically set the default and max size of the New Generation
- XX:NewRatio=3: Set the size of the Young Generation as a ratio of the size of the Old Generation.
-XX:SurvivorRatio=10: Set the size of Eden space relative to the size of a survivor space.
____________________________________________________________________________
Head Dump vs Thread Dump vs Core Dump
jmap command is used to generate heap dump if your using Sun JDK.
jrcmd command is used to generate heap dump if your using Jrocket.
Don’t generate heap dump multiple times as it causes your application performance degradation as
generating heap dump will take at least 15 to 30 minutes and generated file size will be minimum 2 GB.
And use Eclipse mat only to analyze it. IBM heap analyzer will not help you in this case due to IBM heap
analyzer supports hprof formated files only to my knowledge.
Heap dump – Collection of objects that are in memory (JVM)
Performance & JVM & GC 35 of 36
Thread dump – Shows what each thread in a process is doing at a given point in time along with the
stack trace.
Core dump – O/S level dump file which has O/S level info in addition to the heap dump.
Heap dump – is useful to analyse OOM situations.
Thread dump – To troubleshoot slow running of your application.
Core dump – When your JVM has crashed abruptly. To find details about native calls and so on.
jmap -heap:live,format=b,file=filename pid
jrcmd pid hprofdump filename=name_of_dump_file
A thread dump is a dump of the stacks of all live threads. Thus useful for analyzing what an app is up to
at some point in time, and if done at intervals handy in diagnosing some kinds of ‘execution’ problems
(e.g. thread deadlock).
A heap dump is a dump of the state of the Java heap memory. Thus useful for analyzing what use of
memory an app is making at some point in time so handy in diagnosing some memory issues, and if
done at intervals handy in diagnosing memory leaks.
Heapdump can be taken in number of ways:
Via Java VM parameters:
-XX:+HeapDumpOnOutOfMemoryError writes heap dump on OutOfMemoryError (recommended)
-XX:+HeapDumpOnCtrlBreak writes heap dump together with thread dump on CTRL+BREAK
Using JRockit:
-jrcmd pid hprofdump filename=name_of_dump_file
Using Jmap:
-jmap -heap:format=b pid
Note: use -J-d64 jvm option if your JVM is 64 Bit Jvm “jmap -J-d64 -heap pid”
You can also manually generate a heap dump with tool VisualVM.
Using HPROF:
Performance & JVM & GC 36 of 36
You can use HPROF: Heap and CPU Profiling Agent.
A complete dump of the current live objects in the heap can be obtained with:
-java -agentlib:hprof=heap=dump,format=b -jar application
This will automatically dump heap when java application is terminated. You can also force heap dump by
sending QUIT signal to java process with kill -QUIT pid command.
Analysing Heapdump file using Jhat
You can use jhat (Java Heap Analysis Tool) to read the generated file:
– jhat [ options ]
The jhat command parses a java heap dump file and launches a webserver. jhat enables you to browse
heap dumps using your favorite webbrowser.
Note that you should have a hprof binary format output to be able to parse it with jhat. You can
useformat=b option to generate the dump in this format.
Top Related