My Old Friend Malloc
38
www.hazelcast.com
-
Upload
christoph-engelbert -
Category
Technology
-
view
1.706 -
download
1
description
Today higher heaps and bigger RAM amounts are typical for standard Java server applications. We often then get to the limit of performance, predictability or GC pause times and using Off-Heap technologies we can lower that pains.
Transcript of My Old Friend Malloc
www.hazelcast.com
WHO AM IChristoph Engelbert (@noctarius2k)8+ years of professional Java developmentSpecialized to performance, GC, traffictopicsApache DirectMemory PMCPrevious companies incl. Ubisoft and HRSOfficial Inhabitant of HazelcastiaCastMapR MapReduce for Hazelcast 3Co-Author Java Off-heap JEP Proposal
www.hazelcast.com
TOPICSJava Memory LayoutDefinition of Off-Heap?Direct Memory Access?Why do we need Off-Heap Memory?Options for Off-Heap StorageAdvantages and Disadvantages of (Off-)HeapAvailable Frameworks(Not so) Secret FutureSmall demonstration
www.hazelcast.com
JAVA MEMORY LAYOUTA FAST WALK-THROUGH
www.hazelcast.com
JAVA MEMORY LAYOUT (1/5)
www.hazelcast.com
JAVA MEMORY LAYOUT (2/5)
YOUNG GENERATION - EDEN SPACEOnly used for Object-AllocationTLAB¹ allocation availableAffected by Minor and Major GCObjects moved to Survivor on Minor GC
¹ TLAB: Thread Local Allocation Buffer
www.hazelcast.com
JAVA MEMORY LAYOUT (3/5)
YOUNG GENERATION - SURVIVOR SPACESAlways one From-Space and one To-SpaceFrom-Space contains living objects on last GCMinor GC switches the Survivor SpacesAlive Objects are moved to To-SpaceAffected by Minor and Major GCLong living objects eventually moved to Tenured
www.hazelcast.com
JAVA MEMORY LAYOUT (4/5)
TENURED SPACEContains only long living objectsAffected only by Major GCMore objects means more GC spend timeConcurrent object inspectation availableLong Stop-The-World pauses possibleNot optimal for big datasets
www.hazelcast.com
JAVA MEMORY LAYOUT (5/5)
PERMGEN / META SPACEClass bytecodesClass metadataRuntime informationCode Compilation Cacheetc.
www.hazelcast.com
G1 (GARBAGE FIRST COLLECTOR)Generational GCHeap splitted into same sized regionsEvery region is either Eden, Survivor or Tenured space
www.hazelcast.com
DEFINITION OFOFF-HEAP?
www.hazelcast.com
Off-Heap is a (continuously) self allocated, managed and freed direct memory region.
It is not under control of the Java GarbageCollector and needscustom written allocation and cleanup of data-regions.
Off-Heap can not be used for allocation of Java objects.
DEFINITION OF OFF-HEAP?
www.hazelcast.com
I'M DUKE SKYWALKERI'M HERE TO RESCUE YOU!
www.hazelcast.com
DIRECT MEMORYACCESS?
GIVE YOURSELF TO THE DARK SIDE
www.hazelcast.com
DIRECT MEMORY ACCESS?ARE YOU SERIOUS?
Fast memory areaNot affecting the GCOfficially available since Java 1.4Serialization overhead when storing objectsLimited by -XX:MaxDirectMemorySize
www.hazelcast.com
WHY DO WE NEEDOFF-HEAP MEMORY?
www.hazelcast.com
WHY DO WE NEED OFF-HEAP MEMORY?Storing of huge datasetsZero-Copy writes to channelsLower pressure on GC / less GC pausesStorage Space only limited by RAMCompact data representation possibleIPC SharedMemory with Memory-Mapped-Files
www.hazelcast.com
OPTIONS FOROFF-HEAP STORAGE
www.hazelcast.com
OPTIONS FOR OFF-HEAP STORAGE (1/3)
JNI (Java Native Interface)int* buffer = ( int* ) malloc( 1024 * sizeof(int) );if( buffer == NULL ) throwOutOfMemoryException();for( int i = 0; i < 1024; i++ ) buffer[sizeof(int) * i] = i;free( buffer );
www.hazelcast.com
OPTIONS FOR OFF-HEAP STORAGE (2/3)
(Direct)ByteBufferByteBuffer buffer = ByteBuffer.allocateDirect( 1024 * 4 );for( int i = 0; i < 1024; i++) buffer.putInt( i );// buffer is automatically freed by GC
www.hazelcast.com
OPTIONS FOR OFF-HEAP STORAGE (3/3)
sun.misc.UnsafeUnsafe unsafe = trickToRetrieveUnsafe();long address = unsafe.allocateMemory( 1024 * 4 );for( int i = 0; i < 1024; i++ ) unsafe.putInt( address + 4 * i, i );unsafe.freeMemory( address );
www.hazelcast.com
ADVANTAGESAND
DISADVANTAGESOF (OFF-)HEAP
www.hazelcast.com
ADVANTAGES ON-HEAPNo messing with mallocAutomatic Garbage CollectionNo need for Serialization
www.hazelcast.com
DISADVANTAGES ON-HEAPSlowdown on high allocation rateBig overhead on small objects (header)Heavily depending on GC Combination
www.hazelcast.com
ADVANTAGES OFF-HEAPNo limitation* in sizeCompact data layoutZero-Copy socket read/write possibleFrameworks available to manage memory
www.hazelcast.com
DISADVANTAGES OFF-HEAPAllocation is up to youDeallocation is up to youData layout is up to youSerialization may be requiredJNI requires native library
www.hazelcast.com
AVAILABLEFRAMEWORKS
www.hazelcast.com
AVAILABLE FRAMEWORKS (1/5)Apache DirectMemoryTerracotta BigMemoryHazelcast ElasticMemoryMapDB.orgetc.
www.hazelcast.com
AVAILABLE FRAMEWORKS (2/5)
Apache DirectMemoryCacheService<String, String> cacheService = new DirectMemory<...>() .setNumberOfBuffers( 10 ).newCacheService();
cacheService.put( "MyKey", "SomeValue" );String value = cacheService.get( "MyKey" );
www.hazelcast.com
AVAILABLE FRAMEWORKS (3/5)
Terracotta BigMemory<ehcache xml:noNamespaceSchemaLocation="..." name="MyCacheDefinition"> <cache name="MyOffheapCache" maxBytesLocalOffHeap="2G" /></ehcache>
CacheManager cacheManager = new CacheManager();Cache dataStore = cacheManager.get( "MyOffheapCache" );Element element = new Element( "MyKey", "SomeValue" );dataStore.put( element );String value = (String) dataStore.get( "MyKey" ).getObjectValue();
www.hazelcast.com
AVAILABLE FRAMEWORKS (4/5)
Hazelcast Offheap<hazelcast xml:noNamespaceSchemaLocation="..."> <map name="MyMap"> <storage-type>OFFHEAP</storage-type> </map></hazelcast>
HazelcastInstance hz = Hazelcast.newInstance();// Returns a IMap extends ConcurrentMapMap<String, String> map = hz.getMap( "MyMap" );map.put( "MyKey", "SomeValue" );String value = map.get( "MyKey" );
www.hazelcast.com
AVAILABLE FRAMEWORKS (5/5)
MapDB.orgDB db = DBMaker.newDirectMemoryDB().sizeLimit( 2 ).make();// Returns a HTreeMap extends ConcurrentMapMap<String, String> map = db.createHashMap( "cache" ).make();map.put( "MyKey", "SomeValue" );String value = map.get( "MyKey" );
www.hazelcast.com
FURTHER INFORMATION AVAILABLEThe Must-Read for Off-Heap
http://bit.ly/must-read-off-heap
PacketObject Description from IBMhttp://bit.ly/packet-objects
Compact Datastructures (Martin Thompson)http://bit.ly/compact-datastructures
C++ Like Performance for Serializationhttp://bit.ly/serialization-performance
Tricks with Direct Memory Access in Javahttp://bit.ly/direct-memory-tricks
www.hazelcast.com
THE
OFF-HEAP JEP PROPOSALMAY THE FORCE BE WITH YOU.
THE FORCE IS STRONG WITH THIS ONE.
www.hazelcast.com
OFF-HEAP JEP PROPOSALByteBuffer like interfaceFully 64-bit sizes and offsetsCompare-And-Swap operationsVolatile and ordered operationsOptional bounding checksSupports Memory mappingSupport Foreign Function Interface (FFI) JEP
www.hazelcast.com
OFF-HEAP JEP PROPOSAL - CODE EXAMPLEDISCLAIMER: PROVISIONAL API
import javax.direct.*;
BytesFactory factory = createBytesFactory();Bytes bytes = factory.boundsChecking( false ).deallocationChecks( false ) .freeOnGC( false ).create( ByteOrder.LITTLE_ENDIAN, 1024 * 4 );
for( int i = 0; i < 1024; i++ ) bytes.putVolatileInt( i );
bytes.release();
www.hazelcast.com
OFF-HEAP JEP PROPOSALProposal Discussion Group
http://bit.ly/offheap-ml
Proposal Texthttp://bit.ly/proposal-text
FFI JEP 191http://bit.ly/ffi-jep
www.hazelcast.com
@noctarius2k@hazelcast
http://www.sourceprojects.comhttp://github.com/noctarius
THANK YOU!ANY QUESTIONS?
Images: www.clipartist.info, Gnome Nebula Theme, KDE theme, www.grabsteine-klnt.de
www.hazelcast.com
