The Impact of Solid State Drive on Search Engine Cache...

The Impact of Solid State Drive onSearch Engine Cache Management

Jiancong TongPh.D. candidate at Nankai University

Visiting student at University of Melbournelingfenghx@gmail.com

withGang Wang, Xiaoguang Liu (Nankai University)

andJianguo Wang, Eric Lo, Man Lung Yiu (Hong Kong Polytechnic University)

Monash UniversityMay 12, 2014

Background and Motivation Research Questions and Answers

Outline

1 Background and Motivation

2 Research Questions and AnswersRQ1: What is the impact of SSD on buffer management?RQ2: How could we deal with that?

1 / 30

Hard Disk Drive

Hard Disk Drive (HDD)

Magnetic head

How does HDD work? [Garcia et al., 2000]

Random read latency of HDD

I Seek time

I Rotational latency

I Transfer time

Caching technology is used to reduce the latency.

2 / 30

Hard Disk Drive

Magnetic head

I Seek time

I Transfer time

2 / 30

Hard Disk Drive

Magnetic head

I Seek time

I Transfer time

2 / 30

What is a Cache?

Small, fast memory used to improve average access time tolarge, slow storage media.

Exploits locality: both spacial and temporal.

Almost everything is a cache in computer architecture...

3 / 30

What is a Cache?

3 / 30

What is a Cache?

3 / 30

What is a Cache? (Cont.)

I Cache Hit: the requested data is found in the memory

I Cache Miss: the requested data is not found in the memory

Cache hit Cache miss

Hit ratio = #Hits#Memory accesses

Miss ratio = #Misses#Memory accesses

Hit ratio + Miss ratio = 1

4 / 30

What is Solid State Drive?

Hard Disk Drive (HDD)with magnetic moving head

Solid State Drive (SSD)based on semiconductor chips

I SSD: New Faster (10 ∼ 100x) HDD with compatible interface

I Strong technical merits: [Chen et al., 2009]

1 Lower power consumption2 More compact size3 Better shock resistance4 Extraordinarily faster random data access

5 / 30

What is Solid State Drive?

Hard Disk Drive (HDD)with magnetic moving head

Solid State Drive (SSD)based on semiconductor chips

I SSD: New Faster (10 ∼ 100x) HDD with compatible interface

I Strong technical merits: [Chen et al., 2009]

1 Lower power consumption2 More compact size3 Better shock resistance4 Extraordinarily faster random data access

5 / 30

The Rise of SSD

Words of Pioneer (by Jim Gray, 2006)

Tape is dead; Disk is tape; Flash is disk; RAM locality is King.

SSD in Large-Scale System Architectures

I Google 2008 (or later)

I Baidu 2008

I Facebook 2010

I Myspace 2010

I Oracle 2011

I Microsoft Azure 2012

Trends

Flash memory based SSD is replacing and is going to completelyreplace HDD as the major storage medium!

6 / 30

The Rise of SSD

I Baidu 2008

I Facebook 2010

I Myspace 2010

I Oracle 2011

Trends

6 / 30

The Rise of SSD

I Baidu 2008

I Facebook 2010

I Myspace 2010

I Oracle 2011

Trends

6 / 30

Challenges

Existing caching policies were originally designed for HDD

I HDD: Very slow random read (compare to sequential read)

I Cache design principle: minimize random read

However...

[Tong et al., 2013]

What now?

7 / 30

Challenges

However...

[Tong et al., 2013]

What now?

7 / 30

Challenges

However...

[Tong et al., 2013]

What now?

7 / 30

Research Questions

RQ1: What is the impact of SSD on buffer management?

I Are the existing cache techniques designed for HDD-basedsearch engine still good for SSD-based search engine?

I What measure(s) should be used to define a ‘good’ cachepolicy in this case?

I If the performance of caching is improved or degraded, whatdoes that mean to the entire system?

RQ2: How could we deal with that?

I What to do if the efficiency of the entire system is affected bysuch impact?

I Can we propose better cache policies for SSD-based systems?

8 / 30

Research Questions

8 / 30

Research Questions

8 / 30

Research Questions

8 / 30

Research Questions

8 / 30

Research Questions

8 / 30

Research Questions

8 / 30

9 / 30

Large-scale experimental study

RQ1 can be answered by evaluating the effectiveness ofexisting caching policies on an SSD-based search engine.

Settings

I Datasets

Web documents: 12,000,000 (∼100GB)Queries: 1,000,000

I Device

SSD: ADATA 256GB SSDHDD: Seagate 3TB 7200rpm

I System: Apache Lucene

I Measure: Query time (NOT hit ratio)

10 / 30

Overview of search engine architecture

[Wang et al., 2013]

11 / 30

[Wang et al., 2013]

12 / 30

[Wang et al., 2013]

12 / 30

Index Server and List (Index) Cache

Posting Lists (Inverted Lists)

I Clayton (2, 4, 5, 10, 100, 107, ..., 1,000,000)

I Monash (1, 2, 3, 23, ...)

Index Server

13 / 30

Index Server and List (Index) Cache

Posting Lists (Inverted Lists)

I Clayton (2, 4, 5, 10, 100, 107, ..., 1,000,000)

I Monash (1, 2, 3, 23, ...)

Index Server

13 / 30

Which lists should be kept in the cache?

Metrics to consider

I High Frequency First?

I High Frequency/Size First?

14 / 30

Which lists should be kept in the cache?

Metrics to consider

I High Frequency First?

I High Frequency/Size First?

14 / 30

Previous conclusion [Baeza-Yates et al., 2007]

Frequency/Size is a better metric (for HDD).

The reason

On HDD

I Random read is 130 ∼ 170x slower than sequential read=⇒ List access time is (almost) a constant!

I It is #access that dominates, not |access data|=⇒ Short lists are favored (more lists can be held)

I High benefit gained when size is taken into account

15 / 30

The reason

On HDD

15 / 30

The reason

On HDD

15 / 30

The reason

On HDD

15 / 30

The reason

On HDD

15 / 30

How does SSD change the story? [Wang et al., 2013]

Differences

On SSD

I Random read is only 2 ∼ 10x slower than sequential read=⇒ List access time varies a lot!

I Long lists have slower access time=⇒ Admitting too many short lists offers less benefit

16 / 30

Differences

On SSD

16 / 30

Differences

On SSD

16 / 30

How does SSD change the story? (Cont.)

Read access latency of posting lists of varying lengths

Conjecture 1

The best cache policy may have changed

17 / 30

How does SSD change the story? (Cont.)

Read access latency of posting lists of varying lengths

Conjecture 1

The best cache policy may have changed

17 / 30

Evaluation on Index Server

Static List Cache Policy

on HDD on SSD

Conjecture 1

The best cache policy may have changed – ConfirmedFDo not rely on your knowledge acquired in HDD era.

18 / 30

on HDD on SSD

Conjecture 1

The best cache policy may have changed – Confirmed

FDo not rely on your knowledge acquired in HDD era.

18 / 30

on HDD on SSD

Conjecture 1

The best cache policy may have changed – ConfirmedFDo not rely on your knowledge acquired in HDD era.

18 / 30

Is Hit Ratio Reliable?

Previously on HDD Now on SSDReliable ?As list access is a constant NOSo, can measure the effectiveness of the cache ?

Conjecture 2

Cache hit ratio might not be reliable.

19 / 30

Conjecture 2

19 / 30

Conjecture 2

19 / 30

Is Hit Ratio Reliable? (Cont.)

Static List Cache Policy (on SSD)

Conjecture 2

Cache hit ratio might not be reliable – ConfirmedFUse query time, not hit ratio

20 / 30

Conjecture 2

Cache hit ratio might not be reliable – Confirmed

FUse query time, not hit ratio

20 / 30

Conjecture 2

Cache hit ratio might not be reliable – ConfirmedFUse query time, not hit ratio

20 / 30

Document Server and Its Cache

Document Server

21 / 30

Document Server and Its Cache (Cont.)

22 / 30

Evaluation on Document Server

Document retrieval is no longer the bottleneck on document server

Conjecture 3

Is it the same case for the whole system?

23 / 30

Conjecture 3

23 / 30

Conjecture 3

23 / 30

The Impact on the Entire System Efficiency

Time break down for query processing (on SSD)

All caches are enabled

Conjecture 3

Is it the same case for the whole system? – ConfirmedFDisk accessing is no longer the bottleneck!

24 / 30

Conjecture 3

Is it the same case for the whole system? – Confirmed

FDisk accessing is no longer the bottleneck!

24 / 30

Conjecture 3

Is it the same case for the whole system? – ConfirmedFDisk accessing is no longer the bottleneck!

24 / 30

25 / 30

The Impact on the Entire System Efficiency (Cont.)

Problems The time spent on intersection and ranking should be reduced

Solution Adopting full-term-ranking-cache (FTRC) [Altingovde et al., 2011]and two-term-intersection-cache (TTIC) [Long and Suel, 2005]

26 / 30

Incorporating PLC+FTRC+TTIC

27 / 30

New Static List Caching Policy

Can we convert the latencies of random seek and sequentialread into a uniform measure?

28 / 30

New Static List Caching Policy

Can we convert the latencies of random seek and sequentialread into a uniform measure?

28 / 30

New Static List Caching Policy (Cont.)

Latency-Aware Caching (Static): BLOCK [Tong et al., 2013]

I For term t, B(t) represents the I/O cost that can be saved during the wholequery evaluating process if `(t) is kept in the cache

I Estimating model and existing static caching policies:

I New method: BLOCK

1 access = 1 random seek + several sequential reads= 1 random seek + a few equivalent random seeks

29 / 30

I New method: BLOCK

29 / 30

I New method: BLOCK

29 / 30

I New method: BLOCK

29 / 30

Experimental Results

on SSD on HDD

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF/MECH

0.91.01.11.21.31.41.5

2560 3072 3584 5

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF

MECHBLOCK

2560 3072 3584

I BLOCK outperforms other methods on SSD1 BLOCK beats QTFDF (by far the best policy on HDD) by 20%-60%2 BLOCK outperforms QTF/MECH (identical on SSD) too

I BLOCK is better even on HDD (though not that significantly)

30 / 30

on SSD on HDD

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF/MECH

0.91.01.11.21.31.41.5

2560 3072 3584 5

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF

MECHBLOCK

2560 3072 3584

I BLOCK outperforms other methods on SSD

1 BLOCK beats QTFDF (by far the best policy on HDD) by 20%-60%2 BLOCK outperforms QTF/MECH (identical on SSD) too

30 / 30

on SSD on HDD

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF/MECH

0.91.01.11.21.31.41.5

2560 3072 3584 5

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF

MECHBLOCK

2560 3072 3584

I BLOCK outperforms other methods on SSD1 BLOCK beats QTFDF (by far the best policy on HDD) by 20%-60%

2 BLOCK outperforms QTF/MECH (identical on SSD) too

30 / 30

on SSD on HDD

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF/MECH

0.91.01.11.21.31.41.5

2560 3072 3584 5

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF

MECHBLOCK

2560 3072 3584

30 / 30

on SSD on HDD

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF/MECH

0.91.01.11.21.31.41.5

2560 3072 3584 5

512 1024 1536 2048 2560 3072 3584

Cache size (MB)

QTFDFQTF

MECHBLOCK

2560 3072 3584

30 / 30

References

Altingovde, I. S., Ozcan, R., Cambazoglu, B. B., and Ulusoy, O. (2011).

Second chance: A hybrid approach for dynamic result caching in search engines.In ECIR, pages 510–516.

Baeza-Yates, R. A., Gionis, A., Junqueira, F., Murdock, V., Plachouras, V., and Silvestri, F. (2007).

The impact of caching on search engines.In SIGIR, pages 183–190.

Chen, F., Koufaty, D. A., and Zhang, X. (2009).

Understanding intrinsic characteristics and system implications of flash memory based solid state drives.In SIGMETRICS/Performance, pages 181–192.

Garcia, J. M., Frohock, M., Reding, E., Reding, J. A., Garcia, M. F., DeLuca, S. A., and Whalen, E. (2000).

Microsoft SQL Server(tm) 2000 Administrator’s Companion.Microsoft Press.

Long, X. and Suel, T. (2005).

Three-level caching for efficient query processing in large web search engines.In WWW, pages 257–266.

Tong, J., Wang, G., and Liu, X. (2013).

Latency-aware strategy for static list caching in flash-based web search engines.In CIKM, pages 1209–1212.

Wang, J., Lo, E., Yiu, M. L., Tong, J., Wang, G., and Liu, X. (2013).

The impact of solid state drive on search engine cache management.In SIGIR, pages 693–702.

Thanks for your time!

Questions & Comments?

For more details of RQ1, please refer to:

I J. Wang, E. Lo, M, Yiu, J. Tong, G. Wang, X. Liu,The impact of Solid State Disk on Search Engine Cache Management,In SIGIR, 2013, pp. 693-702.

For more details of RQ2, please refer to:

I J. Tong, G. Wang, X. Liu,Latency-Aware Strategy for Static List Caching in Flash-based Web Search Engines,In CIKM, 2013, pp. 1209-1212.

The Impact of Solid State Drive on Search Engine Cache...

Documents

Transcript of The Impact of Solid State Drive on Search Engine Cache...

INF3380: Parallel Programming for Natural Sciences · Cache Cache Core Core Core Core Cache Cache Bus Compute Node Memory Core Core Core Core Cache Cache Core Core Core Core ... Compute

Cache in Chromium: Disk Cache

Weibo / Cache-Cache

thaicom.dkthaicom.dk/assets/files/Thanulux Company Profile.pdfladieswear, childrenswear and leather goods under brand Lollipop , CACHE CACHE , ERAWON Cadeau , Lollipop , CACHE: CACHE

NET+OS 6.1 Training. Cache API NET+OS 6.1 Cache API H/W Features NET+OS Cache Initialization Cache and DMA Configuring Cache API Functions.

System Cache v1.00 - Xilinx · The Cache memory provides the actual cache functionality in the System Cache. The cache is configurable in terms of size and associativity. The cache

LogiCORE IP System Cache v1The Cache memory provides the actual cache functionality in the System Cache. The cache is configurable in terms of size and associativity. The cache size

d’autonomie - Cache-cache 1 - laclassebleuelaclassebleue.fr/wp-content/uploads/2011/08/Cache-cache.pdf · d’autonomie - Cache-cache 12 . Les blasons d’autonomie - Cache-cache

Cache-Oblivious Ray Reordering - KAISTsglab.kaist.ac.kr/CORR/CORR_techreport.pdf · cache-oblivious. Cache-aware algorithms utilize the knowledge of cache parameters, such as cache

Modeling and Verifying Cache-Coherent Protocols, VIP, and Designs · 2020-06-13 · Cache Memory Agent – A2 Cache Figure 1: Cache coherence management One way to manage cache coherence

Samsung Solid State Drive RAPID mode · Samsung Solid State Drive RAPID mode ... leverages spare system DRAM and CPU resources to manage a cache of ... Samsung EVO is the most advanced

Managing Cache Coherency on Cortex-M7 Based MCUsww1.microchip.com/downloads/en/DeviceDoc/Managing-Cache-Cohe… · The cache hits only update the cache memory. Cache misses on a write,

Intel® Solid State Drive 540s Series (M.2) · Performance measured within the SLC cache buffer using IOMeter* with Queue Depth 32. Table 5: Sustained Performance ... Intel® Solid

Oracle Cache Fusion Cache Fusion Concepts, Data Block Shipping, and Recovery with Cache Fusion.

Guide to JCT.pdf

Solid State Storage - Flash Memory Summit€¢ 3PAR architecture autonomically changes cache-to-media offload frequency based on utilization • 3PAR also keeps track of read cache

Brochure géotouristique CACHE-CACHE AVEC LA GLACE

Topic 6: Cache Microarchitecture ECE 4750 Computer ... · ECE 4750 T06: Cache Microarchitecture 31 / 36. Single-Bank Cache uArchMulti-Bank Cache uArch • Basic Optimizations •Cache

d’autonomie - Cache-cache 1 · Les blasons d’autonomie - Cache-cache 2

Azure Redis Cache - Cache on Steroids!