Earticle

On-chip caches of graphics processing units (GPUs) have contributed to improved GPU performance by reducing long memory access latency. However, cache efficiency remains low despite the facts that recent GPUs have considerably mitigated the bottleneck problem of L1 data cache. Although the cache miss rate is a reasonable metric for cache efficiency, it is not necessarily proportional to GPU performance. In this study, we introduce a second key determinant to overcome the problem of predicting the performance gains from L1 data cache based on the assumption that miss rate only is not accurate. The proposed technique estimates the benefits of the cache by measuring the balance between cache efficiency and throughput. The throughput of the cache is predicted based on the warp occupancy information in the warp pool. Then, the warp occupancy is used for a second bypass phase when workloads show an ambiguous miss rate. In our proposed architecture, the L1 data cache is turned off for a long period when the warp occupancy is not high. Our two-level bypassing technique can be applied to recent GPU models and improves the performance by 6% on average compared to the architecture without bypassing. Moreover, it outperforms the conventional bottleneck-based bypassing techniques.

최근 인터넷의 폭발적인 인기로 인해 인터넷 객체의 캐슁 기법이 매우 중요한 문제가 되었다. 인터넷에서의 캐슁은 전통적인 캐슁 기법과 여러 면에서 다르다. 특히 일반적인 캐슁 알고리즘은 인터넷에 적합하지 못한데, 이는 크기가 서로 다른 객체들을 함께 처리함으로써 발생하는 작은 객체들의 불이익에 서 기인한다. 본 논문에서는 인터넷 캐슁을 위해 설계된 기존의 기법들을 살펴보고 새로운 알고리즘을 제안한다. 그리고 여러 가변 길이 대체 알고리즘에 대한 성능 평가의 결과를 제시하고, 객체들을 크기에 따라 구분하여 처리함으로써 바이트 적중률을 향상시키는 방안을 도출한다.

With the recent explosion in using of the Internet, the problem of caching web objects has gained considerable importance. Caching on the Internet differs from traditional caching in several ways. Especially, the conventional caching algorithms are not well suited for the Internet caching. The poor performance is mainly due to its unfair treatment of small objects since all the objects are treated the same even though they differ in size. In this paper, I give an overview of caching policies designed for web objects, and provide a new algorithm of my own. I also have performed trace-driven simulations about variable-size replacement algorithms, and derived a new algorithm to improve byte hit-ratio by classifying objects based on their sizes.

Virtual Desktop Infrastructure(VDI) is rising with cloud service as critical IT technology. In order to minimize cost and improve efficiency, many company is adopting VDI. The problem with VDI is the cost of implementation that requires high cost because ensuring of Quality of Experience(QoE) is necessary. Therefore, VDI server should have the ability that can effectively process a voluminous data. In this paper, We suggest an adaptive block level virtual cache device that will improve a processing of voluminous data in VDI server.

In cloud and game servers, SSDs are used as cache on large capacity disk-based storage. In that configuration, the SSD can be exploited as write-back or write-through cache. Though the write-back cache can improve write performance significantly, data can be lost when the SSD fails. On the contrary, performance improvement is limited in the write-through cache though it has high reliability. By the way, reliability can be improved by applying RAID technique to SSD cache while exploiting write-back for better performance. In this paper, we analyse the reliability and the performance of hybrid storage using SSD cache over disk-based primary storage. In particular, we mathematically analyse reliability and also measure performance of real storage systems with various SSD cache configurations.

공정 기술이 발전하면서 프로세서의 전력 밀도가 높아지면서 프로세서의 많은 부분이 가동되지 않는 상태에 놓이는 검은 실리콘 문제가 대두되었다. STTRAM (Spin Torque Transfer Random Access Memory)은 기존 SRAM에 비해 우수한 누수 전력 효율성 및 읽기 에너지 소모량으로 인해 이러한 문제를 해결할 새로운 온칩 메모리 소자로 주목받고 있다. 본 논문에서는 STTRAM만을 데이터 저장 공간으로 사용한 (SRAM-STTRAM 혼용이 아닌) L1 데이터 캐쉬와 명령어 캐쉬 메모리를 제안하고 이를 에너지, 성능, 면적 측면에서 평가한다. STTRAM을 L1 데이터 캐쉬 및 명령어 캐쉬에 모두 사용한 경우, 기존 SRAM으로 구성된 캐쉬 메모리 대비 최대 58% 정도의 에너지 소모를 절감할 수 있다. 또한, 빠른 읽기 성능 덕분에 STTRAM을 L1 명령어 캐쉬 메모리에 적용할 경우 약 3%의 성능 증가 효과도 가져온다. 면적 측면에서는 약 80% 정도의 면적을 SRAM 기반 캐쉬 메모리 대비 절감 가능하다.

As process technology advances, the dark silicon problem, which means a large portion of the processor components remains in the idle or inactive state, has emerged due to increased power density in processors. STTRAM (Spin Torque Transfer Random Access Memory) is one of the promising memory device technologies in the dark silicon era, which enables much lower leakage power consumption and higher read energy efficiency compared to SRAM cells. In this paper, we propose STTRAM-only (i.e., not SRAM-STTRAM hybrid) L1 data and instruction cache architecture and evaluate them in terms of energy, performance and area. In the case where STTRAM cells are used for both L1 data and instruction cache, energy consumption in the L1 caches is reduced by up to 58%. In addition, thanks to faster read access latency, we can improve performance by 3% when STTRAM cells are used for constructing the L1 instruction cache. In terms of area, using STTRAM cells reduces L1 cache area by up to 80% compared to using the conventional SRAM cells in the L1 caches.

이기종 처리환경은 이종의 CPU/GPU가 혼재하는 다처리 환경으로서 빠른 연산속도를 필요로 하는 다양한 응용 분야에서 리 사용되고 있다. 이러한 이기종 처리환경은 CPU/GPU/APU등 로세서를 통합리하여 데이터 공 유가 가능하며 력, 소형화, 고성능을 지향하는 계산 환경변화에 따라 이기종 처리 시스템은 지속으로 발하 고 있다. 여러 가지 응용분야 상처리는 이기종 환경에 가장 합하다. 필터링, 보간, 히스토그램 계산과 같은 상처리 련 연산은 공유 지역 메모리를 효율으로 사용할 수 있는 연산이므로 특별히 이기종 처리 시스템에서 효 과가 극화 된다. 본 논문에서는 이기종 처리 시스템에서의 공유 메모리가 가지는 효과를 소개하고 공유 메모리를 사용하는 상처리 기능을 통해 캐쉬 공유가 효율임을 보이고 이를 통해 CPU와 GPU간의 효율인 작업을 분배 방법을 제안한다.

Heterogeneous processing systems are multi-processing environment containing heterogeneous CPU and GPU. They are widely used in a variety of areas that require fast computation speed. Recently, it is becoming possible to share address space and data through the integration of GPU and CPU. Furthermore, on-die integration of GPU architecture is developing as low power consumption, miniaturization and high performance are becoming more important. Operations in image processing such as filtering, interpolation, and histogram computation can efficiently use shared local memory and applying it to the on-die integration of GPU system in which CPU and GPU share the memory and address system can maximize the effect of those systems. We introduced the study of the effects of shared memory in the existing on-die integration of GPU system, we propose a method of efficient workload distribution between CPU and GPU by measuring the efficiency of cache sharing through image processing operations that use the shared local memory space.

Interval caching is one of the representative caching strategies used in multimedia streaming systems. However, there has been no theoretical analysis on interval caching. In this paper, we present an optimality proof of the interval caching policy. Specifically, we propose a caching performance model for multimedia streaming systems and show the optimality of the interval caching policy based on this model.

In order to improve performance in IoT, edge computing system, a memory is usually configured in a hierarchical structure. Based on the distance from CPU, the access speed slows down in the order of registers, cache memory, main memory, and storage. Similar to the change in performance, energy consumption also increases as the distance from the CPU increases. Therefore, it is important to develop a technique that places frequently used data to the upper memory as much as possible to improve performance and energy consumption. However, the technique should solve the problem of cache performance degradation caused by lack of spatial locality that occurs when the data access stride is large. This study proposes a technique to selectively place data with large data access stride to a software-controlled cache. By using the proposed technique, data spatial locality can be improved by reducing the data access interval, and consequently, the cache performance can be improved.

AM-heap is a heap data structure with O(1) amortized insertion time and O(logn) deletion time complexities. This paper presents cache-aligned 4-ary AM-heap in which every node has four children except that the root has three children. In this heap, the leftmost node of each sibling has an index of a multiple of four. Also, every node index matches its corresponding array index and every sibling nodes smoothly fits into a single cache block. Our experimental results show that cache-aligned 4-ary AM-heap is faster than AM-heap as well as post-order heap.

A hash table is a fundamental data structure implementing an associative memory that maps a key to its associative value. Due to its very fast mapping operation of O(1), it has been widely used in various areas such as databases, bioinformatics, and distributed computing. Besides, the paradigm of micro-architecture design of CPUs is shifting away from faster uniprocessors toward slower chip multiprocessors. In order to fully exploit the performance of such modern computer architectures, the data structures and algorithms considering parallelism become more important than ever. This paper implements three cache-conscious hashing methods, linear hashing and chained hashing, and also, a modern hashing method, hopscotch hashing, and analyzes their performance under Intel 32-core CPU of Nehalem microarchitecture. We implement each hashing method using state-of-the-art techniques such as lock-free data structures, especially based on compare-and-swap (CAS) operations, and refinable data structures. To the best of our knowledge, the work done by this paper is the first work analyzing the performance of three all hashing methods under the same implementation framework. Experimental results using data of 223 (i.e., about eight millions) key-value pairs shows that lock-free linear hashing is the best for insert operation among three hashing methods, and lock-free chained hashing is the best for lookup operation. Hopscotch hashing shows the second best performance of lookup operation. However, hopscotch hash table size is much bigger than other hash table size. Through experiments, we have found that the hopscotch hashing is relatively not efficient than other hash methods.

This paper proposes a novel cross layer collaborating cache scheme for HTTP clients. It is designed to accelerate the process that an HTTP client retrieves Web content in mobile ad hoc network environment. Unlike other techniques to exploit Web proxy caching or to build P2P protocol overlay to improve HTTP, our scheme incorporates collaboratively working capability into the HTTP client response cache, and thus there is no impact on the compatibility of upper level applications. In addition, as the sharing file information is diffused with routing information, there is no redundant traffic to maintain the cooperating cache mechanism. Using this scheme, cacheable Web content can be distributed among local peers and retrieved later by a client if needed. In our simulation results, this novel cache scheme significantly decreases the latency experienced by a client to download Web content.

멀티미디어 스트리밍 데이터는 용량이 매우 크고 순차적으로 접근이 이루어지는 특성이 있어 전통적인 캐슁 환경에서 입출력의 성능을 개선하기 위해 널리 사용되고 있는 LRU 알고리즘이 효과적이지 못한 것으로 알려져 있다. 이에 대한 실험적인 분석은 인터벌 기반 캐슁의 LRU 대비 성능 상의 비교 우위를 통해 입증된 바 있으나, 이론적인 근거가 증명되지는 않았다. 본 논문에서는 멀티미디어 스트리밍 환경을 위한 캐슁의 최적성을 분석하기 위해 캐쉬 성능 평가 모델을 설계하고, 이론적으로 최적인 캐슁 알고리즘을 인터벌 캐슁에 기반해서 설계한다. 그런 다음 설계된 알고리 즘이 스트리밍 데이터의 캐쉬 미스를 최소화하는 교체 알고리즘임을 제안된 모델에 근거한 최적성 분석을 통해 입증한다.

Multimedia streaming data is very large in size and accessed sequentially, making the LRU(Least Recently Used) algorithm widely used to improve I/O performance in traditional caching environments ineffective. Experimental analysis of this has shown the superiority of interval-based caching over LRU, but the theoretical basis has not been proven. In this paper, we design a cache performance model to analyze the optimality of caching for multimedia streaming environments and design a theoretically optimal caching algorithm based on interval caching. Then, we show that the algorithm we design is an optimal algorithm that minimizes cache misses of streaming data based on the proposed model.

인공지능과 고성능 컴퓨팅 기술이 급속히 발전하면서 다양한 분야에 딥러닝 기술이 활용되고 있다. 딥러닝은 학습 과정에서 대량의 데이터를 무작위로 읽어 학습을 진행하고, 이 과정을 반복한다. 많은 수의 파일들이 무작위로 반복 참조되는 딥러닝의 파일 입출력은 시간적 지역성을 지닌 일반적인 응용과는 다른 특징을 보인다. 이로 인한 캐싱의 어려 움을 극복하기 위해 본 연구에서는 딥러닝 데이터셋 읽기의 무작위성을 줄이고 기존의 버퍼 캐시 알고리즘에 적응적으로 동작하는 새로운 데이터 읽기 방안을 제안한다. 본 논문에서는 실험을 통해 제안하는 방식이 버퍼 캐시의 미스율을 기존 의 방식에 비해 평균 16%, 최대 33% 감소시키고, 수행시간을 24%까지 개선함을 보인다.

With the rapid advance in AI (artificial intelligence) and high-performance computing technologies, deep learning is being used in various fields. Deep learning proceeds training by randomly reading a large amount of data and repeats this process. A large number of files are randomly repeatedly referenced during deep learning, which shows different access characteristics from traditional workloads with temporal locality. In order to cope with the difficulty in caching caused by deep learning, we propose a new sampling method that aims at reducing the randomness of dataset reading and adaptively operating on existing buffer cache algorithms. We show that the proposed policy reduces the miss rate of the buffer cache by 16% on average and up to 33% compared to the existing method, and improves the execution time by up to 24%.

최근에 이르러, 수퍼스칼라 프로세서의 하드웨어 복잡도와 성능 한계의 문제를 극복하기 위하여 멀티코어프로세서가 각종 컴퓨터 시스템에 상용화되어 널리 이용되고 있다. 이 때, 멀티코어 프로세서의 성능에 큰 영향을 미치는 것은 명령어 캐쉬와 데이터 캐쉬의 구성 방법과 용량이다. 본 논문에서는 캐쉬의 구조와 용량이 멀티코어 프로세서의 성능에 미치는 영향을 분석하기 위하여, 다양한 캐쉬의 구조와 용량으로 구성되는 2 개에서 16 개까지의 멀티코어 프로세서에 대하여 SPEC 2000 벤치마크를 입력으로 하여 모의실험을 수행하였다. 모의실험 결과, 명령어 캐쉬와 데이터 캐쉬의 구조를 2 차 연관도로 구성하고 각 용량을 64 KB로 설정할 때 하드웨어의 비용 대 성능 효과가 가장 높았다.

In order to overcome the complexity and performance limit problems of superscalar processors, the multicore architecture has been prevalent recently. The configuration and the size of instruction and data caches greatly gives effect on the performance of multicore processors. Using SPEC 2000 benchmarks as input, the trace-driven simulation has been performed for the 2-core to 16-core architectures with different sizes of caches extensively. As a result, the 2-way set associative instruction and data cache with the size of 64KB brought the best cost-effective performance.

스마트폰의 메모리로 사용되고 있는 DRAM은 고집적화의 한계로 인해 더 이상 용량 증대가 어려울 뿐 아니라 배터리 소모의 상당 부분을 차지하는 것으로 분석되고 있다. 이에 비해 페이스북 등의 소셜 네트워크 서비스는 점점 많은 메모리를 필요로 하고 메모리 용량 부족시 스토리지를 접근하게 되어 상당히 느린 응답 시간을 나타내고 있다. 본 논문은 스마트폰 저장장치의 성능 개선을 위해 차세대 비휘발성메모리를 버퍼캐쉬로 탑재하고 이를 효율적으로 관리하는 알고리즘을 제안한다. 제안하는 기법은 파일의 쓰기 연산이 발생한 데이터를 비휘발성메모리에 보관하여 스토리지 접근 횟수를 크게 줄이는 동시에, 읽기 연산과 쓰기 연산의 기록을 별도로 유지하여 두 연산 모두 성능 저하가 발생하지 않도록 한다. 트레이스 기반 시뮬레이션 실험을 통해 제안한 기법이 기존 방식에 비해 성능이 개선되는 것을 보인다.

DRAM is commonly used as a smartphone memory medium, but extending its capacity is challenging due to DRAM’s large battery consumption and density limit. Meanwhile, smartphone applications such as social network services need increasingly large memory, resulting in long latency due to additional storage accesses. To alleviate this situation, we adopt emerging nonvolatile memory (NVRAM) as smartphone’s buffer cache and propose an efficient management scheme. The proposed scheme stores all dirty data in NVRAM, thereby reducing the number of storage accesses. Moreover, it separately exploits read and write histories of data accesses, leading to more efficient management of volatile and nonvolatile buffer caches, respectively. Trace-driven simulations show that the proposed scheme improves I/O performances significantly.

Worst-case execution time (WCET) analysis is critical for hard real-time systems to ensure that different tasks can meet their respective deadlines. While significant progress has been made for WCET analysis of instruction caches, the data cache timing analysis, especially for set-associative data caches, is rather limited. This paper proposes an approach to safely and tightly bounding data cache performance by computing the worst-case stack distance of data cache accesses. Our approach can not only be applied to direct-mapped caches, but also be used for set-associative or even fully-associative caches without increasing the complexity of analysis. Moreover, the proposed approach can statically categorize worst-case data cache misses into cold, conflict, and capacity misses, which can provide useful insights for designers to enhance the worst-case data cache performance. Our evaluation shows that the proposed data cache timing analysis technique can safely and accurately estimate the worst-case data cache performance, and the overestimation as compared to the observed worst-case data cache misses is within 1% on average.

Reduction in cache miss rates continues to be an important issue in processor design, especially at the last level cache(LLC). The Least Recently Used(LRU) replacement policy has been widely adapted in processors for the past decades. The LRU replacement policy represents the cache blocks in a set as LRU stack. So, it only use the recency information. And it is expensive to implement in hardware. Also in some workload, it raises the thrashing, i.e., blocks with high reuse evicting each other from the cache. We propose an new replacement policy to solve problems of LRU replacement policy. In the proposed policy, a incoming block is inserted at the bottom of the set. To exploit the frequency information, a hit block exchanges its position with its adjacent block above in the set. A victim block can be chosen from bottom. Selecting the bottom block as the victim makes the proposed policy to protect the cache from polluting by less frequently used blocks. It also proposes to divide the blocks in a cache set into groups to resolve the thrashing. The victim block is selected alternately from each group. The proposed policy reduces the average MPKI(Miss Per Kilo Instructions) of the baseline 1MB 16-way LLC cache using LRU replacement policy by 12.9%. It reduces the storage requirement by 47% compared to LRU replacement policy.

Although there are many efficient techniques to minimize the speed gap between processor and the memory, it remains a bottleneck for various commercial implementations. Since secondary memory technologies are much slower than main memory, it is challenging to match memory speed to the processor. Usually, hard disk drives include semiconductor caches to improve their performance. A hit in the disk cache eliminates the mechanical seek time and rotational latency. To further improve performance a divided disk cache, subdivided between metadata and data, has been proposed previously. We propose a new algorithm to apply the SSD that is flash memory-based solid state drive by applying FTL. First, this paper evaluates the performance of such a disk cache via simulations using DiskSim. Then, we perform an experiment to evaluate the performance of the proposed algorithm.