This invention introduces a novel method and system for optimizing memory management in GPUs.
Description
By dynamically adjusting the threshold for page migration based on access patterns, this technology significantly enhances performance, particularly for applications with irregular memory access patterns. This innovation promises up to 80% performance improvement under memory oversubscription conditions, making it a game-changer for GPU-based computing.
Applications
• Diagnostic Imaging
• Therapeutic Medical Device
• Drug Discovery - Bioinformatics
• Gene Therapy - CRISPR
• High Performance Computing
Advantages
The proposed system offers several advantages over existing memory management solutions for GPUs. Traditional methods rely on static thresholds and compile-time user hints, which can be inefficient and inflexible. In contrast, this invention uses a dynamic threshold based on real-time access patterns, allowing for more adaptive and efficient memory management. By prioritizing cold pages for eviction and pinning hot pages to device memory, the system reduces thrashing and improves performance by up to 80% for applications with irregular memory access patterns. This approach is particularly beneficial under memory oversubscription conditions, where it can significantly enhance computational efficiency and reduce latency.
Invention Readiness
The technology is currently at the prototype stage, with initial testing demonstrating significant performance improvements. The next steps involve further validation and optimization of the system, followed by integration with existing GPU architectures. Collaboration with industry partners such as NVIDIA, AMD, and Alibaba Cloud, who have expressed interest in this technology, will be crucial for its commercialization.
IP Status
https://patents.google.com/patent/WO2020226880A1Related Publication(s)
Ganguly, D., Zhang, Z., Yang, J., & Melhem, R. (2020). Adaptive Page Migration for Irregular Data-intensive Applications under GPU Memory Oversubscription. In 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS). 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE. https://doi.org/10.1109/ipdps47924.2020.00054
