This invention introduces a novel system and method for managing memory oversubscription in GPUs.
Description
By leveraging a full binary tree structure, this invention dynamically adapts page eviction granularity, significantly improving performance and reducing memory thrashing. This innovative approach addresses the limitations of existing fixed granularity methods, offering a more efficient and flexible solution for memory management in GPUs.
Applications
• Memory Management
• Performance Improvement
• Flexibility
• Reduced Memory Thrashing
Advantages
The proposed invention offers several advantages over existing memory management solutions. Traditional methods often rely on fixed page eviction granularity, which can either disable hardware prefetchers (in the case of 4KB pages) or cause severe memory thrashing (with 2MB pages). This invention overcomes these limitations by dynamically adapting the eviction granularity based on current memory usage and access patterns. This adaptability leads to significant performance improvements, with the system providing an order of magnitude better performance compared to 4KB eviction and up to 60% improvement over 2MB eviction granularity. Additionally, the use of a full binary tree structure allows for more granular tracking and management of memory pages, further enhancing efficiency and reducing overhead.
Invention Readiness
The invention is currently at the prototype stage, demonstrating its feasibility and potential benefits. Initial testing has shown promising results, with significant performance improvements over existing methods. The next steps in development include further validation and optimization of the system, as well as potential collaborations with industry partners for real-world testing and implementation.
IP Status
https://patents.google.com/patent/WO2020226878A1Related Publication(s)
Ganguly, D., Zhang, Z., Yang, J., & Melhem, R. (2019). Interplay between hardware prefetcher and page eviction policy in CPU-GPU unified virtual memory. In Proceedings of the 46th International Symposium on Computer Architecture (pp. 224–235). ISCA ’19: The 46th Annual International Symposium on Computer Architecture. ACM. https://doi.org/10.1145/3307650.3322224
