Abstract:
DRAM is a precious resource in extreme-scale
machines and is increasingly becoming scarce, mainly due to
the growing number of cores per node. On future multi-petaflop
and exaflop machines, the memory pressure is likely to be so
severe that we need to rethink our memory usage models.
Fortunately, the advent of non-volatile memory (NVM) offers
a unique opportunity in this space. Current NVM offerings
possess several desirable properties, such as low cost and power
efficiency, but also suffer from high latency and lifetime issues.
We need rich techniques to be able to use them alongside
DRAM.
In this paper, we propose a novel approach for exploiting
NVM as a secondary memory partition so that applications
can explicitly allocate and manipulate memory regions therein.
More specifically, we propose an NVMalloc library with a suite
of services that enables applications to access a distributed
NVM storage system. We have devised ways within NVMalloc
so that the storage system, built from compute node-local
NVM devices, can be accessed in a byte-addressable fashion
using the memory mapped I/O interface. Our approach has
the potential to re-energize out-of-core computations on large-
scale machines by having applications allocate certain variables
through NVMalloc, thereby increasing the overall memory
available for the application. Our evaluation on a 128-core
cluster shows that NVMalloc enables applications to compute
problem sizes larger than the physical memory in a cost-
effective manner. It can achieve better performance with
increased computation time between NVM memory accesses or
increased data access locality. In addition, our results suggest
that while NVMalloc enables transparent access to NVM-
resident variables, the explicit control it provides is crucial
to optimize application performance.