Summary
-------

Enable the HotSpot VM to allocate the Java object heap on an alternative memory device, such as an NV-DIMM, specified by the user.

Motivation
----------

With the availability of cheap NV-DIMM memory, future systems may be equipped with heterogeneous memory architectures. One example of such technology is Intel's 3D XPoint. Such an architecture, in addition to DRAM, will have one or more types of non-DRAM memory with different characteristics.

This JEP targets alternative memory devices that have the same semantics as DRAM, including the semantics of atomic operations, and can therefore be used instead of DRAM for the object heap without any change to existing application code. All other memory structures such as the code heap, metaspace, thread stacks, etc., will continue to reside in DRAM. 

Some use cases for this proposal are:

1. In multi-JVM deployments some JVMs such as daemons, services, etc., have lower priority than others. NV-DIMMs would potentially have higher access latency compared to DRAM. Low-priority processes can use NV-DIMM memory for the heap, allowing high-priority processes to use more DRAM.

2. Applications such as big data and in-memory databases have an ever-increasing demand for memory. Such applications could use NV-DIMM for the heap since NV-DIMMs would potentially have a larger capacity, at lower cost, compared to DRAM.

Description
-----------

Some operating systems already expose non-DRAM memory through the file system. Examples are [NTFS DAX mode][ntfs-dax] and [ext4 DAX][ext4-dax]. Memory-mapped files in these file systems bypass the page cache and provide a direct mapping of virtual memory to the physical memory on the device.

[ntfs-dax]: https://channel9.msdn.com/events/build/2016/p470
[ext4-dax]: https://lwn.net/Articles/618064

To allocate the heap in such memory we can add a new option, `-XX:AllocateHeapAt=<path>`. This option would take a path to the file system and use memory mapping to achieve the desired result of allocating the object heap on the memory device. The JEP does not intend to share a non-volatile region between multiple running JVMs or re-use the same region for further invocations of the JVM.

The existing heap related flags such as `-Xmx`, `-Xms`, etc., and garbage-collection related flags would continue to work as before.

To ensure application security the implementation must ensure that file(s) created in the file system are:

1. Protected by correct permissions, to prevent other users from accessing it.
2. Removed when the application terminates, in any possible scenario.

Testing
-------

Testing will not necessarily require any special memory; it can be performed on in-memory file systems such as ramfs or tmpfs.