.\" Automatically generated by Pandoc 2.1.3
.\"
.TH "LIBVMMALLOC" "7" "2018-07-18" "PMDK - vmmalloc API version 1.1" "PMDK Programmer's Manual"
.hy
.\" Copyright 2014-2018, Intel Corporation
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\"
.\"     * Redistributions of source code must retain the above copyright
.\"       notice, this list of conditions and the following disclaimer.
.\"
.\"     * Redistributions in binary form must reproduce the above copyright
.\"       notice, this list of conditions and the following disclaimer in
.\"       the documentation and/or other materials provided with the
.\"       distribution.
.\"
.\"     * Neither the name of the copyright holder nor the names of its
.\"       contributors may be used to endorse or promote products derived
.\"       from this software without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.SH NAME
.PP
\f[B]libvmmalloc\f[] \- general purpose volatile memory allocation
library
.SH SYNOPSIS
.IP
.nf
\f[C]
$\ LD_PRELOAD=libvmmalloc.so.1\ command\ [\ args...\ ]
\f[]
.fi
.PP
or
.IP
.nf
\f[C]
#include\ <stdlib.h>
#ifndef\ __FreeBSD__
\ \ \ \ #include\ <malloc.h>
#else
\ \ \ \ #include\ <malloc_np.h>
#endif
#include\ <libvmmalloc.h>

cc\ [\ flag...\ ]\ file...\ \-lvmmalloc\ [\ library...\ ]
\f[]
.fi
.IP
.nf
\f[C]
void\ *malloc(size_t\ size);
void\ free(void\ *ptr);
void\ *calloc(size_t\ number,\ size_t\ size);
void\ *realloc(void\ *ptr,\ size_t\ size);
int\ posix_memalign(void\ **memptr,\ size_t\ alignment,\ size_t\ size);
void\ *aligned_alloc(size_t\ alignment,\ size_t\ size);
void\ *memalign(size_t\ alignment,\ size_t\ size);
void\ *valloc(size_t\ size);
void\ *pvalloc(size_t\ size);
size_t\ malloc_usable_size(const\ void\ *ptr);
void\ cfree(void\ *ptr);
\f[]
.fi
.SH DESCRIPTION
.PP
\f[B]libvmmalloc\f[] transparently converts all dynamic memory
allocations into Persistent Memory allocations.
.PP
The typical usage of \f[B]libvmmalloc\f[] does not require any
modification of the target program.
It is enough to load \f[B]libvmmalloc\f[] before all other libraries by
setting the environment variable \f[B]LD_PRELOAD\f[].
When used in that way, \f[B]libvmmalloc\f[] interposes the standard
system memory allocation routines, as defined in \f[B]malloc\f[](3),
\f[B]posix_memalign\f[](3) and \f[B]malloc_usable_size\f[](3), and
provides that all dynamic memory allocations are made from a \f[I]memory
pool\f[] built on a memory\-mapped file, instead of the system heap.
The memory managed by \f[B]libvmmalloc\f[] may have different
attributes, depending on the file system containing the memory\-mapped
file.
In particular, \f[B]libvmmalloc\f[] is part of the \f[I]Persistent
Memory Development Kit\f[] because it is sometimes useful to use
non\-volatile memory as a volatile memory pool, leveraging its capacity,
cost, or performance characteristics.
.PP
\f[B]libvmmalloc\f[] may be also linked to the program, by providing the
**\-lvmmalloc* argument to the linker.
Then it becomes the default memory allocator for the program.
.RS
.PP
NOTE: Due to the fact the library operates on a memory\-mapped file,
\f[B]it may not work properly with programs that perform fork(2) not
followed by exec(3).\f[] There are two variants of experimental
\f[B]fork\f[](2) support available in libvmmalloc.
The desired library behavior may be selected by setting the
\f[B]VMMALLOC_FORK\f[] environment variable.
By default variant #1 is enabled.
See \f[B]ENVIRONMENT\f[] for more details.
.RE
.PP
\f[B]libvmmalloc\f[] uses the \f[B]mmap\f[](2) system call to create a
pool of volatile memory.
The library is most useful when used with \f[I]Direct Access\f[] storage
(DAX), which is memory\-addressable persistent storage that supports
load/store access without being paged via the system page cache.
A Persistent Memory\-aware file system is typically used to provide this
type of access.
Memory\-mapping a file from a Persistent Memory\-aware file system
provides the raw memory pools, and this library supplies the traditional
\f[I]malloc\f[] interfaces on top of those pools.
.PP
The memory pool acting as a system heap replacement is created
automatically at library initialization time.
The user may control its location and size by setting the environment
variables described in \f[B]ENVIRONMENT\f[], below.
The allocated file space is reclaimed when the process terminates or in
case of system crash.
.PP
Under normal usage, \f[B]libvmmalloc\f[] will never print messages or
intentionally cause the process to exit.
The library uses \f[B]pthreads\f[](7) to be fully MT\-safe, but never
creates or destroys threads itself.
The library does not make use of any signals, networking, and never
calls \f[B]select\f[](2) or \f[B]poll\f[](2).
.SH ENVIRONMENT
.PP
The \f[B]VMMALLOC_POOL_DIR\f[] and \f[B]VMMALLOC_POOL_SIZE\f[]
environment variables \f[B]must\f[] be set for \f[B]libvmmalloc\f[] to
work properly.
If either of them is not specified, or if their values are not valid,
the library prints an appropriate error message and terminates the
process.
Any other environment variables are optional.
.IP \[bu] 2
\f[B]VMMALLOC_POOL_DIR\f[]=\f[I]path\f[]
.PP
Specifies a path to the directory where the memory pool file should be
created.
The directory must exist and be writable.
.IP \[bu] 2
\f[B]VMMALLOC_POOL_SIZE\f[]=\f[I]len\f[]
.PP
Defines the desired size (in bytes) of the memory pool file.
It must be not less than the minimum allowed size
\f[B]VMMALLOC_MIN_POOL\f[] as defined in \f[B]<libvmmalloc.h>\f[].
.RS
.PP
NOTE: Due to the fact the library adds some metadata to the memory pool,
the amount of actual usable space is typically less than the size of the
memory pool file.
.RE
.IP \[bu] 2
\f[B]VMMALLOC_FORK\f[]=\f[I]val\f[] (EXPERIMENTAL)
.PP
\f[B]VMMALLOC_FORK\f[] controls the behavior of \f[B]libvmmalloc\f[] in
case of \f[B]fork\f[](3), and can be set to the following values:
.IP \[bu] 2
\f[B]0\f[] \- \f[B]fork\f[](2) support is disabled.
The behavior of \f[B]fork\f[](2) is undefined in this case, but most
likely results in memory pool corruption and a program crash due to
segmentation fault.
.IP \[bu] 2
\f[B]1\f[] \- The memory pool file is remapped with the
\f[B]MAP_PRIVATE\f[] flag before the fork completes.
From this moment, any access to memory that modifies the heap pages,
both in the parent and in the child process, will trigger creation of a
copy of those pages in RAM (copy\-on\-write).
The benefit of this approach is that it does not significantly increase
the time of the initial fork operation, and does not require additional
space on the file system.
However, all subsequent memory allocations, and modifications of any
memory allocated before fork, will consume system memory resources
instead of the memory pool.
.PP
This is the default option if \f[B]VMMALLOC_FORK\f[] is not set.
.IP \[bu] 2
\f[B]2\f[] \- A copy of the entire memory pool file is created for the
use of the child process.
This requires additional space on the file system, but both the parent
and the child process may still operate on their memory pools, not
consuming system memory resources.
.RS
.PP
NOTE: In case of large memory pools, creating a copy of the pool file
may stall the fork operation for a quite long time.
.RE
.IP \[bu] 2
\f[B]3\f[] \- The library first attempts to create a copy of the memory
pool (as for option #2), but if it fails (i.e.\ because of insufficient
free space on the file system), it will fall back to option #1.
.RS
.PP
NOTE: Options \f[B]2\f[] and \f[B]3\f[] are not currently supported on
FreeBSD.
.RE
.PP
Environment variables used for debugging are described in
\f[B]DEBUGGING\f[], below.
.SH CAVEATS
.PP
\f[B]libvmmalloc\f[] relies on the library destructor being called from
the main thread.
For this reason, all functions that might trigger destruction (e.g.
\f[B]dlclose\f[](3)) should be called in the main thread.
Otherwise some of the resources associated with that thread might not be
cleaned up properly.
.SH DEBUGGING
.PP
Two versions of \f[B]libvmmalloc\f[] are typically available on a
development system.
The normal version is optimized for performance.
That version skips checks that impact performance and never logs any
trace information or performs any run\-time assertions.
A second version, accessed when using libraries from
\f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace
points.
The typical way to access the debug version is to set the
\f[B]LD_LIBRARY_PATH\f[] environment variable to
\f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as
appropriate.
Debugging output is controlled using the following environment
variables.
These variables have no effect on the non\-debug version of the library.
.IP \[bu] 2
\f[B]VMMALLOC_LOG_LEVEL\f[]
.PP
The value of \f[B]VMMALLOC_LOG_LEVEL\f[] enables trace points in the
debug version of the library, as follows:
.IP \[bu] 2
\f[B]0\f[] \- Tracing is disabled.
This is the default level when \f[B]VMMALLOC_LOG_LEVEL\f[] is not set.
.IP \[bu] 2
\f[B]1\f[] \- Additional details on any errors detected are logged, in
addition to returning the \f[I]errno\f[]\-based errors as usual.
.IP \[bu] 2
\f[B]2\f[] \- A trace of basic operations is logged.
.IP \[bu] 2
\f[B]3\f[] \- Enables a very verbose amount of function call tracing in
the library.
.IP \[bu] 2
\f[B]4\f[] \- Enables voluminous tracing information about all memory
allocations and deallocations.
.PP
Unless \f[B]VMMALLOC_LOG_FILE\f[] is set, debugging output is written to
\f[I]stderr\f[].
.IP \[bu] 2
\f[B]VMMALLOC_LOG_FILE\f[]
.PP
Specifies the name of a file where all logging information should be
written.
If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of
the current process will be appended to the file name when the log file
is created.
If \f[B]VMMALLOC_LOG_FILE\f[] is not set, output is written to
\f[I]stderr\f[].
.IP \[bu] 2
\f[B]VMMALLOC_LOG_STATS\f[]
.PP
Setting \f[B]VMMALLOC_LOG_STATS\f[] to 1 enables logging human\-readable
summary statistics at program termination.
.SH NOTES
.PP
Unlike the normal \f[B]malloc\f[](3), which asks the system for
additional memory when it runs out, \f[B]libvmmalloc\f[] allocates the
size it is told to and never attempts to grow or shrink that memory
pool.
.SH BUGS
.PP
\f[B]libvmmalloc\f[] may not work properly with programs that perform
\f[B]fork\f[](2) and do not call \f[B]exec\f[](3) immediately
afterwards.
See \f[B]ENVIRONMENT\f[] for more details about experimental
\f[B]fork\f[](2) support.
.PP
If logging is enabled in the debug version of the library and the
process performs \f[B]fork\f[](2), no new log file is created for the
child process, even if the configured log file name ends with
\[lq]\-\[rq].
All logging information from the child process will be written to the
log file owned by the parent process, which may lead to corruption or
partial loss of log data.
.PP
Malloc hooks (see \f[B]malloc_hook\f[](3)), are not supported when using
\f[B]libvmmalloc\f[].
.SH ACKNOWLEDGEMENTS
.PP
\f[B]libvmmalloc\f[] depends on jemalloc, written by Jason Evans, to do
the heavy lifting of managing dynamic memory allocation.
See: <http://www.canonware.com/jemalloc>
.SH SEE ALSO
.PP
\f[B]fork\f[](2), \f[B]dlclose(3)\f[], \f[B]exec\f[](3),
\f[B]malloc\f[](3), \f[B]malloc_usable_size\f[](3),
\f[B]posix_memalign\f[](3), \f[B]libpmem\f[](7), \f[B]libvmem\f[](7) and
\f[B]<http://pmem.io>\f[]
.PP
On Linux:
.PP
\f[B]jemalloc\f[](3), \f[B]malloc_hook\f[](3), \f[B]pthreads\f[](7),
\f[B]ld.so\f[](8)
.PP
On FreeBSD:
.PP
\f[B]ld.so\f[](1), \f[B]pthread\f[](3)