User Tools

Site Tools


realtime:documentation:howto:applications:memory

This is an old revision of the document!


Memory for Real-time Applications

Proper handling of memory will improve a real-time application's deterministic behavior. Three areas of memory management within the purview of a real-time application are considered :

  1. Memory Locking
  2. Stack Memory for RT threads
  3. Dynamic memory allocation

Keep in mind that the usual sequence is for an application to begin its execution as a regular (non-RT) application, then create the RT threads with appropriate resources and scheduling parameters.

Memory Locking

Memory locking will prevent, some or whole of an application's memory pages, from being paged-out to the swap, anytime during its lifetime. Also note that this will populate the applications page table with backing pages. This means that the first write access to the memory will already have physical memory assigned and will not page fault due to COW from the ZERO-page. This obviates the need to pre-fault the memory otherwise.

Through the mlockall(…) call, it is possible for an application to instruct the kernel to lock it's entire virtual address space (i.e globals, stack, heap, code) in RAM.

The snippet below illustrates the usage of mlockall(…) :

  /* Lock all current and future pages from preventing of being paged to swap */
  if (mlockall( MCL_CURRENT | MCL_FUTURE )) { 
          perror("mlockall failed");
          /* exit(-1) or do error handling */
  }

Specifics of the C Library call can be found here The GNU C Library: Locking pages. Note that mlockall(…) requires the application to have sufficient privileges (i.e. CAP_IPC_LOCK capability) to succeed.

Real-time applications should do mlockall(…) early in their lifetime (i.e prior to spawning the real-time threads). Since memory access latency to a paged-out address is significantly worse (than if were present in RAM), failing to do so may significantly impact the determinism of the application based on the overall system's memory pressure.

Stack Memory for RT threads

All threads (RT and non-RT) within an application have their own private stack. It is recommended that an application should understand the stack size needs for its RT threads and set them explicitly before spawning them. This can be done via the pthread_attr_setstacksize(…) call as shown in the snippet below. If the size is not explicitly set, then the thread gets the default stack size (pthread_attr_getstacksize() can be used to find out how much this is, it was 8MB at the time of this writing).

Aforementioned mlockall(…) is sufficient to pin the entire thread stack in RAM, so that pagefaults are not incurred while the thread stack is being used. If the application spawns a large number of RT threads, it is advisable to specify a smaller stack size (than the default) in the interest of not exhausting memory.

 static void create_rt_thread(void)
 {
         pthread_t thread;
         pthread_attr_t attr;
 
         /* init to default values */
         if (pthread_attr_init(&attr))
   	         error(1);
         /* Set a specific stack size   */
         if (pthread_attr_setstacksize(&attr, PTHREAD_STACK_MIN + MY_STACK_SIZE))
   	        error(2);
         /* And finally start the actual thread */
         pthread_create(&thread, &attr, rt_func, NULL);
 }

Details: The entire stack of every thread inside the application is forced to RAM when mlockall(MCL_CURRENT) is called. Threads created after a call to mlockall(MCL_CURRENT | MCL_FUTURE) will generate page faults immediately (on creation), as the new stack is immediately forced to RAM (due to the MCL_FUTURE flag). So all RT threads need to be created at startup time, before the RT show time. With mlockall(…) no explicit additional prefaulting necessary to avoid pagefaults during first (or subsequent) access.

Dynamic memory allocation for RT usage

Dynamic memory allocations will result in page-faults, which will impact the determinism and latency of the real-time thread. So the suggested recipe is to pre-allocate the max amount of dynamic memory required (for the RT thread) during application startup and configure/trick the dynamic-allocator to service runtime malloc(…) request's from this pre-allocated pool, so that page-faults are not incurred on the real-time critical path (with this approach, page-faults are incurred only during startup).

Dynamic memory support in user space applications (written in C) is provided by the GNU C Library's memory allocator. This allocator (i.e. C Library's userland allocator), gets memory from kernel via mmap/ sbrk syscalls. Based on the application's malloc(…)/ free(…) usage, this allocator decides when to request more memory from kernel and when to release excess free memory back to kernel. Glibc offers interfaces that can be used to configure/tailor these behaviors via the mallopt() call (see the usage of mallopt(…) call in the following sample). Refer to Ezolt's paper for a more detailed discussion on glibc malloc(…) tuning.

Dynamic memory related steps for an RT application is listed below:

  1. Configure glibc during startup to:
    • Hold on to the allocated memory for the programs entire life-cycle (i.e. to never release memory back to kernel, even when the user-space application has freed it).
    • Not use mmap underneath.
  2. Further:
    • malloc(…) the maximum amount of memory required.
    • free(…) all of it.
  3. Subsequent malloc(…)s (within RT critical path) will be serviced without page-faults.

As with all real-time memory related stuff, we need to do mlockall(…), prior to this, to force page-faults during the initial dummy malloc(…).

The following example shows how we can create a pool of memory during startup, and lock it into memory. At startup a block of memory is allocated through the malloc(…) call. Prior to it Glibc will be configured such that it uses the sbrk() call to fulfill this allocation. After locking it, we can free this block of memory, knowing that it is not released to the kernel and still assigned to our application.

We have now created a pool of memory that will be used by Glibc for dynamic memory allocation. We can malloc(…) and free(…) as much as we want without being interfered by any page fault. Even if the system is fully stressed, and swapping is continuously active, the RT-application will never run into any page fault…

#include <malloc.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h> /* needed for mlockall() */
#include <sys/resource.h> /* needed for getrusage */
#include <sys/time.h> /* needed for getrusage */
#include <unistd.h> /* needed for sysconf(int name) */
 
#define MAX_DYN_SIZE (100*1024*1024) /* 100MB */
 
int main(int argc, char *argv[])
{
	struct rusage usage;
	int i, page_size;
	char *buffer;
 
	/* Lock all current and future pages from preventing of being paged */
	if (mlockall(MCL_CURRENT | MCL_FUTURE)) {
		perror("mlockall failed");
		exit(-1);
	}
 
	/* Turn off malloc trimming */
	if (mallopt(M_TRIM_THRESHOLD, -1) != 1) {
		printf("mallopt M_TRIM_THRESHOLD failed\n");
		exit(-1);
	}
 
	/* Turn off mmap usage */
	if (mallopt(M_MMAP_MAX, 0) != 1) {
		printf("mallopt M_MMAP_MAX failed\n");
		exit(-1);
	}
 
	page_size = sysconf(_SC_PAGESIZE);
 
	/* Dummy malloc to force pagefaults during startup */
	buffer = malloc(MAX_DYN_SIZE);
	if (buffer == NULL) {
		printf("malloc failed\n");
		exit(-1);
	}
 
	getrusage(RUSAGE_SELF, &usage);
	printf("Major-pagefaults:%ld, Minor Pagefaults:%ld\n",
			usage.ru_majflt, usage.ru_minflt);
 
	/* Touch page to prove there will be no page fault later */
	for (i = 0; i < MAX_DYN_SIZE; i += page_size) {
		/* 
                 * Each write to this buffer will *not* generate a pagefault.
		 * Even if nothing has been written to the newly allocated
		 * memory, the physical page is still provisioned to the process
		 * because mlockall() has been called with the MCL_FUTURE flag
		 */
		buffer[i] = 0;
		/* 
                 * print the number of major and minor pagefaults this
		 * application has triggered
		 */
		getrusage(RUSAGE_SELF, &usage);
		printf("Major-pagefaults:%ld, Minor Pagefaults:%ld\n",
				usage.ru_majflt, usage.ru_minflt);
	}
	free(buffer);
 
	/* 
         * Dummy free, buffer is now released. As glibc is configured such
	 * that it never gives back memory to the kernel, the memory allocated
	 * above is locked for this process. All subsequent malloc()
	 * calls come from the memory pool reserved and locked above. Issuing
	 * free() does NOT make this locking undone. So, with this
	 * locking mechanism we can build applications that will never run
	 * into a major/minor pagefault, even with swapping enabled.
	 */
 
	/* <do your RT stuff> */
        create_rt_thread();
 
	return 0;
}

Q. Is is always possible to know the required size of Dynamic memory beforehand ?

A. While this approach of malloc(…)/free(…) ing the maximum required dynamic-memory at startup provides a straightforward way of adapting existing source code to real-time, it is debatable as to whether it is possible to accurately pre-assess this size of needed dynamic memory beforehand. We have to keep in mind that a real-time application cannot possibly go on asking beyond a finite amount of memory- lest there is no difference between RT/Non-RT application from a memory perspective.

Q. The aforementioned technique is specific to applications written in C. How do I do this for applications in other languages - say Python, Java ?

A. Language specifics TBD. There should be a way to configure the dynamic memory allocator to operate from a pre-allocated pool of memory. During startup of the application, you have to program the base and extent of this pool meant for dynamic memory and program the allocator to use this.

realtime/documentation/howto/applications/memory.1499795958.txt.gz · Last modified: 2017/07/11 17:59 by jithu