(Currently WIP (Work in Progress), Ignore the contents until this line is removed )

This page is meant to document the things to be considered from a Real Time Application's memory perspective. The following things fall into the purview of this topic:

1. Memory Locking
2. Stack Memory for RT threads
3. Dynamic memory allocation and prefaulting

Keep in mind that the usual sequence is for an app to begin its execution as a regular(non RT) App, then create the RT threads with appropriate resources and scheduling parameters.

Through the mlockall() system-call, it is possible for an app to instruct the kernel to lock the calling process's entire virtual address space - current and future - into RAM, thereby preventing it from being paged-out to the swap during the process's lifetime. See the snippet below:

  /* Lock all current and future pages from preventing of being paged to swap */
  if (mlockall( MCL_CURRENT | MCL_FUTURE ))
  {
      perror("mlockall failed");
  }

Real-time apps should do this at first (i.e prior to spawning the real-time threads), since, memory access latency of a paged-out address will be significantly worse (than if it is present in the RAM). Consequently, failing to do so will significantly affect the determinism based on the overall system's memory pressure.

Note that this call would be applicable for all memoriy areas in the process address space - i.e globals, stack, heap, code etc

All threads(RT and non-RT) within a process have their own private stack. The aforementioned mlockall is sufficient to pin the entire thread stack in RAM. Note that it is possible to specify a different size for a thread's stack-size(default being 8MB) as shown in the snippet. If the process spawns a large number of RT threads it is advisable to reduce the stack-sizes as these-too cannot be paged out.

 static void create_rt_thread(void)
 {
   pthread_t thread;
   pthread_attr_t attr;
 
   /* init to default values */
   if (pthread_attr_init(&attr))
   	error(1);
   /* Set a smaller stack   */
   if (pthread_attr_setstacksize(&attr, PTHREAD_STACK_MIN + MY_STACK_SIZE))
   	error(2);
   /* And finally start the actual thread */
   pthread_create(&thread, &attr, rt_func, NULL);
 }

Details : The entire stack of every thread inside the application is forced to RAM when mlockall(MCL_CURRENT) is called. Threads started after a call to mlockall(MCL_CURRENT | MCL_FUTURE) will generate page faults immediately since the new stack is immediately forced to RAM (due to the MCL_FUTURE flag).

With mlockall(), page faults for thread stack area, happens during thread creation (no explicit additional prefaulting necessary to avoid pagefaults during first access). So all threads need to be created at startup time, before RT show time.

Dynamic memory allocations will result in page-faults, which will impact the determinism / latency of the real-time thread. So it is recommended to pre-allocate the max amount of dynamic memory required prior to the RT show time (and free() - explained later), so that pagefaults are not incurred on the critical path.