User Tools

Site Tools


realtime:documentation:howto:applications:memory

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
realtime:documentation:howto:applications:memory [2017/07/12 20:55]
jithu [Memory Locking] - wip - not final yet
realtime:documentation:howto:applications:memory [2017/07/15 02:25]
jithu [Memory Locking]
Line 7: Line 7:
  
 ===== Memory Locking ===== ===== Memory Locking =====
-Memory locking ​allows ​an application to instruct the kernel to lock virtual address space (either whole or in partsin RAM. Memory locking prevents an application'​s ​memory pages, from being paged-out, anytime during its lifetimeFurther memory ​locking ​calls (mlock or mlockall()) ​will trigger the necessary page-faults to bring in the relevant ​pages to the physical ​RAM and populates the application'​s page-table with the corresponding entriesThis means that the first access to the memory (following ​the mlock*() call) will already have physical memory assigned and will not page fault in the RT-critical path. This obviates ​the need to pre-fault the memory ​otherwise.+Memory locking ​APIs allow an application to instruct the kernel to associate ​(some or all of itsvirtual ​memory pages with real page frames and keep it that wayIn other words : 
 +  * Memory ​locking ​APIs will trigger the necessary page-faultsto bring in the pages being locked, ​to physical ​memoryConsequently ​first access to a locked-memory (following ​an ''​mlock*()'' ​call) will already have physical memory assigned and will not page fault (in  RT-critical path). This removes ​the need to explicitly ​pre-fault ​these memory.  
 +  * Further memory locking prevents an application'​s memory pages, from being paged-out, anytime during its lifetime even in when the overall system is facing ​memory ​pressure.
  
-Applications can either use ''​mlock(...)''​ or ''​mlockall(...)''​ for memory locking. While ''​mlock(...)''​ locks specific pages (described by //start// and //size//) whereas ​''​mlockall(...)'' ​instructs the kernel to lock it's entire virtual address space (i.e [[realtime:​documentation:​howto:​applications:​memory:​mlockall_globals_sample|globals]], ​[[realtime:​documentation:​howto:​applications:​memory:​mlockall_stack_sample|stack]], heap, code) in RAMThere is a tradeoff involved ​between convenience ​vs memory efficiency ​should drive the usage of mlock() ​vs mlockall().+Applications can either use ''​mlock(...)''​ or ''​mlockall(...)''​ for memory locking. ​Specifics of these C Library calls can be found here  
 +[[http://​www.gnu.org/​software/​libc/​manual/​html_node/​Locking-Pages.html|The GNU C Library: Locking pages]]. Note that these calls requires the application to have sufficient privileges (i.e.  [[http://​man7.org/​linux/​man-pages/​man7/​capabilities.7.html|CAP_IPC_LOCK capability]]) to succeed. 
 + 
 +While ''​mlock(<​addr>,​ <​length>​)''​ locks specific pages (described by //address// and //length//)''​mlockall(...)'' ​locks an application's entire virtual address space (i.e [[realtime:​documentation:​howto:​applications:​memory:​mlockall_globals_sample|globals]],​ stack, heap, code) in physical memoryThe trade-off ​between convenience ​and locking-up excess RAM should drive the choice ​of one over the other. Locking only those areas which are accessed by RT-threads (using ''​mlock(...)''​) could be cheaper than blindly using ''​mlockall(...)''​ which will end-up locking all memory pages of the application (i.e. even those which are used only by non-RT threads).
  
  
Line 20: Line 25:
   }   }
 </​code>​ </​code>​
-Specifics of the C Library call can be found here  
-[[http://​www.gnu.org/​software/​libc/​manual/​html_node/​Locking-Pages.html|The GNU C Library: Locking pages]]. Note that ''​mlockall(...)''​ requires the application to have sufficient privileges (i.e.  [[http://​man7.org/​linux/​man-pages/​man7/​capabilities.7.html|CAP_IPC_LOCK capability]]) to succeed. 
  
-Real-time applications should ​do ''​mlockall(...)'' ​early in their lifetime (i.e prior to spawning the real-time ​threads). Since memory access latency ​to a paged-out address is significantly worse (than if were  present ​in RAM), failing ​to do so may significantly impact the determinism of the application ​based on the overall ​system'​s ​memory pressure.+ 
 +Real-time applications should ​use memory-locking APIs early in their life, prior to performing ​real-time ​activities, so as to not incur page-faults ​in RT critical path. Failing ​to do so may significantly impact the determinism of the application.  
 + 
 +Note that memory locking is required irrespective of whether //swap area// is configured for a system or not. This is because pages for read-only memory areas (like program code) could be dropped from the memory, when the system ​is facing ​memory pressure. Such read-only pages (being identical to on-disk copy), would be brought back straight from the disk (and not swap), resulting in page-faults even on setups without a swap-memory.
 ===== Stack Memory for RT threads ===== ===== Stack Memory for RT threads =====
  
Line 50: Line 56:
  
  
-=====  Dynamic memory allocation ​for RT usage ===== +=====  Dynamic memory allocation in RT threads=====
-Dynamic memory allocations will result ​in page-faults,​ which will impact the determinism and latency of the real-time thread. So the suggested recipe is to pre-allocate the max amount of dynamic memory required (for the RT thread) during application startup and configure/​trick the dynamic-allocator to service runtime ''​malloc(...)''​ request'​s from this pre-allocated pool, so that page-faults are not incurred on the real-time critical path (with this approach, page-faults are incurred only during startup). +
- +
-Dynamic memory support in user space applications (written in C) is provided by the [[http://​www.gnu.org/​software/​libc/​manual/​html_node/​The-GNU-Allocator.html|GNU C Library'​s memory allocator]]. This allocator (i.e. C Library'​s userland allocator), gets memory from kernel via mmap/ sbrk syscalls. Based on the application'​s ''​malloc(...)/​ free(...)''​ usage, this allocator decides when to request more memory from kernel and when to release excess free memory back to kernel. Glibc offers interfaces that can be used to configure/​tailor these behaviors via the [[https://​www.gnu.org/​software/​libc/​manual/​html_node/​Malloc-Tunable-Parameters.html|mallopt()]] call (see the usage of ''​mallopt(...)''​ call in the following sample). Refer to  [[http://​www.usenix.org/​publications/​library/​proceedings/​als01/​full_papers/​ezolt/​ezolt.ps|Ezolt'​s]] paper for a more detailed discussion on glibc ''​malloc(...)''​ tuning. +
- +
-Dynamic memory related steps for an RT application is listed below: +
-  -  Configure glibc during startup to:  +
-      * Hold on to the allocated memory for the programs entire life-cycle (i.e. to never release memory back to kernel, even when the user-space application has freed it). +
-      * Not use mmap underneath. +
-  - Further: +
-      * ''​malloc(...)''​ the maximum amount of memory required.  +
-      * ''​free(...)''​ all of it. +
-  - Subsequent ''​malloc(...)''​s (within RT critical path) will be serviced without page-faults. +
- +
-//As with all real-time memory related stuff, ​ we need to do ''​mlockall(...)'',​ prior to this, to force page-faults during the initial dummy ''​malloc(...)''​. +
-// +
- +
- +
-The following example shows how we can create a pool of memory during startup, and lock it into memory. At startup a block of memory is allocated through the ''​malloc(...)''​ call. Prior to it Glibc will be configured such that it uses the sbrk() call to fulfill this allocation. After locking it, we can free this block of memory, knowing that it is not released to the kernel and still assigned to our application. +
- +
-We have now created a pool of memory that will be used by Glibc for dynamic memory allocation. We can ''​malloc(...)''​ and ''​free(...)''​ as much as we want without being interfered by any page fault. Even if the system is fully stressed, and swapping is continuously active, the RT-application will never run into any page fault... +
- +
-<code c> +
-#include <​malloc.h>​ +
-#include <​stdio.h>​ +
-#include <​stdlib.h>​ +
-#include <​sys/​mman.h>​ /* needed for mlockall() */ +
-#include <​sys/​resource.h>​ /* needed for getrusage */ +
-#include <​sys/​time.h>​ /* needed for getrusage */ +
-#include <​unistd.h>​ /* needed for sysconf(int name) */ +
- +
-#define MAX_DYN_SIZE (100*1024*1024) /* 100MB */ +
- +
-int main(int argc, char *argv[]) +
-+
- struct rusage usage; +
- int i, page_size;​ +
- char *buffer; +
- +
- /* Lock all current and future pages from preventing of being paged */ +
- if (mlockall(MCL_CURRENT | MCL_FUTURE)) { +
- perror("​mlockall failed"​);​ +
- exit(-1);​ +
-+
- +
- /* Turn off malloc trimming */ +
- if (mallopt(M_TRIM_THRESHOLD,​ -1) !1) { +
- printf("​mallopt M_TRIM_THRESHOLD failed\n"​);​ +
- exit(-1);​ +
-+
- +
- /* Turn off mmap usage */ +
- if (mallopt(M_MMAP_MAX,​ 0) !1) { +
- printf("​mallopt M_MMAP_MAX failed\n"​);​ +
- exit(-1);​ +
-+
- +
- page_size ​sysconf(_SC_PAGESIZE);​ +
- +
- /* Dummy malloc to force pagefaults during startup */ +
- buffer ​malloc(MAX_DYN_SIZE);​ +
- if (buffer ​== NULL) { +
- printf("​malloc failed\n"​);​ +
- exit(-1);​ +
-+
- +
- getrusage(RUSAGE_SELF,​ &​usage);​ +
- printf("​Major-pagefaults:​%ld,​ Minor Pagefaults:​%ld\n",​ +
- usage.ru_majflt,​ usage.ru_minflt);​ +
- +
- /* Touch page to prove there will be no page fault later */ +
- for (i = 0; i < MAX_DYN_SIZE;​ i += page_size) { +
- /*  +
-                 * Each write to this buffer will *not* generate a pagefault. +
- * Even if nothing has been written to the newly allocated +
- * memory, the physical page is still provisioned to the process +
- * because mlockall() has been called with the MCL_FUTURE flag +
- */ +
- buffer[i] = 0; +
- /*  +
-                 * print the number of major and minor pagefaults this +
- * application has triggered +
- */ +
- getrusage(RUSAGE_SELF,​ &​usage);​ +
- printf("​Major-pagefaults:​%ld,​ Minor Pagefaults:​%ld\n",​ +
- usage.ru_majflt,​ usage.ru_minflt);​ +
-+
- free(buffer);​ +
- +
- /*  +
-         * Dummy free, buffer is now released. As glibc is configured such +
- * that it never gives back memory to the kernel, the memory allocated +
- * above is locked for this process. All subsequent malloc() +
- * calls come from the memory pool reserved and locked above. Issuing +
- * free() does NOT make this locking undone. So, with this +
- * locking mechanism we can build applications that will never run +
- * into a major/minor pagefault, even with swapping enabled. +
- */ +
- +
- /* <do your RT stuff> */ +
-        create_rt_thread();​ +
- +
- return 0; +
-+
- +
-</​code>​ +
- +
-Q. Is is always possible to know the required size of Dynamic memory beforehand ? +
- +
-A. While this approach of ''​malloc(...)/​free(...)''​ ing the maximum required dynamic-memory at startup provides a straightforward way of adapting existing source code to real-time, it is debatable as to whether it is possible to accurately pre-assess this size of needed dynamic memory beforehand. We have to keep in mind that a real-time application cannot possibly go on asking beyond a finite amount of memory- lest there is no difference between RT/Non-RT application from a memory perspective.+
  
-Q. The aforementioned technique ​is specific ​to applications written in C. How do I do this for applications in other languages ​say PythonJava ?+Real-time threads should avoid doing dynamic memory allocation / freeing while in RT critical path. The suggested recommendation for real-time threads, ​is to do the allocations,​ prior-to entering RT critical path. Subsequently RT threads, within their RT-critical path, can use this pre-allocated dynamic memoryprovided that it is locked as described [[memory#​Memory Locking|here]].
  
-A. Language specifics TBD. There should be a way to configure the dynamic memory allocator to operate from a pre-allocated pool of memory. During startup of the application,​ you have to program the base and extent of this pool meant for dynamic ​memory and program the allocator to use this.+Non RT-threads within ​the applications ​have no restrictions on dynamic ​allocation / free
  
realtime/documentation/howto/applications/memory.txt · Last modified: 2017/07/15 02:25 by jithu