Embedded/lwmem
1
0
Fork 0
mirror of https://github.com/MaJerle/lwmem.git synced 2025-01-13 21:42:53 +08:00
Code Issues Projects Releases Wiki Activity

Update documentation

Browse Source
This commit is contained in:
Tilen Majerle 2019-12-07 03:49:49 +01:00
parent 9e7f144acc
commit 0cce822c7b
24 changed files with 506 additions and 910 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/api-reference/index.rst
View File

@ -6,4 +6,6 @@ List of all the modules:
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2
ringbuff lwmem
lwmem_config
lwmem_sys

4
docs/api-reference/lwmem.rst
View File

@ -1,8 +1,6 @@
.. contents:: .. _api_lwmem:
LwMEM LwMEM
===== =====
.. contents::
.. doxygengroup:: LWMEM .. doxygengroup:: LWMEM

6
docs/api-reference/lwmem_config.rst Normal file
View File

@ -0,0 +1,6 @@
.. _api_lwmem_config:
LwMEM Configuration
===================
.. doxygengroup:: LWMEM_CONFIG

9
docs/api-reference/lwmem_sys.rst Normal file
View File

@ -0,0 +1,9 @@
.. _api_lwmem_sys:
System functions
================
System function are used in conjunction with thread safety.
Please check :ref:`thread_safety` section for more information
.. doxygengroup:: LWMEM_SYS

12
docs/conf.py
View File

@ -17,9 +17,9 @@ from sphinx.builders.html import StandaloneHTMLBuilder
import subprocess, os import subprocess, os
# Run doxygen first # Run doxygen first
read_the_docs_build = os.environ.get('READTHEDOCS', None) == 'True' # read_the_docs_build = os.environ.get('READTHEDOCS', None) == 'True'
if read_the_docs_build: # if read_the_docs_build:
subprocess.call('doxygen doxy_lwmem.doxy', shell=True) subprocess.call('doxygen doxy_lwmem.doxy', shell=True)
# -- Project information ----------------------------------------------------- # -- Project information -----------------------------------------------------
project = 'LwMEM' project = 'LwMEM'
@ -79,13 +79,17 @@ html_theme_options = {
'includehidden': True, 'includehidden': True,
'titles_only': False 'titles_only': False
} }
html_logo = 'static/images/logo.svg' html_logo = 'static/images/logo_tm.png'
github_url = 'https://github.com/MaJerle/lwmem' github_url = 'https://github.com/MaJerle/lwmem'
# Add any paths that contain custom static files (such as style sheets) here, # Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files, # relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css". # so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['static'] html_static_path = ['static']
html_css_files = [
'css/common.css',
'css/custom.css',
]
master_doc = 'index' master_doc = 'index'

12
docs/examples/_lwmem_config.h
View File

@ -1,12 +0,0 @@
#ifndef LWMEM_HDR_CONFIG_H
#define LWMEM_HDR_CONFIG_H
/* Rename this file to "lwmem_config.h" for your application */
/* Enable operating system support */
#define LWMEM_CFG_OS 1
/* After user configuration, call default config to merge config together */
#include "lwmem/lwmem_config_default.h"
#endif /* LWMEM_HDR_CONFIG_H */

14
docs/examples/_lwmem_config_template.h
View File

@ -1,14 +0,0 @@
#ifndef LWMEM_HDR_CONFIG_H
#define LWMEM_HDR_CONFIG_H
/* Rename this file to "lwmem_config.h" for your application */
/*
* Open "include/lwmem/lwmem_config_default.h" and
* copy & replace here settings you want to change values
*/
/* After user configuration, call default config to merge config together */
#include "lwmem/lwmem_config_default.h"
#endif /* LWMEM_HDR_CONFIG_H */

31
docs/examples/_lwmem_sys.c
View File

@ -1,31 +0,0 @@
/* Config header in lwmem_config.h file */
#define LWMEM_CFG_OS_MUTEX_HANDLE osMutexId
/* System file */
uint8_t
lwmem_sys_mutex_create(LWMEM_CFG_OS_MUTEX_HANDLE* m) {
osMutexDef(mut);
*m = osMutexCreate(osMutex(mut));
return 1;
}
uint8_t
lwmem_sys_mutex_isvalid(LWMEM_CFG_OS_MUTEX_HANDLE* m) {
return *m != NULL;
}
uint8_t
lwmem_sys_mutex_wait(LWMEM_CFG_OS_MUTEX_HANDLE* m) {
if (osMutexWait(*m, osWaitForever) != osOK) {
return 0;
}
return 1;
}
uint8_t
lwmem_sys_mutex_release(LWMEM_CFG_OS_MUTEX_HANDLE* m) {
if (osMutexRelease(*m) != osOK) {
return 0;
}
return 1;
}

27
docs/examples/example_minimal.c Normal file
View File

@ -0,0 +1,27 @@
#include "lwmem/lwmem.h"
void* ptr;
/* Create regions, address and length of regions */
static
lwmem_region_t regions[] = {
/* Set start address and size of each region */
{ (void *)0x10000000, 0x00001000 },
{ (void *)0xA0000000, 0x00008000 },
{ (void *)0xC0000000, 0x00008000 },
};
/* Later in the initialization process */
/* Assign regions for manager */
lwmem_assignmem(regions, sizeof(regions) / sizeof(regions[0]));
ptr = lwmem_malloc(8); /* Allocate 8 bytes of memory */
if (ptr != NULL) {
/* Allocation successful */
}
/* Later... */ /* Free allocated memory */
lwmem_free(ptr);
ptr = NULL;
/* .. or */
lwmem_free_s(&ptr);

46
docs/get-started/index.rst
View File

@ -1,3 +1,5 @@
.. _get_started:
Get started Get started
=========== ===========
@ -43,39 +45,25 @@ At this point it is assumed that you have successfully download library, either
* Copy ``lwmem`` folder to your project * Copy ``lwmem`` folder to your project
* Add ``lwmem/src/include`` folder to `include path` of your toolchain * Add ``lwmem/src/include`` folder to `include path` of your toolchain
* Add source files from ``lwmem/src/`` folder to toolchain build * Add source files from ``lwmem/src/`` folder to toolchain build
* Copy ``lwmem/src/include/lwmem/lwmem_config_template.h`` to project folder and rename it to ``lwmem_config.h``
* Build the project * Build the project
Configuration file
^^^^^^^^^^^^^^^^^^
Library comes with template config file, which can be modified according to needs.
This file shall be named ``lwmem_config.h`` and its default template looks like the one below:
.. tip::
Check :ref:`api_lwmem_config` section for possible configuration settings
.. literalinclude:: ../../lwmem/src/include/lwmem/lwmem_config_template.h
:language: c
Minimal example code Minimal example code
^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
Run below example to test and verify library Run below example to test and verify library
.. code-block:: c .. literalinclude:: ../examples/example_minimal.c
:language: c
#include "lwmem/lwmem.h"
void* ptr;
/* Create regions, address and length of regions */
static
lwmem_region_t regions[] = {
/* Set start address and size of each region */
{ (void *)0x10000000, 0x00001000 },
{ (void *)0xA0000000, 0x00008000 },
{ (void *)0xC0000000, 0x00008000 },
};
/* Later in the initialization process */
/* Assign regions for manager */
lwmem_assignmem(regions, sizeof(regions) / sizeof(regions[0]));
ptr = lwmem_malloc(8); /* Allocate 8 bytes of memory */
if (ptr != NULL) {
/* Allocation successful */
}
/* Later... */ /* Free allocated memory */
lwmem_free(ptr);
ptr = NULL;
/* .. or */
lwmem_free_s(&ptr);

3
docs/index.rst
View File

@ -3,9 +3,6 @@ LwMEM documentation!
LwMEM is lightweight dynamic memory manager optimized for embedded systems. LwMEM is lightweight dynamic memory manager optimized for embedded systems.
.. image:: static/images/logo.svg
:align: center
.. class::center .. class::center
:ref:`download_library` · `Github <https://github.com/MaJerle/lwmem>`_ :ref:`download_library` · `Github <https://github.com/MaJerle/lwmem>`_

403
docs/src/_application_note.h
View File

@ -1,403 +0,0 @@
/**
* \page page_appnote Application note
* \tableofcontents
*
* \section sect_getting_started Getting started
*
* Repository <a href="https://github.com/MaJerle/lwmem"><b>lwmem is hosted on Github</b></a>. It combines source code and example projects.
*
* \subsection sect_clone Clone repository
*
* \par First-time clone
*
* - Download and install `git` if not already
* - Open console and navigate to path in the system to clone repository to. Use command `cd your_path`
* - Run `git clone --recurse-submodules https://github.com/MaJerle/lwmem` command to clone repository including submodules
* - Navigate to `examples` directory and run favourite example
*
* \par Already cloned, update to latest version
*
* - Open console and navigate to path in the system where your resources repository is. Use command `cd your_path`
* - Run `git pull origin master --recurse-submodules` command to pull latest changes and to fetch latest changes from submodules
* - Run `git submodule foreach git pull origin master` to update & merge all submodules
*
* \section sect_config Library configuration
*
* To make library as efficient as possible, different configuration parameters are available
* to make sure all the requirements are met for different purposes as possible.
*
* A list of all configurations can be found in \ref LWMEM_CONFIG section.
*
* \subsection sect_conf_file Project configuration file
*
* Library comes with `2` configuration files:
*
* - Default configuration file `lwmem_config_default.h`
* - Project template configuration file `lwmem_config_template.h`
*
* When project is started, user has to rename template file to `lwmem_config.h`
* and if required, it should override default settings in this file.
*
* Default template file comes with something like this:
*
* \include _lwmem_config_template.h
*
* If bigger buffer is required, modification must be made like following:
*
* \include _lwmem_config.h
*
* \note Always modify default settings by overriding them in user's custom `lwmem_config.h` file
* which was previously renamed from `lwmem_config_template.h`
*
* \section sect_how_it_works How memory allocation works
*
* For the sake of example memory manager on images uses `3` regions:
*
* - Region `1` memory starts at `0x1000 0000` and is `0x0000 1000` bytes long
* - Region `2` memory starts at `0xA000 0000` and is `0x0000 8000` bytes long
* - Region `3` memory starts at `0xC000 0000` and is `0x0000 8000` bytes long
* - Total size of memory used by application for memory manager is `0x0001 1000` bytes or `69 kB`
*
* Furthermore, example assumes:
*
* - Size of any kind of pointer is `4-bytes`, `sizeof(any_pointer_type) = 4`
* - Size of `size_t` type is `4-bytes`, `sizeof(size_t) = 4`
*
* In C code, defining these regions for memory manager would look similar to example below.
*
* \include example_regions_definitions.c
*
* After the initialization process, memory is written with some values, available on picture below.
*
* \image html structure_default.svg Default memory structure after initialization
*
* Memory managers sets some default values, these are:
*
* - All regions are connected through single linked list. Each member of linked list represents free memory slot
* - `Start block` is variable in the library and points to first free memory on the list. By default, this is beginning of first region
* - Each region consists of `2 free slots`
* - One at the end of each region. It takes `8 bytes` of memory:
* - Size of slot is set to `0` and it means no available memory
* - Its `next` value points to next free slot in another region. Set to `NULL` if there is no free slot available anymore after or in case of last region on a list
* - One on beginning of region. It also takes `8 bytes` of memory:
* - Size of slot is set to `region_size - 8`, ignoring size of last slot.
* Effective size of memory application may allocate in region is always for `2` meta slots less than region size,
* which means `max_app_malloc_size = region_size - 2 * 8 bytes`
* - Its `next` value points to end slot in the same region
*
* When application calls one of allocation functions and if requested size of memory is less than maximal one,
* manager will allocate desired memory and will mark it as used. If requested size is bigger than available memory
* in first region, manager will continue to check size of next free slots in other regions.
*
* \image html structure_first_alloc.svg Memory structure after first allocation
*
* - Light red background slot indicates memory `in use`.
* - All blocks marked `in use` have:
* - `next` value is set to `NULL`
* - `size` value has MSB bit set to `1`, indicating block is allocated and the rest of bits represent size of block, including metadata size
* - If application asks for `8 bytes`, fields are written as `next = 0x0000 0000` and `size = 0x8000 000F`
* - `Start block` now points to free slot somewhere in the middle of region
*
* Follow description of bottom image for more information about allocation and deallocation.
*
* \image html structure_alloc_free_steps.svg Step-by-step memory structure after multiple allocations and deallocations
*
* Image shows only first region to simplify process. Same procedure applies to other regions aswell.
*
* - `Case A`: Second block allocated. Remaining memory is now smaller and `Start block` points to it
* - `Case B`: Third block allocated. Remaining memory is now smaller and `Start block` points to it
* - `Case C`: Forth block allocated. Remaining memory is now smaller and `Start block` points to it
* - `Case D`: Third block freed and added back to linked list of free slots.
* - `Case E`: Forth block freed. Manager detects blocks before and after current are free and merges all to one big contiguous block
* - `Case F`: First block freed. `Start block` points to it as it has been added back to linked list
* - `Case G`: Second block freed. Manager detects blocks before and after current are free and merges all to one big contiguous block.
* - No any memory allocated anymore, regions are back to default state
*
* \section sect_realloc_how_it_works Optimized re-allocation algorithm
*
* why would you need re-allocation algorithm at first place?
*
* Sometimes application uses variable length of memory,
* specially when number of (for example) elements in not fully known in advance.
* An example, application anticipates `12` numbers but may (for some reason) in some cases receive more than this.
* If application needs to hold all received numbers, it may be necessary to:
*
* - Option `1`: Increase memory block size using reallocations on demand
* - Option `2`: Use very big (do we know how much?) array, allocated statically or dynamically,
* which would hold all numbers at any time possible
*
* Application wants to use as less memory as possible, so we will take option `1`, to increase memory only on demand.
*
* Let's first define our region(s). For the sake of example, application uses single region:
*
* \include example_realloc_region.c
*
* When executed, it prints (test machine)
*
* \include example_realloc_region_log.c
*
* Next, application allocates memory for `12 integers`.
*
* \include example_realloc_first_malloc.c
*
* When executed, it prints (test machine)
*
* \include example_realloc_first_malloc_log.c
*
* We can see, there is one block with `64` bytes available.
* It means that manager spent `120 - 64 = 56` bytes to allocate memory for `12` integers.
*
* \note Every allocated block holds meta data.
* On test machine, `sizeof(int) = 4` therefore `8` bytes are used for metadata as `56 - 12 * sizeof(int) = 8`.
* Size of meta data header depends on CPU architecture and may be different on final architecture
*
* What happens, if (for some reason) application needs more memory than current block is?
* Let's say application needs to increase memory to be able to hold `13` integers.
*
* The easiest way would be:
*
* 1. Allocate new memory block with new size and check if allocation was successful
* 2. Manually copy content from old block to new block
* 3. Free old memory block
* 4. Use new block for all future operations
*
* \include example_realloc_custom_realloc.c
*
* When executed, it prints (test machine)
*
* \include example_realloc_custom_realloc_log.c
*
* Looking at the debug output:
*
* - Memory was successfully allocated for `12` integers, it took `56 bytes`
* - Memory was successfully allocated for another `13` integers , it took `64 bytes`
* - There is no more free memory available
* - First `12` integers array was successfully freed, manager has `56` bytes of free memory
* - Second `13` integers block was successfully freed, manager has all `120` bytes available for new allocations
*
* What would happen, if application needs to increase block size for `3` more bytes? From `12` to `15`.
* When same code is executed, but with `15` in second case:
*
* \include example_realloc_custom_realloc_log_2.c
*
* It was not possible to allocate new memory for `15` integers.
* We just need to increase for `3` more integers, which is `12 bytes`,
* there are `64` bytes of memory available, but we were not able to use them!
*
* The same effect would happen if application wants to decrease integer count from `15` to `12` numbers.
* There is not enough memory to allocate new block for `12` integers.
*
* \note Effectively it means that maximal block size we can allocate is less than `50%` of full memory,
* if we intend to increase it in the future using above method.
*
* \par Solution: Try to manipulate existing block as much as possible
*
* To avoid having multiple temporary allocations, we could only manipulate existing block to modify its size.
* Let's start with simplest example.
*
* \subsection sect_realloc_shrinking Shrinking existing block
*
* When applications tries to decrease memory size, manager could only modify block size parameters.
* This will effectively shrink block size and allow other parts of application to
* allocate memory using memory manager.
*
* \include example_realloc_shrink.c
*
* When executed, it prints (test machine)
*
* \include example_realloc_shrink_log.c
*
* \image html structure_realloc_shrink.svg New size is smaller than existing one
*
* Looking at the debug output and image above:
*
* - Memory was successfully allocated for `15` integers, it took `68` bytes; part `A` on image
* - Memory was successfully re-allocated to `12` integers, now it takes `56` bytes, part `B` on image
* - In both cases on image, final returned memory points to the same address
* - Manager does not need to copy data from existing memory to new address as it is the same memory used in both cases
* - Empty block start address has been modified and its size has been increased, part `B` on image
* - Reallocated block was successfully freed, manager has all `120` bytes for new allocations
*
* This was very basic and primitive example. However, it is not always possible to increase free block size.
* Consider new example and dedicated image below:
*
* \include example_realloc_shrink_fragmented.c
*
* \image html structure_realloc_shrink_fragmented.svg Decreasing size on fragmented blocks
*
* When executed, it prints (test machine)
*
* \include example_realloc_shrink_fragmented_log.c
*
* Looking at the debug output and image above:
*
* - Size of all `4` blocks is `24` bytes; `16` for user data, `8` for metadata
* - Reallocating block first time from `16` to `12` user data bytes did not affect internal memory structure
* - It is not possible to create new empty block as it would be too small, only `4` bytes available, minimum is `8` bytes
* - It is not possible to enlarge next empty block as current and next empty do not create contiguous block
* - Block is internally left unchanged
* - Reallocating block second time to `8` bytes was successful
* - Difference between old and new size is `8` bytes which is enough for new empty block.
* Its size is `8` bytes, effectively `0` for user data
*
* \par Summary for reallocation to smaller size
*
* When reallocating already allocated memory block, one of `3` cases will happen:
*
* - Case `1`: When current block and next free block could create contigouos block of memory,
* current block is decreased (size parameter) and next free is enlarged by the size difference
* - Case `2`: When difference between current size and new size is more or equal to minimal size for new empty block,
* new empty block is created with size `current_size - new_size` and added to list of free blocks
* - Case `3`: When difference between current size and new size is less than minimal size for new empty block,
* block is left unchanged
*
* \subsection sect_realloc_enlarging Enlarging existing block
*
* Things get more tricky when we need to enlarge existing block.
* Since we are increasing block size, we cannot just anticipate memory after already allocated block is free.
* We need to take some actions and optimize reallocation procedure.
*
* Possible memory structure cases are explained below.
*
* \note Size numbers represent block size, including meta size.
* Number `32` represents `24` user bytes and `8` metadata bytes.
*
* \par Free block before allocated [block] create one big contiguous block
*
* \image html structure_realloc_enlarge_1.svg Case 1: Free block before allocated [block] create one big contiguous block
*
* This is the simplest way to show how reallocation works. Let's start by code and output:
*
* \include example_realloc_enlarge_1.c
*
* When executed, it prints (test machine)
*
* \include example_realloc_enlarge_1_log.c
*
* Looking at the debug output and image above:
*
* - Allocation for first block of memory (`24` user bytes) uses `32` bytes of data
* - Reallocation is successful, block has been extended to `40` bytes and next free shrinked down to `80` bytes
*
* \par Free block after allocated [block] create one big contiguous block
*
* \image html structure_realloc_enlarge_2.svg Case 2: Free block after allocated [block] create one big contiguous block
*
* Second case is a block with free block as previous.
* As you can see on image, it is possible to reallocate existing block by moving it to the left.
* Example code which would effectively produce this:
*
* \include example_realloc_enlarge_2.c
*
* When executed, it prints (test machine)
*
* \include example_realloc_enlarge_2_log.c
*
* Looking at the debug output and image above:
*
* - First we allocate big block (`88` bytes), followed by smaller block (`32` bytes)
* - We then free big block to mark it as free. This is effectively state `2a`
* - During reallocation, manager did not find suitable block after, but did find suitable block before current one:
* - Empty block and allocated block are temporary merged to one big block (`120` bytes)
* - Content of allocated block is shifted up to beginning of big block
* - Big block is later splitted to required size, the rest is marked as free
* - This is effectively state `2b`
*
* \par Free block before and after allocated [block] create one big contiguous block
*
* \image html structure_realloc_enlarge_3.svg Case 2: Free block before and after allocated [block] create one big contiguous block
*
* In this example we always have `2` allocated blocks and we want to reallocate `green` block.
* `Red` block is there acting as an obstacle to show different application use cases.
*
* Image shows `4` optional cases. For every case, case labeled with `3` is initial state.
*
* \note Case labelled with `3` is initial state for any of `3a - 3d` cases
*
* Before moving to the description of the cases, let's make initial state with C code example
*
* \include example_realloc_enlarge_3.c
*
* When executed, it prints (test machine)
*
* \include example_realloc_enlarge_3_log.c
*
* As seen on the image (and confirmed in log), there are `3` free slots of `16, 12 and 56` bytes respectively.
*
* Cases:
* - Case `3a`: Application tries to reallocate green block from `12` to `16` bytes
* - Reallocation is successful, there is a free block just after and green block is successfully enlarged
* - Block after is shrinked from `12` to `8` bytes
* - Code example (follows initial state code example)
* \include example_realloc_enlarge_3a.c
* - When executed, it prints (test machine)
* \include example_realloc_enlarge_3a_log.c
* - Case `3b`: Application tries to reallocate green block from `12` to `28` bytes
* - Block after green is not big enough to merge them to one block (`12 + 12 < 28`)
* - Block before green is big enough (`16 + 12 >= 28`)
* - Green block is merged with previous free block and content is shifted to the beginning of new block
* - Code example (follows initial state code example)
* \include example_realloc_enlarge_3b.c
* - When executed, it prints (test machine)
* \include example_realloc_enlarge_3b_log.c
* - Case `3c`: Application tries to reallocate green block from `12` to `32` bytes
* - Block after green is not big enough to merge them to one block (`12 + 12 < 32`)
* - Block before green is also not big enough (`12 + 16 < 32`)
* - All three blocks together are big enough (`16 + 12 + 12 >= 32`)
* - All blocks are effectively merged together and there is a new temporary block with its size set to `40` bytes
* - Content of green block is shifted to the beginning of new block
* - New block is limited to `32` bytes, keeping `8` bytes marked as free at the end
* - Code example (follows initial state code example)
* \include example_realloc_enlarge_3c.c
* - When executed, it prints (test machine)
* \include example_realloc_enlarge_3c_log.c
* - Case `3d`: Application tries to reallocate green block from `12` to `44` bytes
* - None of the methods (`3a - 3c`) are available as blocks are too small
* - Completely new block is created and content is copied to it
* - Existing block is marked as free. Since all `3` free blocks create big contiguous block,
* we can merge them to one block with its size set to `40`
* - Code example (follows initial state code example)
* \include example_realloc_enlarge_3d.c
* - When executed, it prints (test machine)
* \include example_realloc_enlarge_3d_log.c
*
* \par Full code example with new debug output
*
* Advanced debugging features has been added for development purposes.
* It is now possible to simulate different cases within single executable,
* by storing states to different memories.
*
* Example has been implemented which runs on WIN32 and
* relies on dynamic allocation using `malloc` standard C function,
* to prepare blocks of memory later used by lwmem
*
* How it works:
* - Code prepares state `3` and saves memory to temporary memory for future restore
* - Code restores latest saved state (case `3`) and executes case `3a`
* - Code restores latest saved state (case `3`) and executes case `3b`
* - Code restores latest saved state (case `3`) and executes case `3c`
* - Code restores latest saved state (case `3`) and executes case `3d`
*
* Full code example
* \include example_realloc_enlarge_full.c
*
* When executed, it prints (test machine)
* \include example_realloc_enlarge_full_log.c
*
* \section sect_thread_safety Concurrent access & thread safety
*
* Many embedded applications run operating system and this is where thread safety is important.
* LwMEM by default uses single resource for all allocations, meaning that during memory operation (allocation, freeing)
* different threads MUST NEVER interrupt current running thread, or memory structure (and system) may collapse.
*
* \note Same concern applies when allocating/freeing memory from main thread and interrupt context,
* when not using operating system. It is advised NOT to do any LwMEM operation in interrupt context
* or memory structure (and system) may collapse.
*
* When using LwMEM in operating system environment, it is possible to use it with system protection functions.
* One more layer of functions needs to be implemented.
*
* Function description which needs to be implemented and added to project are available in \ref LWMEM_SYS section.
* Example codes are available in official git repository under `system` folder.
*
*/

10
docs/src/_lwmem_config.h
View File

@ -1,10 +0,0 @@
/**
* \addtogroup LWMEM_CONFIG
* \{
*
* List of all possible library configuration settings.
*
* Please refer to \ref sect_config for more information how to setup custom config.
*
* \}
*/

13
docs/src/_lwmem_sys.h
View File

@ -1,13 +0,0 @@
/**
* \addtogroup LWMEM_SYS
* \{
*
* System functions, required when operating system mode is enabled.
*
* \par Example for CMSIS OS
*
* \include _lwmem_sys.c
*
* \}
*
*/

56
docs/src/_mainpage.h
View File

@ -1,56 +0,0 @@
/**
* \mainpage
* \tableofcontents
*
* LwMEM is a dynamic memory manager which implements standard C allocation functions (`malloc`, `realloc`, `calloc`, `free`) and is platform independant.
*
* \section sect_features Features
*
* - Written in ANSI C99, compatible with `size_t` for size data types
* - Implements standard C library functions for memory allocation, `malloc`, `calloc`, `realloc` and `free`
* - Uses `first-fit` algorithm to search free block
* - Supports different memory regions to allow use of fragmented memories
* - Suitable for embedded applications with fragmented memories
* - Suitable for automotive applications
* - Supports advanced free/realloc algorithms to optimize memory usage
* - Operating system ready, thread-safe API
* - User friendly MIT license
*
* \section sect_resources Download & Resources
*
* - <a class="download_url" href="https://github.com/MaJerle/lwmem/releases">Download library at Github releases</a>
* - <a href="https://github.com/MaJerle/lwmem">Resources and examples repository</a>
* - Read \ref page_appnote before you start development
* - <a href="https://github.com/MaJerle/lwmem">Official development repository on Github</a>
*
* \section sect_contribute How to contribute
*
* - Official development repository is hosted on Github
* - <a href="https://github.com/MaJerle/c_code_style">Respect C style and coding rules</a>
*
* \section sect_license License
*
* \verbatim
* Copyright (c) 2019 Tilen Majerle
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without restriction,
* including without limitation the rights to use, copy, modify, merge,
* publish, distribute, sublicense, and/or sell copies of the Software,
* and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE
* AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE. \endverbatim
*
*/

3
docs/static/css/common.css vendored Normal file
View File

@ -0,0 +1,3 @@
.center {
text-align: center;
}

0
docs/static/css/custom.css vendored Normal file
View File

BIN
docs/static/images/logo_tm.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 3.3 KiB

BIN
docs/static/images/logo_tm_full.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

115
docs/user-manual/how-it-works.rst
View File

@ -3,59 +3,98 @@
How it works How it works
============ ============
This section shows different buffer corner cases and provides basic understanding how data are managed internally. This section shows different buffer corner cases and provides basic understanding how memory allocation works within firmware.
.. figure:: ../static/images/buff_cases.svg As it is already known, library supports multiple memory regions (or addresses) to allow multiple memory locations within embedded systems:
:align: center
:alt: Different buffer corner cases
Different buffer corner cases
Let's start with reference of abbreviations in picture: * Internal RAM memory
* External RAM memory
* Optional fragmented internal memory
* ``R`` represents `Read` pointer. Read on read/write operations. Modified on read operation only For the sake of this understanding, application is using ``3`` regions
* ``W`` represents `Write` pointer. Read on read/write operations. Modified on write operation only
* ``S`` represents `Size` of buffer. Used on all operations, never modified (atomic value)
* Valid number of ``W`` and ``R`` pointers are between ``0`` and ``S - 1`` * Region ``1`` memory starts at ``0x1000 0000`` and is ``0x0000 1000`` bytes long
* Region ``2`` memory starts at ``0xA000 0000`` and is ``0x0000 8000`` bytes long
* Region ``3`` memory starts at ``0xC000 0000`` and is ``0x0000 8000`` bytes long
* Buffer size is ``S = 8``, thus valid number range for ``W`` and ``R`` pointers is ``0 - 7``.
* ``R`` and ``W`` numbers overflow at ``S``, thus valid range is always ``0, 1, 2, 3, ..., S - 2, S - 1, 0, 1, 2, 3, ..., S - 2, S - 1, 0, ...``
* Example ``S = 4``: ``0, 1, 2, 3, 0, 1, 2, 3, 0, 1, ...``
* Maximal number of bytes buffer can hold is always ``S - 1``, thus example buffer can hold up to ``7`` bytes
* ``R`` and ``W`` pointers always point to the next read/write operation
* When ``W == R``, buffer is considered empty.
* When ``W == R - 1``, buffer is considered full.
* ``W == R - 1`` is valid only if ``W`` and ``R`` overflow at buffer size ``S``.
* Always add ``S`` to calculated number and then use modulus ``S`` to get final value
.. note:: .. note::
Total size of memory used by application for memory manager is ``0x0001 1000`` bytes or ``69 kB``.
This is a sum of all ``3`` regions.
Example 1, add ``2`` numbers: ``2 + 3 = (3 + 2 + S) % S = (3 + 2 + 4) % 4 = (5 + 4) % 4 = 1`` Example also assumes that:
Example 2, subtract ``2`` numbers: ``2 - 3 = (2 - 3 + S) % S = (2 - 3 + 4) % 4 = (-1 + 4) % 4 = 3`` * Size of any kind of pointer is ``4-bytes``, ``sizeof(any_pointer_type) = 4``
* Size of ``size_t`` type is ``4-bytes``, ``sizeof(size_t) = 4``
First step is to define custom regions and assign them to memory manager.
.. figure:: ../static/images/buff_cases.svg .. literalinclude:: ../examples/example_regions_definitions.c
:language: c
.. note::
Order of regions must be lower address first. Regions must not overlap with their sizes.
When calling ``lwmem_assignmem``, manager prepares memory blocks and assigns default values.
.. figure:: ../static/images/structure_default.svg
:align: center :align: center
:alt: Different buffer corner cases :alt: Default memory structure after initialization
Different buffer corner cases
Different image cases: Default memory structure after initialization
* Case **A**: Buffer is empty as ``W == R = 0 == 0`` Memory managers sets some default values, these are:
* Case **B**: Buffer holds ``W - R = 4 - 0 = 4`` bytes as ``W > R``
* Case **C**: Buffer is full as ``W == R - 1`` or ``7 == 0 - 1`` or ``7 = (0 - 1 + S) % S = (0 - 1 + 8) % 8 = (-1 + 8) % 8 = 7``
* ``R`` and ``W`` can hold ``S`` different values, from ``0`` to ``S - 1``, that is modulus of ``S`` * All regions are connected through single linked list. Each member of linked list represents free memory slot
* Buffer holds ``W - R = 7 - 0 = 7`` bytes as ``W > R`` * Variable ``Start block`` is by default included in library and points to first free memory on the list
* Case **D**: Buffer holds ``S - (R - W) = 8 - (5 - 3) = 6`` bytes as ``R > W`` * Each region has ``2 free slot`` indicators
* Case **E**: Buffer is full as ``W == R - 1`` (``4 = 5 - 1``) and holds ``S - (R - W) = 8 - (5 - 4) ) = 7`` bytes
* One at the end of each region. It takes ``8 bytes`` of memory:
* Size of slot is set to ``0`` which `means no available memory`
* Its next value points to next free slot in another region. Set to ``NULL`` if there is no free slot available anymore after and is *last region* indicator
* One at the beginning of region. It also takes ``8 bytes`` of memory:
* Size of slot is set to ``region_size - 8``, ignoring size of last slot. Effective size of memory, application may allocate in region, is always for ``2`` meta slots less than region size, which means ``max_app_malloc_size = region_size - 2 - 8 bytes``
* Its next value points to end slot in the same region
When application tries to allocate piece of memory, library will check linked list of empty blocks until it finds first with sufficient size. If there is a block bigger than requested size, it will be marked as allocated and removed from linked list.
.. note::
Further optimizations are implemented, such as possibility to split block when requested size is smaller than empty block size is.
.. figure:: ../static/images/structure_first_alloc.svg
:align: center
:alt: Memory structure after first allocation
Memory structure after first allocation
* Light red background slot indicates memory in use.
* All blocks marked in use have
* ``next`` value is set to ``NULL``
* ``size`` value has MSB bit set to ``1``, indicating block *is allocated* and the rest of bits represent size of block, including metadata size
* If application asks for 8 bytes, fields are written as ``next = 0x0000 0000`` and ``size = 0x8000 000F``
* ``Start block`` now points to free slot somewhere in the middle of region
.. figure:: ../static/images/structure_alloc_free_steps.svg
:align: center
:alt: Step-by-step memory structure after multiple allocations and deallocations
Step-by-step memory structure after multiple allocations and deallocations
Image shows only first region to simplify process. Same procedure applies to other regions too.
* ``Case A``: Second block allocated. Remaining memory is now smaller and ``Start block points`` to it
* ``Case B``: Third block allocated. Remaining memory is now smaller and ``Start block points`` to it
* ``Case C``: Forth block allocated. Remaining memory is now smaller and ``Start block points`` to it
* ``Case D``: Third block freed and added back to linked list of free slots.
* ``Case E``: Forth block freed. Manager detects blocks before and after current are free and merges all to one big contiguous block
* ``Case F``: First block freed. ``Start block`` points to it as it has been added back to linked list
* ``Case G``: Second block freed. Manager detects blocks before and after current are free and merges all to one big contiguous block.
* No any memory allocated anymore, regions are back to default state
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2

292
docs/user-manual/hw-dma-usage.rst
View File

@ -1,292 +0,0 @@
DMA on embedded systems
=======================
One of the key features of ringbuffer library is that it can be seamlessly integrated with DMA controllers on embedded systems.
.. note::
DMA stands for *Direct Memory Access* controller and is usually used to off-load CPU.
More about DMA is available on `Wikipedia <https://en.wikipedia.org/wiki/Direct_memory_access>`_.
DMA controllers normally use source and destination memory addresses to transfer data in-between.
This features, together with ringbuffer, allows seamless integration and zero-copy of application data at interrupts after DMA transfer has been completed.
Some manual work is necessary to be handled, but this is very minor in comparison of writing byte-by-byte to buffer at (for example) each received character.
Below are ``2`` common use cases:
* DMA transfers data from ringbuffer memory to (usually) some hardware IP
* DMA transfers data from hardware IP to memory
Zero-copy data from memory
^^^^^^^^^^^^^^^^^^^^^^^^^^
This describes how to pass ringbuffer output memory address as pointer to DMA (or any other processing function).
After all the data are successfully processed, application can skip processed data and free ringbuff for new data being written to it.
.. figure:: ../static/images/buff_lin_read_skip.svg
:align: center
:alt: Data transfer from memory to hardware IP
* Case **A**: Initial state, buffer is full and holds ``7`` bytes
* Case **B**: State after skipping ``R`` pointer for ``3`` bytes. Buffer now holds ``4`` remaining bytes
* Case **C**: Buffer is empty, no more memory available for read operation
Code example:
.. code-block:: c
/* Declare buffer variables */
ringbuff_t buff;
uint8_t buff_data[8];
size_t len;
uint8_t* data;
/* Initialize buffer, use buff_data as data array */
ringbuff_init(&buff, buff_data, sizeof(buff_data));
/* Use write, read operations, process data */
/* ... */
/* IMAGE PART A */
/* At this stage, we have buffer as on image above */
/* R = 5, W = 4, buffer is considered full */
/* Get length of linear memory at read pointer */
/* Function returns 3 as we can read 3 bytes from buffer in sequence */
/* When function returns 0, there is no memory available in the buffer for read anymore */
if ((len = ringbuff_get_linear_block_read_length(&buff)) > 0) {
/* Get pointer to first element in linear block at read address */
/* Function returns &buff_data[5] */
data = ringbuff_get_linear_block_read_address(&buff);
/* Send data via DMA and wait to finish (for sake of example) */
send_data(data, len);
/* Now skip sent bytes from buffer = move read pointer */
ringbuff_skip(&buff, len);
/* Now R points to top of buffer, R = 0 */
/* At this point, we are at image part B */
}
/* IMAGE PART B */
/* Get length of linear memory at read pointer */
/* Function returns 4 as we can read 4 bytes from buffer in sequence */
/* When function returns 0, there is no memory available in the buffer for read anymore */
if ((len = ringbuff_get_linear_block_read_length(&buff)) > 0) {
/* Get pointer to first element in linear block at read address */
/* Function returns &buff_data[0] */
data = ringbuff_get_linear_block_read_address(&buff);
/* Send data via DMA and wait to finish (for sake of example) */
send_data(data, len);
/* Now skip sent bytes from buffer = move read pointer */
/* Read pointer is moved for len bytes */
ringbuff_skip(&buff, len);
/* Now R points to 4, that is R == W and buffer is now empty */
/* At this point, we are at image part C */
}
/* IMAGE PART C */
/* Buffer is considered empty as R == W */
Part **A** on image clearly shows that not all data bytes are linked in single contiguous block of memory.
To send all bytes from ringbuff, it might be necessary to repeat procedure multiple times
.. code-block:: c
/* Initialization part skipped */
/* Get length of linear memory at read pointer */
/* When function returns 0, there is no memory available in the buffer for read anymore */
while ((len = ringbuff_get_linear_block_read_length(&buff)) > 0) {
/* Get pointer to first element in linear block at read address */
data = ringbuff_get_linear_block_read_address(&buff);
/* If max length needs to be considered */
/* simply decrease it and use smaller len on skip function */
if (len > max_len) {
len = max_len;
}
/* Send data via DMA and wait to finish (for sake of example) */
send_data(data, len);
/* Now skip sent bytes from buffer = move read pointer */
ringbuff_skip(&buff, len);
}
Zero-copy data to memory
^^^^^^^^^^^^^^^^^^^^^^^^
Similar to reading data from buffer with zero-copy overhead, it is possible to write to ringbuff with zero-copy overhead too.
Only difference is that application now needs pointer to write memory address and length of maximal number of bytes to directly copy in buffer.
After processing is successful, buffer advance operation is necessary to manually increase write pointer and to increase number of bytes in buffer.
.. figure:: ../static/images/buff_lin_write_advance.svg
:align: center
:alt: Data transfer from memory to hardware IP
* Case **A**: Initial state, buffer is empty as ``R == W``
* Based on ``W`` pointer position, application could write ``4`` bytes to contiguous block of memory
* Case **B**: State after advancing `W` pointer for `4` bytes. Buffer now holds `4` bytes and has ``3`` remaining available
* Case **C**: Buffer is full, no more free memory available for write operation
Code example:
.. code-block:: c
/* Declare buffer variables */
ringbuff_t buff;
uint8_t buff_data[8];
size_t len;
uint8_t* data;
/* Initialize buffer, use buff_data as data array */
ringbuff_init(&buff, buff_data, sizeof(buff_data));
/* Use write, read operations, process data */
/* ... */
/* IMAGE PART A */
/* At this stage, we have buffer as on image above */
/* R = 4, W = 4, buffer is considered empty */
/* Get length of linear memory at write pointer */
/* Function returns 4 as we can write 4 bytes to buffer in sequence */
/* When function returns 0, there is no memory available in the buffer for write anymore */
if ((len = ringbuff_get_linear_block_write_length(&buff)) > 0) {
/* Get pointer to first element in linear block at write address */
/* Function returns &buff_data[4] */
data = ringbuff_get_linear_block_write_address(&buff);
/* Receive data via DMA and wait to finish (for sake of example) */
/* Any other hardware may directly write to data array */
/* Data array has len bytes length */
/* Or use memcpy(data, my_array, len); */
receive_data(data, len);
/* Now advance buffer for written bytes to buffer = move write pointer */
/* Write pointer is moved for len bytes */
ringbuff_advance(&buff, len);
/* Now W points to top of buffer, W = 0 */
/* At this point, we are at image part B */
}
/* IMAGE PART B */
/* Get length of linear memory at write pointer */
/* Function returns 3 as we can write 3 bytes to buffer in sequence */
/* When function returns 0, there is no memory available in the buffer for write anymore */
if ((len = ringbuff_get_linear_block_read_length(&buff)) > 0) {
/* Get pointer to first element in linear block at write address */
/* Function returns &buff_data[0] */
data = ringbuff_get_linear_block_read_address(&buff);
/* Receive data via DMA and wait to finish (for sake of example) */
/* Any other hardware may directly write to data array */
/* Data array has len bytes length */
/* Or use memcpy(data, my_array, len); */
receive_data(data, len);
/* Now advance buffer for written bytes to buffer = move write pointer */
/* Write pointer is moved for len bytes */
ringbuff_advance(&buff, len);
/* Now W points to 3, R points to 4, that is R == W + 1 and buffer is now full */
/* At this point, we are at image part C */
}
/* IMAGE PART C */
/* Buffer is considered full as R == W + 1 */
Example for DMA transfer from memory
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is an example showing pseudo code for implementing data transfer using DMA with zero-copy overhead.
For read operation purposes, application gets direct access to ringbuffer read pointer and length of contigouos memory.
It is assumed that after DMA transfer completes, interrupt is generated (embedded system) and buffer is skipped in the interrupt.
.. note::
Buffer skip operation is used to mark sent data as processed and to free memory for new writes to buffer
.. code-block:: c
/* Buffer */
ringbuff_t buff;
uint8_t buff_data[8];
/* Working data length */
size_t len;
/* Send data function */
void send_data(void);
int
main(void) {
/* Initialize buffer */
ringbuff_init(&buff, buff_data, sizeof(buff_data));
/* Write 4 bytes of data */
ringbuff_write(&buff, "0123", 4);
/* Send data over DMA */
send_data();
while (1);
}
/* Send data over DMA */
void
send_data(void) {
/* If len > 0, DMA transfer is on-going */
if (len) {
return;
}
/* Get maximal length of buffer to read data as linear memory */
len = ringbuff_get_linear_block_read_length(&buff);
if (len) {
/* Get pointer to read memory */
uint8_t* data = ringbuff_get_linear_block_read_address(&buff);
/* Start DMA transfer */
start_dma_transfer(data, len);
}
/* Function does not wait for transfer to finish */
}
/* Interrupt handler */
/* Called on DMA transfer finish */
void
DMA_Interrupt_handler(void) {
/* Transfer finished */
if (len) {
/* Now skip the data (move read pointer) as they were successfully transferred over DMA */
ringbuff_skip(&buff, len);
/* Reset length = DMA is not active */
len = 0;
/* Try to send more */
send_data();
}
}
.. toctree::
:maxdepth: 2

3
docs/user-manual/index.rst
View File

@ -7,4 +7,5 @@ User manual
:maxdepth: 2 :maxdepth: 2
how-it-works how-it-works
hw-dma-usage realloc-algorithm
thread-safety

323
docs/user-manual/realloc-algorithm.rst Normal file
View File

@ -0,0 +1,323 @@
.. _realloc_algorithm:
Reallocation algorithm
======================
What makes this library different to others is its ability for memory re-allocation.
This section explains how it works and how it achieves best performances and less memory fragmentation vs others.
Sometimes application uses variable length of memory,
especially when number of (as an example) elements in not fully known in advance.
For the sake of this example, application anticipates ``12`` numbers (`integers`) but may (due to unknown reason in some cases) receive more than this.
If application needs to hold all received numbers, it may be necessary to:
* Option ``1``: Increase memory block size using reallocations
* Option ``2``: Use very big (do we know how big?) array, allocated statically or dynamically, which would hold all numbers at any time possible
.. note::
LwMEM has been optimized to handle well option ``1``.
Application needs to define at least single region:
.. literalinclude:: ../examples/example_realloc_region.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_region_log.c
.. note::
Please check :ref:`how_it_works` section for more information
After region has been defined, application tries to allocate memory for ``12 integers``.
.. literalinclude:: ../examples/example_realloc_first_malloc.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_first_malloc_log.c
At first, manager had ``120`` bytes of available memory while after allocation of ``48`` bytes, it only left ``64`` bytes.
Effectively ``120 - 64 = 56`` bytes have been used to allocate ``48`` bytes of memory.
.. note::
Every allocated block holds meta data. On test machine, ``sizeof(int) = 4`` therefore ``8`` bytes are used for metadata as ``56 - 12 * sizeof(int) = 8``. Size of meta data header depends on CPU architecture and may be different between architectures
Application got necessary memory for ``12`` integers. How to proceed when application needs to extend size for one more integer?
Easiest would be to:
#. Allocate new memory block with new size and check if allocation was successful
#. Manually copy content from old block to new block
#. Free old memory block
#. Use new block for all future operations
Here is the code:
.. literalinclude:: ../examples/example_realloc_custom_realloc.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_custom_realloc_log.c
Outcome of the debug messages:
#. Memory was successfully allocated for ``12`` integers, it took ``56`` bytes
#. Memory was successfully allocated for another ``13`` integers , it took ``64`` bytes
#. There is no more free memory available
#. First ``12`` integers array was successfully freed, manager has ``56`` bytes of free memory
#. Second ``13`` integers block was successfully freed, manager has all ``120`` bytes available for new allocations
This was therefore successful custom reallocation from ``12`` to ``13`` integers.
Next step is to verify what would happen when application wants to reallocate to ``15`` integers instead.
When same code is executed (but with ``15`` instead of ``12``), it prints:
.. literalinclude:: ../examples/example_realloc_custom_realloc_log_2.c
Oooops! It is not anymore possible to allocate new block for new ``15`` integers as there was no available block with at least ``15 * sizeof(int) + metadata_size`` bytes of free memory.
.. note::
With this reallocation approach, maximal size of application block is only ``50%`` of region size. This is not the most effective memory manager!
Fortunately there is a solution. Every time application wants to resize existing block, manager tries to manipulate existing block and shrink or expand it.
Shrink existing block
^^^^^^^^^^^^^^^^^^^^^
Easiest reallocation algorithm is when application wants to decrease size of previously allocated memory.
When this is the case, manager only needs to change the size of existing block to lower value.
.. literalinclude:: ../examples/example_realloc_shrink.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_shrink_log.c
Outcome of our reallocation:
* Memory was successfully allocated for ``15`` integers, it took ``68`` bytes; part ``A`` on image
* Memory was successfully re-allocated to ``12`` integers, now it takes ``56`` bytes, part ``B`` on image
* In both cases on image, final returned memory points to the same address
* Manager does not need to copy data from existing memory to new address as it is the same memory used in both cases
* Empty block start address has been modified and its size has been increased, part ``B`` on image
* Reallocated block was successfully freed, manager has all ``120`` bytes for new allocations
.. tip::
This was a success now, much better.
However, it is not always possible to increase block size of next free block on linked list.
Consider new example and dedicated image below.
.. figure:: ../static/images/structure_realloc_shrink_fragmented.svg
:align: center
:alt: Shrinking fragmented memory block
Shrinking fragmented memory block
.. literalinclude:: ../examples/example_realloc_shrink_fragmented.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_shrink_fragmented_log.c
Outcome of this example:
* Size of all ``4`` blocks is ``24`` bytes; ``16`` for user data, ``8`` for metadata
* Reallocating block first time from ``16`` to ``12`` user data bytes did not affect internal memory structure
* It is not possible to create new empty block as it would be too small, only ``4`` bytes available, minimum is ``8`` bytes for meta data
* It is not possible to enlarge next empty block due to *current* and *next empty* do not create contiguous block
* Block is internally left unchanged
* Reallocating block second time to ``8`` bytes was a success
* Difference between old and new size is ``8`` bytes which is enough for new empty block
* Its size is ``8`` bytes, effectively ``0`` for user data due to meta size
Shrink existing block - summary
*******************************
When reallocating already allocated memory block, one of `3` cases will happen:
* Case ``1``: When *current* block and *next free* block could create contigouos block of memory, *current* block is decreased (size parameter) and *next free* is enlarged by the size difference
* Case ``2``: When difference between *current* size and *new* size is more or equal to minimal size for new empty block, new empty block is created with size ``current_size - new_size`` and added to list of free blocks
* Case ``3``: When difference between *current* size and *new* size is less than minimal size for new empty block, block is left unchanged
Enlarge existing block
^^^^^^^^^^^^^^^^^^^^^^
Now that you master procedure to shrink (or decrease) size of existing allocated memory block, it is time to understand how to enlarge it.
Things here are more complicated, however, they are still easy to understand.
Manager covers ``3`` potential cases:
* Case ``1``: Increase size of currently allocated block
* Case ``2``: Merge previous empty block with existing one and shift data up
* Case ``3``: Block before and after existing block together create contiguous block of memory
*Free block* after + *allocated block* create one big contiguous block
**********************************************************************
.. figure:: ../static/images/structure_realloc_enlarge_1.svg
:align: center
:alt: *Free block* after + *allocated block* create one big contiguous block
*Free block* after + *allocated block* create one big contiguous block
.. literalinclude:: ../examples/example_realloc_enlarge_1.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_1_log.c
* Allocation for first block of memory (``24`` user bytes) uses ``32`` bytes of data
* Reallocation is successful, block has been extended to ``40`` bytes and next free block has been shrinked down to ``80`` bytes
*Free block* before + *allocated block* create one big contiguous block
***********************************************************************
.. figure:: ../static/images/structure_realloc_enlarge_2.svg
:align: center
:alt: *Free block* before + *allocated block* create one big contiguous block
*Free block* before + *allocated block* create one big contiguous block
.. literalinclude:: ../examples/example_realloc_enlarge_2.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_2_log.c
* First application allocates big block (``88`` bytes), followed by smaller block (``32`` bytes)
* Application then frees big block to mark it as free. This is effectively state ``2a``
* During reallocation, manager did not find suitable block after *current* block, but it found suitable block before *current* block:
* Empty block and allocated block are temporary merged to one big block (``120`` bytes)
* Content of allocated block is shifted up to beginning of new big block
* Big block is then splitted to required size, the rest is marked as free
* This is effectively state ``2b``
*Free block* before + *free block* after + *allocated block* create one big contiguous block
********************************************************************************************
When application makes many allocations and frees of memory, there is a high risk of memory fragmentations.
Essentially small chunks of allocated memory prevent manager to allocate new, fresh, big block of memory.
When it comes to reallocating of existing block, it may happen that *first free block after* and *current block* create a contiguous block, but its combined size is not big enough. Same could happen with *last block before* + *current block*. However, it may be possible to combine *free block before* + *current block* + *free block after* current block together.
.. figure:: ../static/images/structure_realloc_enlarge_3.svg
:align: center
:alt: *Free block* before + *free block* after + *allocated block* create one big contiguous block
*Free block* before + *free block* after + *allocated block* create one big contiguous block
In this example manager has always ``2`` allocated blocks and application always wants to reallocate ``green`` block.
``Red`` block is acting as an obstacle to show different application use cases.
.. note::
Image shows ``4`` use cases. For each of them, case labeled with ``3`` is initial state.
Initial state ``3`` is generated using C code:
.. literalinclude:: ../examples/example_realloc_enlarge_3.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_3_log.c
.. tip::
Image shows (and log confirms) ``3`` free slots of ``16, 12 and 56`` bytes in size respectively.
* Case ``3a``: Application tries to reallocate green block from ``12`` to ``16`` bytes
* Reallocation is successful, there is a free block just after and green block is successfully enlarged
* Block after is shrinked from ``12`` to ``8`` bytes
* Code example (follows initial state code example)
.. literalinclude:: ../examples/example_realloc_enlarge_3a.c
:language: c
* When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_3a_log.c
* Case ``3b``: Application tries to reallocate green block from ``12`` to ``28`` bytes
* Block after green is not big enough to merge them to one block (``12 + 12 < 28``)
* Block before green is big enough (``16 + 12 >= 28``)
* Green block is merged with previous free block and content is shifted to the beginning of new block
.. literalinclude:: ../examples/example_realloc_enlarge_3b.c
:language: c
- When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_3b_log.c
* Case ``3c``: Application tries to reallocate green block from ``12`` to ``32`` bytes
* Block after green is not big enough to merge them to one block (``12 + 12 < 32``)
* Block before green is also not big enough (``12 + 16 < 32``)
* All three blocks together are big enough (``16 + 12 + 12 >= 32``)
* All blocks are effectively merged together and there is a new temporary block with its size set to ``40`` bytes
* Content of green block is shifted to the beginning of new block
* New block is limited to ``32`` bytes, keeping ``8`` bytes marked as free at the end
.. literalinclude:: ../examples/example_realloc_enlarge_3c.c
:language: c
* When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_3c_log.c
* Case ``3d``: Application tries to reallocate green block from ``12`` to ``44`` bytes
* None of the methods (``3a - 3c``) are available as blocks are too small
* Completely new block is created and content is copied to it
* Existing block is marked as free. All ``3`` free blocks create big contiguous block, they are merged to one block with its size set to ``40``
.. literalinclude:: ../examples/example_realloc_enlarge_3d.c
:language: c
* When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_3d_log.c
Full test code with assert
**************************
Advanced debugging features have been added for development purposes.
It is now possible to simulate different cases within single executable, by storing states to different memories.
Example has been implemented for WIN32 and relies on dynamic allocation using ``malloc`` standard C function for main block data preparation.
How it works:
* Code prepares state 3 and saves memory to temporary memory for future restore
* Code restores latest saved state (case ``3``) and executes case ``3a``
* Code restores latest saved state (case ``3``) and executes case ``3b``
* Code restores latest saved state (case ``3``) and executes case ``3c``
* Code restores latest saved state (case ``3``) and executes case ``3d``
Initial state ``3`` is generated using C code:
.. literalinclude:: ../examples/example_realloc_enlarge_full.c
:language: c
When executed on test machine, it prints:
.. literalinclude:: ../examples/example_realloc_enlarge_full_log.c
.. toctree::
:maxdepth: 2

30
docs/user-manual/thread-safety.rst Normal file
View File

@ -0,0 +1,30 @@
.. _thread_safety:
Thread safety
=============
With default configuration, LwMEM library is *not* thread safe.
This means whenever it is used with operating system, user must resolve it with care.
Library has locking mechanism support for thread safety, which needs to be enabled.
.. tip::
To enable thread-safety support, parameter `LWMEM_CFG_OS` must be set to `1`.
Please check :ref:`api_lwmem_config` for more information about other options.
After thread-safety features has been enabled, it is necessary to implement
``4`` low-level system functions.
.. tip::
System function template example is available in ``lwmem/src/system/`` folder.
Example code for ``CMSIS-OS V2``
.. note::
Check :ref:`api_lwmem_sys` section for function description
.. literalinclude:: ../../lwmem/src/system/lwmem_sys_cmsis_os.c
:language: c
.. toctree::
:maxdepth: 2