Custom allocators can improve performance, enhance memory efficiency, and address issues with frequent small memory allocations
Cause
Recently, I’ve been working on network packet development, which requires frequent allocation and release of small memory blocks. I initially considered using a memory pool but examined existing implementations and found this…
https://github.com/cacay/MemoryPool
Seeing the interface, I was quite puzzled by this memory pool’s implementation—the concept of MemoryPool的实现逻辑，是在申请固定大小的内存空间。看过boost的内存池接口，提供的是一个模板，用的时候进行实例化。正巧这个库已经有文章进行过介绍，提到了__INLINE_CODE_1
wiki
In C++ programming, an allocator is a crucial component of the C++ standard library. Containers (such as linked lists and sets) are data structures defined in the C++ library that share a common characteristic: their size can change at runtime. Dynamic memory allocation is essential to achieve this, and allocators handle memory allocation and deallocation requests for containers. In other words, an allocator encapsulates the low-level details of memory management for Standard Template Library (STL) containers. By default, the C++ standard library uses its own generic allocator, but programmers can also customize their own allocators as needed.
The allocator was originally invented by Alexander Stepanov as part of the C++ Standard Template Library (STL) with the goal of creating a way to make the library more flexible and independent of the underlying data model, allowing programmers to utilize custom pointer and reference types within the library. However, when incorporating the STL into the C++ standard, the C++ standards committee realized that complete abstraction of the data model would result in unacceptable performance losses. As a compromise, restrictions on allocators were tightened in the standard. Consequently, the current standard describes allocators with significantly less customization than Stepanov originally envisioned.
While customization of allocators is limited, custom allocators are often necessary to manage access to different memory spaces (such as shared and recycled memory) or to improve performance when using memory pools. Furthermore, in programs with frequent small allocations, introducing a specialized allocator can yield significant benefits in terms of memory footprint and runtime.
Please provide the Chinese text you want me to translate. I am ready when you are! Just paste the text after “使用需求.
One primary reason for defining custom allocators is to improve performance. Utilizing a dedicated custom allocator can enhance program performance, increase memory efficiency, or both [4][8]. The default allocator uses the new operator to allocate storage space [Reference 5], often implemented using C’s heap allocation functions (malloc()) [9]. While the default allocator generally performs well when allocating memory for containers requiring large, infrequent allocations (e.g., vectors, double-ended queues) [8], it can be inefficient when used with containers that require frequent small allocations (e.g., associative containers and doubly linked lists) [4][9]. Furthermore, malloc()-based default allocators suffer from issues such as poor locality of reference [4] and potential memory fragmentation [4][9].
In short, this section is like a “I Have a Dream” speech for the standard’s approach to allocators. Before that dream comes true, programmers concerned with portability will limit themselves to stateless custom allocators. Scott Meyers, Effective STL Given this situation, memory pool allocators are often used to address frequent small allocations [8]. Unlike the default “on-demand” allocation approach, with a memory pool allocator, the program pre-allocates a large block of memory (the “memory pool”), and the custom allocator simply returns a pointer to memory within the pool when an allocation is requested; no actual deallocation is performed during object destruction but is deferred until the end of the memory pool’s lifecycle [Note 1] [8].
The topic of “custom allocators” has been extensively discussed by C++ experts and authors, such as Scott Meyers’ “Effective STL” and Andrei Alexandrescu’s “Modern C++ Design.” Meyers observed that if all instances of an allocator for a given type T must be equal, then portable allocator instances must not contain state. While the C++ standard encourages library implementers to support allocators with state [reference 4], Meyers calls this related passage a “seemingly wonderful idea” but almost empty rhetoric, and considers the restriction on allocators “too strict” [4]. For example, STL’s list allows the splice method, where nodes of one list object A can be directly moved into another list object B, requiring that the memory allocated by A’s allocator can be released by B’s allocator, thus inferring that the allocator instances of A and B must be equal. Meyers concludes that allocators are best defined as types using static methods. For example, according to the C++ standard, an allocator must provide an other class template that implements the rebind method.
Additionally, in “The C++ Programming Language,” Bjarne Stroustrup argues that “strictly limiting the allocator to avoid object information differences” is not a significant issue (in essence), and notes that many allocators do not require state; performance can even be better without it. He proposes three use cases for custom allocators: memory pool allocators, shared-memory allocators, and garbage collection allocators, and demonstrates an implementation utilizing an internal memory pool for rapid allocation/deallocation of small amounts of memory. However, he also mentions that such optimization may already be achieved in the example allocator he provides [3].
Another use of custom allocators is debugging memory-related errors [10]. This can be achieved by writing an allocator that allocates extra memory during allocation to store debug information. Such an allocator not only ensures that memory is allocated/deallocated by the same type of allocator, but also helps protect the program from buffer overflows [11].