Choosing a programming language means accepting how that language manages the allocation and use of memory:
- Java, C# and Go use tracing garbage collection.
- Python and Swift use reference-counted objects.
- C requires the programmer to explicitly manage memory
- Rust encourages use of its static, single-owner mechanism
Unfortunately, there is no perfect memory management strategy. Some specialize in faster throughput or more predictable performance. Others are easier-to-use for programmmers and support a broader range of data structures. C's approach gives powerful control to the programmer, but it is inconvenient and offers no memory safety guarantees. So, when a language restricts programs to a single strategy, that may work out for some programs, but it could easily be a poor fit for others.
With Cone, you can do-it-your-way, using the strategy (or mix of strategies!) that best fit the requirements.
Like Cyclone, Cone enables the programmer to choose the memory management strategies a program uses. In effect, available memory is partitioned into regions, each with its own strategy for allocating and freeing memory as needed for data. Every newly-created object names the region that provides its memory. It is the job of each region to decide when it is safe to dispose of its no-longer-accessible objects.
In addition to the global and stack regions, Cone offers a broad range of region strategies for dynamically creating new objects:
- Arenas. These use a bump pointer to quickly allocate memory. An arena's objects are all freed when the arena's lifetime ends. Arenas offer the best performance and great data structure versatility, but they can waste memory.
- Pool. These use a free-list to quickly allocate and free memory. Pools can be good for performance, but they limit object size to a certain maximum.
- Single-owner. This frees an allocated object at the end of the last lexical block it is moved into. These are medium performers, benefitting by the lack of runtime bookkeeping costs, but they are limited to modeling hierarchical data structures.
- Tracing GC. This periodically scans all references from the root set, freeing any objects that are no longer reachable by the active program. This strategy is versatile and convenient across all data structures and minimizes memory leaks. However, its runtime mechanisms can negatively impact throughput and latency.
- Reference counting. This counts the number of references to an object, freeing it when the count gets to zero. This typically offers the worst performance and can leak memory in the presence of cycles. However, it supports more complex data structures than single-owner, has a simple low-impact implementation, and has better determinism and typically better latency than tracing.
Cone also supports borrowed references. A lifetime-constrained borrowed reference can be obtained from any region-allocated reference, or even within an object. They are particularly useful for easily making code polymorphic. Their use can also improves performance, as they require no runtime logic to manage.
Cone's versatile regional memory managers give the programmer the power to architect a program's use of memory for optimal throughput and latency, without sacrificing safety, convenience, or data structure flexibility. This powerful region choice can be made on an object-by-object basis.
The performance benefit of region-based memory is particularly important given how many CPU cycles are spent allocating, freeing, copying, and liveness tracking thousands or millions of small data objects in most programs today (not to mention accessing those objects indirectly in cache-unfriendly ways). High-speed strategies (arenas, pools, and single owner) can be used where they are a good fit to a program's data structure and memory use requirements. The slower ref-counting and tracing strategies can then be used for objects that require their flexibility and memory efficiency. Even then, extensive use of borrowed references can significantly minimize the throughput and latency overhead usually experienced with reference counting and tracing.
All of the region strategies are memory safe, even though they use different techniques for ensuring that an object's memory is only reclaimed when all references to it are gone.