How Memcached Works

This article discuss memcached internals

What is Memcached?  

From memcached.org:

Memcached is Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

How Memcached Works  

• Memcached assign memory that can be used to store data item via (-m) option. In ElastiCache this was configured using max_cache_memory parameter in parameter group. Value of max_cache_memory parameter cannot be changed. This is not directly requested to operating systems when memcached start but will be requested as needed.
• Memcached split the memory into small part called page. A page in Memcached have fixed size of 1 MB. ( source code reference)
• Memcached is having a concepts of slabs to manage memory. On start, memcached will define slab classes.
• Each slab class will have its own chunk size. Chunk size for each slab class is defined by the following elasticache parameters.
  • chunk_size (default : 48 (bytes))
  • chunk_size_growth_factor (default : 1.25)
  • slab_chunk_max (default : 524288 (bytes))
• When we store item in Memcached, it will find slab class with chunk size that fit with key-value data size and additional metadata. If the total data is not fit to a slab class, it will use the next slab class
• If no chunk left on a page for a slab class Memcached will allocate another page and put them on the same slab class.

Memory Waste in Memcached  

• There are two potential memory waste from memcached:
  • Once a chunk is used to store an item, the remaining free space cannot be used to store another item or part of another items.
  • Since Memcached is requesting memory per page (1MB), the memory is already allocated for specific slab class even if only 1 chunk is being used on the slab class.

Clustering in memcached  

• Clustering in Memcached is just a bunch of nodes
• No communication / replication between nodes
• There is no master node, slave / replica nodes
• Server rely on client (application that use Memcached) hashing mechanism to know which node in cluster have the key value it needs.

Limits  

• Max key size is 250 bytes ( source code reference)
• Max Item Size is 1 MB by default
  • 1M (from start)
  • 128 M (since version ?)
  • 1G (since version ?)

Memcached Slab Classes  

Below table is reference of Memcached slab classes with its chunk size using default settings (default parameter group)

Slab Class	Chunk Size (bytes)	Chunks Per Page	Usable space per chunk
1	96	10922	48
2	120	8738	72
3	152	6898	104
4	192	5461	144
5	240	4369	192
6	304	3449	256
7	384	2730	336
8	480	2184	432
9	600	1747	552
10	752	1394	704
11	944	1110	896
12	1184	885	1136
13	1480	708	1432
14	1856	564	1808
15	2320	451	2272
16	2904	361	2856
17	3632	288	3584
18	4544	230	4496
19	5680	184	5632
20	7104	147	7056
21	8880	118	8832
22	11104	94	11056
23	13880	75	13832
24	17352	60	17304
25	21696	48	21648
26	27120	38	27072
27	33904	30	33856
28	42384	24	42336
29	52984	19	52936
30	66232	15	66184
31	82792	12	82744
32	103496	10	103448
33	129376	8	129328
34	161720	6	161672
35	202152	5	202104
36	252696	4	252648
37	315872	3	315824
38	394840	2	394792
39	524288	2	524240

Chunk Size = 48 bytes metadata

Large Item Size in Memcached  

• Before version 1.4.29 max-item-size parameter is tied largest chunk size.

Benefit Upgrading to Memcached 1.5.10  

• Cumulative fixes, such as ASCII multigets, ( CVE-2017-9951) Fixed in Memcached 1.4.39 and ( limit crawls for metadumper).
• Better connection management by closing connections at the connection limit.
• Improved item-size management for item size above 1MB.
  • Before version 1.5.0 Item larger than 1 MB is always using largest slab size (around 512k). For example if we store 700 k item it will use two chunks in slab 39 (chunk size 524288). Since we only use 700k on two chunk of slab class 39 we will waste around 300k of memory
  • Starting on version 1.5.0, Memcached will use multiple slab class that minimize memory waste.
• Better performance and memory-overhead improvements by reducing memory requirements per-item by a few bytes.
  • On Version 1.4.39 ( Release Notes) : save four bytes per item if client flags is set to 0.