Merge Chunks in a Sharded Cluster
Overview
Changed in version 3.0.
The mergeChunks command allows you to combine continuous chunks on the same shard into a single chunk. This tutorial explains how to merge neighboring chunks in a sharded cluster.
Procedure
Note
Examples in this procedure use a members collection in the test database, using the username field as the shard key.
Identify Chunk Ranges
In the mongo shell, identify the chunk ranges with the following operation:
sh.status()
In the output, the chunk ranges appear after the chunk counts for each sharded collection, as in the following example:
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5c0e886b6602192935f35008")
}
shards:
{ "_id" : "shardA", "host" : "shardA/shardA-m1.example.net:27018,shardA-m2.example.net:27018,shardA-m3.example.net:27018", "state" : 1 }
{ "_id" : "shardB", "host" : "shardB/shardB-m1.example.net:27018,shardB-m2.example.net:27018,shardB-m3.example.net:27018", "state" : 1 }
active mongoses:
"3.6.8" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
7 : Success
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shardA 1
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 0)
{ "_id" : "test", "primary" : "shardA", "partitioned" : true }
test.members
shard key: { "username" : 1 }
unique: false
balancing: true
chunks:
shardA 7
shardB 7
{ "username" : { "$minKey" : 1 } } -->> { "username" : "user16643" } on : shardB Timestamp(2, 0)
{ "username" : "user16643" } -->> { "username" : "user2329" } on : shardB Timestamp(3, 0)
{ "username" : "user2329" } -->> { "username" : "user29937" } on : shardB Timestamp(4, 0)
{ "username" : "user29937" } -->> { "username" : "user36583" } on : shardB Timestamp(5, 0)
{ "username" : "user36583" } -->> { "username" : "user43229" } on : shardB Timestamp(6, 0)
{ "username" : "user43229" } -->> { "username" : "user49877" } on : shardB Timestamp(7, 0)
{ "username" : "user49877" } -->> { "username" : "user56522" } on : shardB Timestamp(8, 0)
{ "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
{ "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
{ "username" : "user69816" } -->> { "username" : "user76462" } on : shardA Timestamp(1, 19)
{ "username" : "user76462" } -->> { "username" : "user83108" } on : shardA Timestamp(1, 21)
{ "username" : "user83108" } -->> { "username" : "user89756" } on : shardA Timestamp(1, 23)
{ "username" : "user89756" } -->> { "username" : "user96401" } on : shardA Timestamp(1, 25)
{ "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)
The chunk ranges appear after the chunk counts for each sharded collection. For example, the following are the chunk ranges for the test.members collection:
{ "username" : { "$minKey" : 1 } } -->> { "username" : "user16643" } on : shardB Timestamp(2, 0)
{ "username" : "user16643" } -->> { "username" : "user2329" } on : shardB Timestamp(3, 0)
{ "username" : "user2329" } -->> { "username" : "user29937" } on : shardB Timestamp(4, 0)
{ "username" : "user29937" } -->> { "username" : "user36583" } on : shardB Timestamp(5, 0)
{ "username" : "user36583" } -->> { "username" : "user43229" } on : shardB Timestamp(6, 0)
{ "username" : "user43229" } -->> { "username" : "user49877" } on : shardB Timestamp(7, 0)
{ "username" : "user49877" } -->> { "username" : "user56522" } on : shardB Timestamp(8, 0)
{ "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
{ "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
{ "username" : "user69816" } -->> { "username" : "user76462" } on : shardA Timestamp(1, 19)
{ "username" : "user76462" } -->> { "username" : "user83108" } on : shardA Timestamp(1, 21)
{ "username" : "user83108" } -->> { "username" : "user89756" } on : shardA Timestamp(1, 23)
{ "username" : "user89756" } -->> { "username" : "user96401" } on : shardA Timestamp(1, 25)
{ "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)
Merge Chunks
Merge contiguous chunks on the same shard.
For example, consider the following chunk ranges on shardA:
Note
The chunks to be merged are highlighted.
{ "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
{ "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
{ "username" : "user69816" } -->> { "username" : "user76462" } on : shardA Timestamp(1, 19)
{ "username" : "user76462" } -->> { "username" : "user83108" } on : shardA Timestamp(1, 21)
{ "username" : "user83108" } -->> { "username" : "user89756" } on : shardA Timestamp(1, 23)
{ "username" : "user89756" } -->> { "username" : "user96401" } on : shardA Timestamp(1, 25)
{ "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)
To merge the highlighted contiguous chunks, issue the mergeChunks command against the admin database:
db.adminCommand( {
mergeChunks: "test.members",
bounds: [ { "username" : "user69816" },
{ "username" : "user96401" } ]
} )
On success, mergeChunks produces the following output:
{
"ok" : 1,
"operationTime" : Timestamp(1544636362, 12),
"$clusterTime" : {
"clusterTime" : Timestamp(1544636362, 13),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
On any failure condition, mergeChunks returns a document where the value of the ok field is 0.
View Merged Chunks Ranges
After merging the identified chunks, confirm the new chunk, as follows:
sh.status()
The output of sh.status() should resemble:
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5c0e886b6602192935f35008")
}
shards:
{ "_id" : "shardA", "host" : "shardA/shardA-m1.example.net:27018,shardA-m2.example.net:27018,shardA-m3.example.net:27018", "state" : 1 }
{ "_id" : "shardB", "host" : "shardB/shardB-m1.example.net:27018,shardB-m2.example.net:27018,shardB-m3.example.net:27018", "state" : 1 }
active mongoses:
"3.6.8" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
8 : Success
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shardA 1
{ "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 0)
{ "_id" : "test", "primary" : "shardA", "partitioned" : true }
test.members
shard key: { "username" : 1 }
unique: false
balancing: true
chunks:
shardA 5
shardB 6
{ "username" : { "$minKey" : 1 } } -->> { "username" : "user16643" } on : shardA Timestamp(9, 0)
{ "username" : "user16643" } -->> { "username" : "user2329" } on : shardB Timestamp(9, 1)
{ "username" : "user2329" } -->> { "username" : "user29937" } on : shardB Timestamp(4, 0)
{ "username" : "user29937" } -->> { "username" : "user36583" } on : shardB Timestamp(5, 0)
{ "username" : "user36583" } -->> { "username" : "user43229" } on : shardB Timestamp(6, 0)
{ "username" : "user43229" } -->> { "username" : "user49877" } on : shardB Timestamp(7, 0)
{ "username" : "user49877" } -->> { "username" : "user56522" } on : shardB Timestamp(8, 0)
{ "username" : "user56522" } -->> { "username" : "user63169" } on : shardA Timestamp(8, 1)
{ "username" : "user63169" } -->> { "username" : "user69816" } on : shardA Timestamp(1, 17)
{ "username" : "user69816" } -->> { "username" : "user96401" } on : shardA Timestamp(8, 2)
{ "username" : "user96401" } -->> { "username" : { "$maxKey" : 1 } } on : shardA Timestamp(1, 26)
After the merge, the balancer may migrate chunks across shards to ensure a more even distribution of chunks.