Migrate a Sharded Cluster to Different Hardware
On this page
The tutorial is specific to MongoDB 3.6. For earlier versions of MongoDB, refer to the corresponding version of the MongoDB Manual.
Starting in MongoDB 3.2, config servers for sharded clusters can be deployed as a replica set. The replica set config servers must run the WiredTiger storage engine. MongoDB 3.2 deprecates the use of three mirrored mongod
instances for config servers.
This procedure moves the components of the sharded cluster to a new hardware system without downtime for reads and writes.
Important
While the migration is in progress, do not attempt to change to the Sharded Cluster Metadata. Do not use any operation that modifies the cluster metadata in any way. For example, do not create or drop databases, create or drop collections, or use any sharding commands.
Disable the Balancer
Disable the balancer to stop chunk migration and do not perform any metadata write operations until the process finishes. If a migration is in progress, the balancer will complete the in-progress migration before stopping.
To disable the balancer, connect to one of the cluster’s mongos
instances and issue the following method:
To check the balancer state, issue the sh.getBalancerState()
method.
For more information, see Disable the Balancer.
Migrate Each Config Server Separately
Changed in version 3.4.
Starting in MongoDB 3.2, config servers for sharded clusters can be deployed as a replica set (CSRS) instead of three mirrored config servers (SCCC). Using a replica set for the config servers improves consistency across the config servers, since MongoDB can take advantage of the standard replica set read and write protocols for the config data. In addition, using a replica set for config servers allows a sharded cluster to have more than 3 config servers since a replica set can have up to 50 members. To deploy config servers as a replica set, the config servers must run the WiredTiger storage engine.
In version 3.4, MongoDB removes support for SCCC config servers.
The following restrictions apply to a replica set configuration when used for config servers:
- Must have zero arbiters.
- Must have no delayed members.
- Must build indexes (i.e. no member should have
buildIndexes
setting set to false).
For each member of the config server replica set:
Important
Replace the secondary members before replacing the primary.
Start the replacement config server.
Start a mongod
instance, specifying the --configsvr
, --replSet
, --bind_ip
options, and other options as appropriate to your deployment.
Warning
Before binding to a non-localhost (e.g. publicly accessible) IP address, ensure you have secured your cluster from unauthorized access. For a complete list of security recommendations, see Security Checklist. At minimum, consider enabling authentication and hardening network infrastructure.
Add the new config server to the replica set.
Connect a mongo
shell to the primary of the config server replica set and use rs.add()
to add the new member.
Tip
When a newly added secondary has its votes
and priority
settings greater than zero, during its initial sync, the secondary still counts as a voting member even though it cannot serve reads nor become primary because its data is not yet consistent.
This can lead to a case where a majority of the voting members are online but no primary can be elected. To avoid such situations, consider adding the new secondary initially with priority :0
and votes :0
. Then, once the member has transitioned into SECONDARY
state, use rs.reconfig()
to update its priority and votes.
The initial sync process copies all the data from one member of the config server replica set to the new member without restarting.
mongos
instances automatically recognize the change in the config server replica set members without restarting.
Update the newly added config server’s votes and priority settings.
Ensure that the new member has reached
SECONDARY
state. To check the state of the replica set members, runrs.status()
:Reconfigure the replica set to update the votes and priority of the new member:
where
n
is the array index of the new member in themembers
array.
Warning
- The
rs.reconfig()
shell method can force the current primary to step down, which causes an election. When the primary steps down, themongod
closes all client connections. While this typically takes 10-20 seconds, try to make these changes during scheduled maintenance periods. - Avoid reconfiguring replica sets that contain members of different MongoDB versions as validation rules may differ across MongoDB versions.
Remove the member to replace from the config server replica set.
Upon completion of initial sync of the replacement config server, from a mongo
shell connected to the primary, use rs.remove()
to remove the old member.
mongos
instances automatically recognize the change in the config server replica set members without restarting.
Restart the mongos
Instances
Changed in version 3.2: With replica set config servers, the mongos
instances specify in the --configdb
or sharding.configDB
setting the config server replica set name and at least one of the replica set members. The mongos
instances for the sharded cluster must specify the same config server replica set name but can specify different members of the replica set.
If a mongos
instance specifies a migrated replica set member in the --configdb
or sharding.configDB
setting, update the config server setting for the next time you restart the mongos
instance.
For more information, see Start a mongos for the Sharded Cluster.
Migrate the Shards
Migrate the shards one at a time. For each shard, follow the appropriate procedure in this section.
Migrate a Replica Set Shard
To migrate a sharded cluster, migrate each member separately. First migrate the non-primary members, and then migrate the primary last.
If the replica set has two voting members, add an arbiter to the replica set to ensure the set keeps a majority of its votes available during the migration. You can remove the arbiter after completing the migration.
Migrate a Member of a Replica Set Shard
Shut down the
mongod
process. To ensure a clean shutdown, use theshutdown
command.Move the data directory (i.e., the
dbPath
) to the new machine.Restart the
mongod
process at the new location.Connect to the replica set’s current primary.
If the hostname of the member has changed, use
rs.reconfig()
to update the replica set configuration document with the new hostname.For example, the following sequence of commands updates the hostname for the instance at position
2