Delayed Replica Set Members

Delayed members contain copies of a replica set’s data set. However, a delayed member’s data set reflects an earlier, or delayed, state of the set. For example, if the current time is 09:52 and a member has a delay of an hour, the delayed member has no operation more recent than 08:52.

Because delayed members are a “rolling backup” or a running “historical” snapshot of the data set, they may help you recover from various kinds of human error. For example, a delayed member can make it possible to recover from unsuccessful application upgrades and operator errors including dropped databases and collections.

Considerations

Requirements

Delayed members:

  • Must be priority 0 members. Set the priority to 0 to prevent a delayed member from becoming primary.
  • Should be hidden members. Always prevent applications from seeing and querying delayed members.
  • do vote in elections for primary, if members[n].votes is set to 1.

Behavior

Delayed members copy and apply operations from the source oplog on a delay. When choosing the amount of delay, consider that the amount of delay:

  • must be equal to or greater than your expected maintenance window durations.
  • must be smaller than the capacity of the oplog. For more information on oplog size, see Oplog Size.

Write Concern

Delayed replica set members can acknowledge write operations issued with w: <number>. For write operations isued with w : "majority", however, delayed members must also be voting members (i.e. members[n].votes greater than 0) to acknowledge the "majority" write operation. Non-voting replica set members (i.e. members[n].votes is 0) cannot contribute to acknowledging write operations with majority write concern.

Delayed secondaries can return write acknowledgment no earlier than the configured slaveDelay.

Sharding

In sharded clusters, delayed members have limited utility when the balancer is enabled. Because delayed members replicate chunk migrations with a delay, the state of delayed members in a sharded cluster are not useful for recovering to a previous state of the sharded cluster if any migrations occur during the delay window.

Example

In the following 5-member replica set, the primary and all secondaries have copies of the data set. One member applies operations with a delay of 3600 seconds (one hour). This delayed member is also hidden and is a priority 0 member.

Diagram of a 5 member replica set with a hidden delayed priority 0 member.

Configuration

A delayed member has its members[n].priority equal to 0, members[n].hidden equal to true, and its members[n].slaveDelay equal to the number of seconds of delay:

{
   "_id" : <num>,
   "host" : <hostname:port>,
   "priority" : 0,
   "slaveDelay" : <seconds>,
   "hidden" : true
}

To configure a delayed member, see Configure a Delayed Replica Set Member.