21.4.24 ndb_restore — Restore an NDB Cluster Backup

The NDB Cluster restoration program is implemented as a separate command-line utility ndb_restore, which can normally be found in the MySQL bin directory. This program reads the files created as a result of the backup and inserts the stored information into the database.

Note

Beginning with NDB 7.5.15 and 7.6.11, this program no longer prints NDBT_ProgramExit: ... when it finishes its run. Applications depending on this behavior should be modified accordingly when upgrading from earlier releases.

ndb_restore must be executed once for each of the backup files that were created by the START BACKUP command used to create the backup (see Section 21.5.8.2, “Using The NDB Cluster Management Client to Create a Backup”). This is equal to the number of data nodes in the cluster at the time that the backup was created.

Note

Before using ndb_restore, it is recommended that the cluster be running in single user mode, unless you are restoring multiple data nodes in parallel. See Section 21.5.6, “NDB Cluster Single User Mode”, for more information.

The following table includes options that are specific to the NDB Cluster native backup restoration program ndb_restore. Additional descriptions follow the table. For options common to most NDB Cluster programs (including ndb_restore), see Section 21.4.32, “Options Common to NDB Cluster Programs — Options Common to NDB Cluster Programs”.

Table 21.255 Command-line options for the ndb_restore program

Format Description Added, Deprecated, or Removed

--allow-pk-changes[=0|1]

Allow changes to set of columns making up table's primary key

ADDED: NDB 7.6.14

--append

Append data to tab-delimited file

(Supported in all MySQL 5.7 based releases)

--backup-path=dir_name

Path to backup files directory

(Supported in all MySQL 5.7 based releases)

--backupid=#,

-b

Restore from backup having this ID

(Supported in all MySQL 5.7 based releases)

--connect,

-c

Alias for --connectstring

(Supported in all MySQL 5.7 based releases)

--disable-indexes

Causes indexes from backup to be ignored; may decrease time needed to restore data

(Supported in all MySQL 5.7 based releases)

--dont-ignore-systab-0,

-f

Do not ignore system table during restore; experimental only; not for production use

(Supported in all MySQL 5.7 based releases)

--exclude-databases=db-list

List of one or more databases to exclude (includes those not named)

(Supported in all MySQL 5.7 based releases)

--exclude-intermediate-sql-tables[=TRUE|FALSE]

If TRUE (default), do not restore any intermediate tables (having names prefixed with '#sql-') that were left over from copying ALTER TABLE operations

(Supported in all MySQL 5.7 based releases)

--exclude-missing-columns

Causes columns from backup version of table that are missing from version of table in database to be ignored

(Supported in all MySQL 5.7 based releases)

--exclude-missing-tables

Causes tables from backup that are missing from database to be ignored

(Supported in all MySQL 5.7 based releases)

--exclude-tables=table-list

List of one or more tables to exclude (includes those in same database that are not named); each table reference must include database name

(Supported in all MySQL 5.7 based releases)

--fields-enclosed-by=char

Fields are enclosed by this character

(Supported in all MySQL 5.7 based releases)

--fields-optionally-enclosed-by

Fields are optionally enclosed by this character

(Supported in all MySQL 5.7 based releases)

--fields-terminated-by=char

Fields are terminated by this character

(Supported in all MySQL 5.7 based releases)

--hex

Print binary types in hexadecimal format

(Supported in all MySQL 5.7 based releases)

--ignore-extended-pk-updates[=0|1]

Ignore log entries containing updates to columns now included in extended primary key

ADDED: NDB 7.6.14

--include-databases=db-list

List of one or more databases to restore (excludes those not named)

(Supported in all MySQL 5.7 based releases)

--include-tables=table-list

List of one or more tables to restore (excludes those in same database that are not named); each table reference must include database name

(Supported in all MySQL 5.7 based releases)

--lines-terminated-by=char

Lines are terminated by this character

(Supported in all MySQL 5.7 based releases)

--lossy-conversions,

-L

Allow lossy conversions of column values (type demotions or changes in sign) when restoring data from backup

(Supported in all MySQL 5.7 based releases)

--no-binlog

If mysqld is connected and using binary logging, do not log restored data

(Supported in all MySQL 5.7 based releases)

--no-restore-disk-objects,

-d

Do not restore objects relating to Disk Data

(Supported in all MySQL 5.7 based releases)

--no-upgrade,

-u

Do not upgrade array type for varsize attributes which do not already resize VAR data, and do not change column attributes

(Supported in all MySQL 5.7 based releases)

--ndb-nodegroup-map=map,

-z

Nodegroup map for NDBCLUSTER storage engine; syntax: list of (source_nodegroup, destination_nodegroup)

(Supported in all MySQL 5.7 based releases)

--nodeid=#,

-n

ID of node where backup was taken

(Supported in all MySQL 5.7 based releases)

--num-slices=#

Number of slices to apply when restoring by slice

ADDED: NDB 7.6.13

--parallelism=#,

-p

Number of parallel transactions to use while restoring data

(Supported in all MySQL 5.7 based releases)

--preserve-trailing-spaces,

-P

Allow preservation of trailing spaces (including padding) when promoting fixed-width string types to variable-width types

(Supported in all MySQL 5.7 based releases)

--print

Print metadata, data, and log to stdout (equivalent to --print-meta --print-data --print-log)

(Supported in all MySQL 5.7 based releases)

--print-data

Print data to stdout

(Supported in all MySQL 5.7 based releases)

--print-log

Print log to stdout

(Supported in all MySQL 5.7 based releases)

--print-meta

Print metadata to stdout

(Supported in all MySQL 5.7 based releases)

--print-sql-log

Write SQL log to stdout; default is FALSE

ADDED: NDB 7.5.4

--progress-frequency=#

Print status of restore each given number of seconds

(Supported in all MySQL 5.7 based releases)

--promote-attributes,

-A

Allow attributes to be promoted when restoring data from backup

(Supported in all MySQL 5.7 based releases)

--rebuild-indexes

Causes multithreaded rebuilding of ordered indexes found in backup; number of threads used is determined by setting BuildIndexThreads

(Supported in all MySQL 5.7 based releases)

--remap-column=[db].[tbl].[col]:[fn]:[args]

Apply offset to value of specified column using indicated function and arguments

ADDED: NDB 7.6.14

--restore-data,

-r

Restore table data and logs into NDB Cluster using NDB API

(Supported in all MySQL 5.7 based releases)

--restore-epoch,

-e

Restore epoch info into status table; useful on replica cluster for starting replication; updates or inserts row in mysql.ndb_apply_status with ID 0

(Supported in all MySQL 5.7 based releases)

--restore-meta,

-m

Restore metadata to NDB Cluster using NDB API

(Supported in all MySQL 5.7 based releases)

--restore-privilege-tables

Restore MySQL privilege tables that were previously moved to NDB

(Supported in all MySQL 5.7 based releases)

--rewrite-database=olddb,newdb

Restore to differently named database

(Supported in all MySQL 5.7 based releases)

--skip-broken-objects

Ignore missing blob tables in backup file

(Supported in all MySQL 5.7 based releases)

--skip-table-check,

-s

Skip table structure check during restore

(Supported in all MySQL 5.7 based releases)

--skip-unknown-objects

Causes schema objects not recognized by ndb_restore to be ignored when restoring backup made from newer NDB version to older version

(Supported in all MySQL 5.7 based releases)

--slice-id=#

Slice ID, when restoring by slices

ADDED: NDB 7.6.13

--tab=dir_name,

-T dir_name

Creates a tab-separated .txt file for each table in path provided

(Supported in all MySQL 5.7 based releases)

--verbose=#

Level of verbosity in output

(Supported in all MySQL 5.7 based releases)


Typical options for this utility are shown here:

ndb_restore [-c connection_string] -n node_id -b backup_id \
      [-m] -r --backup-path=/path/to/backup/files

Normally, when restoring from an NDB Cluster backup, ndb_restore requires at a minimum the --nodeid (short form: -n), --backupid (short form: -b), and --backup-path options. In addition, when ndb_restore is used to restore any tables containing unique indexes, you must include --disable-indexes or --rebuild-indexes. (Bug #57782, Bug #11764893)

The -c option is used to specify a connection string which tells ndb_restore where to locate the cluster management server (see Section 21.3.3.3, “NDB Cluster Connection Strings”). If this option is not used, then ndb_restore attempts to connect to a management server on localhost:1186. This utility acts as a cluster API node, and so requires a free connection slot to connect to the cluster management server. This means that there must be at least one [api] or [mysqld] section that can be used by it in the cluster config.ini file. It is a good idea to keep at least one empty [api] or [mysqld] section in config.ini that is not being used for a MySQL server or other application for this reason (see Section 21.3.3.7, “Defining SQL and Other API Nodes in an NDB Cluster”).

You can verify that ndb_restore is connected to the cluster by using the SHOW command in the ndb_mgm management client. You can also accomplish this from a system shell, as shown here:

shell> ndb_mgm -e "SHOW"

More detailed information about all options used by ndb_restore can be found in the following list:

  • --allow-pk-changes

    Property Value
    Command-Line Format --allow-pk-changes[=0|1]
    Introduced 5.7.29-ndb-7.6.14
    Type Integer
    Default Value 0
    Minimum Value 0
    Maximum Value 1

    When this option is set to 1, ndb_restore allows the primary keys in a table definition to differ from that of the same table in the backup. This may be desirable when backing up and restoring between different schema versions with primary key changes on one or more tables, and it appears that performing the restore operation using ndb_restore is simpler or mor efficient than issuing many ALTER TABLE statements after restoring table schemas and data.

    The following changes in primary key definitions are supported by --allow-pk-changes:

    • Extending the primary key: A non-nullable column that exists in the table schema in the backup becomes part of the table's primary key in the database.

      Important

      When extending a table's primary key, any columns which become part of primary key must not be updated while the backup is being taken; any such updates discovered by ndb_restore cause the restore operation to fail, even when no change in value takes place. In some cases, it may be possible to override this behavior using the --ignore-extended-pk-updates option; see the description of this option for more information.

    • Contracting the primary key (1): A column that is already part of the table's primary key in the backup schema is no longer part of the primary key, but remains in the table.

    • Contracting the primary key (2): A column that is already part of the table's primary key in the backup schema is removed from the table entirely.

    These differences can be combined with other schema differences supported by ndb_restore, including changes to blob and text columns requiring the use of staging tables.

    Basic steps in a typical scenario using primary key schema changes are listed here:

    1. Restore table schemas using ndb_restore --restore-meta

    2. Alter schema to that desired, or create it

    3. Back up the desired schema

    4. Run ndb_restore --disable-indexes using the backup from the previous step, to drop indexes and constraints

    5. Run ndb_restore --allow-pk-changes (possibly along with --ignore-extended-pk-updates, --disable-indexes, and possibly other options as needed) to restore all data

    6. Run ndb_restore --rebuild-indexes using the backup made with the desired schema, to rebuild indexes and constraints

    When extending the primary key, it may be necessary for ndb_restore to use a temporary secondary unique index during the restore operation to map from the old primary key to the new one. Such an index is created only when necessary to apply events from the backup log to a table which has an extended primary key. This index is named NDB$RESTORE_PK_MAPPING, and is created on each table requiring it; it can be shared, if necessary, by multiple instances of ndb_restore instances running in parallel. (Running ndb_restore --rebuild-indexes at the end of the restore process causes this index to be dropped.)

  • --append

    Property Value
    Command-Line Format --append

    When used with the --tab and --print-data options, this causes the data to be appended to any existing files having the same names.

  • --backup-path=dir_name

    Property Value
    Command-Line Format --backup-path=dir_name
    Type Directory name
    Default Value ./

    The path to the backup directory is required; this is supplied to ndb_restore using the --backup-path option, and must include the subdirectory corresponding to the ID backup of the backup to be restored. For example, if the data node's DataDir is /var/lib/mysql-cluster, then the backup directory is /var/lib/mysql-cluster/BACKUP, and the backup files for the backup with the ID 3 can be found in /var/lib/mysql-cluster/BACKUP/BACKUP-3. The path may be absolute or relative to the directory in which the ndb_restore executable is located, and may be optionally prefixed with backup-path=.

    It is possible to restore a backup to a database with a different configuration than it was created from. For example, suppose that a backup with backup ID 12, created in a cluster with two storage nodes having the node IDs 2 and 3, is to be restored to a cluster with four nodes. Then ndb_restore must be run twice—once for each storage node in the cluster where the backup was taken. However, ndb_restore cannot always restore backups made from a cluster running one version of MySQL to a cluster running a different MySQL version. See Section 21.2.9, “Upgrading and Downgrading NDB Cluster”, for more information.

    Important

    It is not possible to restore a backup made from a newer version of NDB Cluster using an older version of ndb_restore. You can restore a backup made from a newer version of MySQL to an older cluster, but you must use a copy of ndb_restore from the newer NDB Cluster version to do so.

    For example, to restore a cluster backup taken from a cluster running NDB Cluster 7.5.20 to a cluster running NDB Cluster 7.4.30, you must use the ndb_restore that comes with the NDB Cluster 7.5.20 distribution.

    For more rapid restoration, the data may be restored in parallel, provided that there is a sufficient number of cluster connections available. That is, when restoring to multiple nodes in parallel, you must have an [api] or [mysqld] section in the cluster config.ini file available for each concurrent ndb_restore process. However, the data files must always be applied before the logs.

  • --backupid=#, -b

    Property Value
    Command-Line Format --backupid=#
    Type Numeric
    Default Value none

    This option is used to specify the ID or sequence number of the backup, and is the same number shown by the management client in the Backup backup_id completed message displayed upon completion of a backup. (See Section 21.5.8.2, “Using The NDB Cluster Management Client to Create a Backup”.)

    Important

    When restoring cluster backups, you must be sure to restore all data nodes from backups having the same backup ID. Using files from different backups will at best result in restoring the cluster to an inconsistent state, and may fail altogether.

    In NDB 7.5.13 and later, and in NDB 7.6.9 and later, this option is required.

  • --connect, -c

    Property Value
    Command-Line Format --connect
    Type String
    Default Value localhost:1186

    Alias for --ndb-connectstring.

  • --disable-indexes

    Property Value
    Command-Line Format --disable-indexes

    Disable restoration of indexes during restoration of the data from a native NDB backup. Afterwards, you can restore indexes for all tables at once with multithreaded building of indexes using --rebuild-indexes, which should be faster than rebuilding indexes concurrently for very large tables.

  • --dont-ignore-systab-0, -f

    Property Value
    Command-Line Format --dont-ignore-systab-0

    Normally, when restoring table data and metadata, ndb_restore ignores the copy of the NDB system table that is present in the backup. --dont-ignore-systab-0 causes the system table to be restored. This option is intended for experimental and development use only, and is not recommended in a production environment.

  • --exclude-databases=db-list

    Property Value
    Command-Line Format --exclude-databases=db-list
    Type String
    Default Value

    Comma-delimited list of one or more databases which should not be restored.

    This option is often used in combination with --exclude-tables; see that option's description for further information and examples.

  • --exclude-intermediate-sql-tables[=TRUE|FALSE]

    Property Value
    Command-Line Format --exclude-intermediate-sql-tables[=TRUE|FALSE]
    Type Boolean
    Default Value TRUE

    When performing copying ALTER TABLE operations, mysqld creates intermediate tables (whose names are prefixed with #sql-). When TRUE, the --exclude-intermediate-sql-tables option keeps ndb_restore from restoring such tables that may have been left over from these operations. This option is TRUE by default.

  • --exclude-missing-columns

    Property Value
    Command-Line Format --exclude-missing-columns

    It is possible to restore only selected table columns using this option, which causes ndb_restore to ignore any columns missing from tables being restored as compared to the versions of those tables found in the backup. This option applies to all tables being restored. If you wish to apply this option only to selected tables or databases, you can use it in combination with one or more of the --include-* or --exclude-* options described elsewhere in this section to do so, then restore data to the remaining tables using a complementary set of these options.

  • --exclude-missing-tables

    Property Value
    Command-Line Format --exclude-missing-tables

    It is possible to restore only selected tables using this option, which causes ndb_restore to ignore any tables from the backup that are not found in the target database.

  • --exclude-tables=table-list

    Property Value
    Command-Line Format --exclude-tables=table-list
    Type String
    Default Value

    List of one or more tables to exclude; each table reference must include the database name. Often used together with --exclude-databases.

    When --exclude-databases or --exclude-tables is used, only those databases or tables named by the option are excluded; all other databases and tables are restored by ndb_restore.

    This table shows several invocations of ndb_restore usng --exclude-* options (other options possibly required have been omitted for clarity), and the effects these options have on restoring from an NDB Cluster backup:

    Table 21.256 Several invocations of ndb_restore using --exclude-* options, and the effects these options have on restoring from an NDB Cluster backup.

    Option Result
    --exclude-databases=db1 All tables in all databases except db1 are restored; no tables in db1 are restored
    --exclude-databases=db1,db2 (or --exclude-databases=db1 --exclude-databases=db2) All tables in all databases except db1 and db2 are restored; no tables in db1 or db2 are restored
    --exclude-tables=db1.t1 All tables except t1 in database db1 are restored; all other tables in db1 are restored; all tables in all other databases are restored
    --exclude-tables=db1.t2,db2.t1 (or --exclude-tables=db1.t2 --exclude-tables=db2.t1) All tables in database db1 except for t2 and all tables in database db2 except for table t1 are restored; no other tables in db1 or db2 are restored; all tables in all other databases are restored

    You can use these two options together. For example, the following causes all tables in all databases except for databases db1 and db2, and tables t1 and t2 in database db3, to be restored:

    shell> ndb_restore [...] --exclude-databases=db1,db2 --exclude-tables=db3.t1,db3.t2

    (Again, we have omitted other possibly necessary options in the interest of clarity and brevity from the example just shown.)

    You can use --include-* and --exclude-* options together, subject to the following rules:

    • The actions of all --include-* and --exclude-* options are cumulative.

    • All --include-* and --exclude-* options are evaluated in the order passed to ndb_restore, from right to left.

    • In the event of conflicting options, the first (rightmost) option takes precedence. In other words, the first option (going from right to left) that matches against a given database or table wins.

    For example, the following set of options causes ndb_restore to restore all tables from database db1 except db1.t1, while restoring no other tables from any other databases:

    --include-databases=db1 --exclude-tables=db1.t1

    However, reversing the order of the options just given simply causes all tables from database db1 to be restored (including db1.t1, but no tables from any other database), because the --include-databases option, being farthest to the right, is the first match against database db1 and thus takes precedence over any other option that matches db1 or any tables in db1:

    --exclude-tables=db1.t1 --include-databases=db1
  • --fields-enclosed-by=char

    Property Value
    Command-Line Format --fields-enclosed-by=char
    Type String
    Default Value

    Each column value is enclosed by the string passed to this option (regardless of data type; see the description of --fields-optionally-enclosed-by).

  • --fields-optionally-enclosed-by

    Property Value
    Command-Line Format --fields-optionally-enclosed-by
    Type String
    Default Value

    The string passed to this option is used to enclose column values containing character data (such as CHAR, VARCHAR, BINARY, TEXT, or ENUM).

  • --fields-terminated-by=char

    Property Value
    Command-Line Format --fields-terminated-by=char
    Type String
    Default Value \t (tab)

    The string passed to this option is used to separate column values. The default value is a tab character (\t).

  • --hex

    Property Value
    Command-Line Format --hex

    If this option is used, all binary values are output in hexadecimal format.

  • --ignore-extended-pk-updates

    Property Value
    Command-Line Format --ignore-extended-pk-updates[=0|1]
    Introduced 5.7.29-ndb-7.6.14
    Type Integer
    Default Value 0
    Minimum Value 0
    Maximum Value 1

    When using the --allow-pk-changes option, columns which become part of a table's primary key must not be updated while the backup is being taken; such columns should keep the same values from the time values are inserted into them until the rows containing the values are deleted. If ndb_restore encounters updates to these columns when restoring a backup, the restore fails. Because some applications may set values for all columns when updating a row, even when some column values are not changed, the backup may include log events appearing to update columns which are not in fact modified. In such cases you can set --ignore-extended-pk-updates to 1, forcing ndb_restore to ignore such updates.

    Important

    When causing these updates to be ignored, the user is responsible for ensuring that there are no updates to the values of any columns that become part of the primary key.

    For more information, see the description of --allow-pk-changes.

  • --include-databases=db-list

    Property Value
    Command-Line Format --include-databases=db-list
    Type String
    Default Value

    Comma-delimited list of one or more databases to restore. Often used together with --include-tables; see the description of that option for further information and examples.

  • --include-tables=table-list

    Property Value
    Command-Line Format --include-tables=table-list
    Type String
    Default Value

    Comma-delimited list of tables to restore; each table reference must include the database name.

    When --include-databases or --include-tables is used, only those databases or tables named by the option are restored; all other databases and