mysqlhotcopy.py script¶

MySQL Hotcopy wrapper¶

Provides a higher level wrapper on top of mysqlhotcopy adding features like

archive tarball creation/restoration/organization/transfers in dated folders
disk space estimation by DB queries and file system free space checking before performing hotcopy

Installation¶

Checkout directory from env repo with:

svn co http://dayabay.phys.ntu.edu.tw/repos/env/trunk/mysqlhotcopy 
cd mysqlhotcopy

Test runs to ensure pre-requisite MySQL-python is present:

./mysqlhotcopy.py --help
./mysqlhotcopy.py --dryrun tmp_ligs_offline_db hotcopy   # dryrun check of space available etc..

Pre-requisites for mysqlhotcopy.py¶

Requires MySQLdb, check that and operating env with:

sudo python -c "import MySQLdb"

If that gives errors will need to:

sudo yum install MySQL-python

Intended to be used in system python from sudo, operating from non-pristine env may cause errors related to setuptools.

Pre-requisites for mysqlhotcopy¶

Perl module DBD::mysql¶

Without this perl module errors like the below are observed:

install_driver(mysql) failed: Can't locate DBD/mysql.pm 

[root@belle1 ~]# perl -mDBD::mysql -e ''
Can't locate DBD/mysql.pm in @INC (@INC contains: /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8 .).
BEGIN failed--compilation aborted.

Features of mysqlhotcopy¶

mysqlhotcopy does low level file copying, making version closeness important

dybdb1.ihep.ac.cn        5.0.45-community-log MySQL Community Edition (GPL)
belle7.nuu.edu.tw        5.0.77-log Source distribution
cms01.phys.ntu.edu.tw    4.1.22-log

Commands¶

The first argument of the mysqlhotcopy.py script specifies the name of a mysql database to operate upon. Subsequent arguments specify actions to take. Order is important.

hotcopy: use mysqlhotcopy to copy the mysql datadir for a single database into a dated folder under backupdir, with path of form /var/dbbackup/mysqlhotcopy/belle1.nuu.edu.tw/channelquality_db/20130530_1954 the folder and its parent folders are created if necessary
coldcopy: Manual file system copy without locking the DB. Used for investigating crashed DBs, DO NOT USE FOR RELIABLE BACKUPS. TREAT WITH CAUTION
archive: archives the dated folder into a tarball and deletes the dated folder
examine: examines the paths within the tarball and determines their common prefix path and classifies as flattop or otherwise examine must be run before doing extract
transfer: uses scp to copy a tarball to a remote node
extract: extracts the content from a tarball into the mysql datadir, CAUTION THIS IS DANGEROUS test on non-critical servers first
ls: list the tarballs for a database
purge: purge the tarballs for a database, keeping the configured number

Usage steps¶

Examples of usage:

cd env/mysqlhotbackup
./mysqlhotbackup.py --help

./mysqlhotbackup.py tmp_ligs_offline_db  hotcopy archive transfer 

      # 1st argument is DB name, subsequent are the actions to take
      # the **hotcopy** action is the one during which the DB tables are locked


./mysqlhotbackup.py -t 20130516_1711 tmp_ligs_offline_db transfer 

      # if need to transfer or archive separately from the hotcopy 
      # then must specify the time tag corresponding to the hotcopy and archive 
      # to be transferred

./mysqlhotcopy.py --regex './^\(DqChannelStatus\|DqChannelStatusVld\)/'  tmp_ligs_offline_db hotcopy archive transfer      

      # using regex to only include 2 tables, this regex is tacked on to the mysqhotcopy 
      # database argument and subsequently interpreted as a perl regular expression 

./mysqlhotcopy.py -l debug --regex "^DqChannelPacked" tmp_offline_db coldcopy 

      # NB when using coldcopy the regex must be a python compatible regexp to select tables to include

./mysqlhotcopy.py --regex './^\(DqChannelPacked\|DqChannelPackedVld\)/'  tmp_offline_db hotcopy       

      # have to escape the brackets and pipe symbol to protect from shell interpretation

./mysqlhotcopy.py -C --regex './^LOCALSEQNO/' tmp_offline_db hotcopy archive 

      # for quick machinery testing, restrict to just handling a small table and disable interactive confirmations
      # note that hotcopy will delete a pre-existing same minute folder however

./mysqlhotcopy.py -l debug --flattop --remotenode S --regex '^(DqChannelStatus|DqChannelStatusVld)\.[^.]*$'  tmp_ligs_offline_db_0  coldcopy 

     # more realistic coldcopy python regex to just handle a single pair of DBI tables, note that the pattern has to match that of 
     # names of the internal mysql files ie `.MYI` `.MYD` `.frm`

rm -rf /tmp/tmp_offline_db && mysqlhotcopy.py --regex "^LOCALSEQNO"  -l debug --ALLOWEXTRACT -x /tmp -C tmp_offline_db coldcopy archive examine extract  

      # quick full coldcopy chain testing 

rm -rf /tmp/tmp_offline_db && mysqlhotcopy.py --regex "./^LOCALSEQNO/"  -l debug --ALLOWEXTRACT -x /tmp -C tmp_offline_db hotcopy archive examine extract  

      # quick full hotcopy chain testing 

mysqlhotcopy.py --regex "^DqChannelPacked"  -l debug --ALLOWEXTRACT --flattop -C --rename tmp_offline_db_ext tmp_offline_db coldcopy archive examine extract  

      # after this, can immediately "use tmp_offline_db_ext" and query against the coldcopied tables

Ownership issue FIXED¶

During development the ownership of coldcopy directories was initially not preserved, until r3745:

rm -rf /tmp/tmp_offline_db && mysqlhotcopy.py --regex "./^LOCALSEQNO/"  -l debug --ALLOWEXTRACT -x /tmp --flattop -C tmp_offline_db hotcopy archive examine extract  && ll tmp_offline_db/ &&  [ $(id -u mysql) -eq $(stat -c %u /tmp) ] && echo OK || echo NOPE

    # succeeds

hotcopy, archive, transfer¶

create mysqlhotcopy section in ~/.my.cnf ie /root/.my.cnf as this must be run as root in order to have access to the mysql DB files

[mysqlhotcopy]
socket    = /var/lib/mysql/mysql.sock
# if somehow the socket config is ignored by mysqlhotcopy then use option `--socket /tmp/mysql.sock`
host      = localhost
user      = root
password  = ***
database = information_schema
# 
# NB needs a database specified to allow DB connection to make the locks, 
# but database to backup is provided as an argument to mysqlhotbackup 
# mitigate the duplicity by using the system metadata databse `information_schema`

The hotcopy is very fast compared to the tgz creation, these are done separated (not in a pipe for example) so the time the DB is locked is kept to a minimum:

[root@belle7 blyth]# mysqlhotcopy.py tmp_offline_db hotcopy archive transfer
2013-05-16 17:11:16,649 env.mysqlhotcopy.mysqlhotcopy INFO     /home/blyth/env/bin/mysqlhotcopy.py tmp_offline_db hotcopy archive transfer
2013-05-16 17:11:16,653 env.mysqlhotcopy.mysqlhotcopy INFO     backupdir /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db 
2013-05-16 17:11:16,673 env.mysqlhotcopy.mysqlhotcopy INFO     db size in MB 152.27 
2013-05-16 17:11:16,673 env.mysqlhotcopy.mysqlhotcopy INFO     ================================== hotcopy 
2013-05-16 17:11:16,673 env.mysqlhotcopy.mysqlhotcopy INFO     sufficient free space,      required 380.675 MB less than    free 497384.726562 MB 
2013-05-16 17:11:16,673 env.mysqlhotcopy.mysqlhotcopy INFO     hotcopy of database tmp_offline_db into outd /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711 
2013-05-16 17:11:16,685 env.mysqlhotcopy.mysqlhotcopy INFO     proceed with MySQLHotCopy /usr/bin/mysqlhotcopy tmp_offline_db /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711   
2013-05-16 17:11:17,256 env.mysqlhotcopy.mysqlhotcopy INFO     seconds {'_hotcopy': 0.58285903930664062} 
2013-05-16 17:11:17,257 env.mysqlhotcopy.mysqlhotcopy INFO     ================================== archive 
2013-05-16 17:11:17,257 env.mysqlhotcopy.mysqlhotcopy INFO     sufficient free space,      required 380.675 MB less than    free 497231.179688 MB 
2013-05-16 17:11:17,257 env.mysqlhotcopy.mysqlhotcopy INFO     tagd /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711  into Tar /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711.tar.gz tmp_offline_db gz  
2013-05-16 17:11:17,258 env.mysqlhotcopy.tar INFO     creating /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711.tar.gz from /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711/tmp_offline_db 
2013-05-16 17:15:35,201 env.mysqlhotcopy.tar WARNING  deleting src /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711/tmp_offline_db directory following archive creation 
2013-05-16 17:15:35,241 env.mysqlhotcopy.mysqlhotcopy INFO     seconds {'_hotcopy': 0.58285903930664062, 'archive': 257.98302602767944, '_archive': 257.98317098617554} 
2013-05-16 17:15:35,241 env.mysqlhotcopy.mysqlhotcopy INFO     ================================== transfer 
2013-05-16 17:15:35,241 env.mysqlhotcopy.mysqlhotcopy INFO     sufficient free space,      required 380.675 MB less than    free 497335.757812 MB 
2013-05-16 17:15:35,241 env.mysqlhotcopy.mysqlhotcopy INFO     transfer Tar /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711.tar.gz tmp_offline_db gz  to remotenode C   
2013-05-16 17:15:35,242 env.mysqlhotcopy.common INFO     transfer /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711.tar.gz /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711.tar.gz C  
ssh C "mkdir -p /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db " 
ssh: connect to host 140.112.101.190 port 22: Connection timed out
time scp /var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711.tar.gz C:/var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db/20130516_1711.tar.gz 
ssh: connect to host 140.112.101.190 port 22: Connection timed out
lost connection

real    3m9.056s
user    0m0.000s
sys     0m0.007s
2013-05-16 17:21:53,351 env.mysqlhotcopy.mysqlhotcopy INFO     seconds {'transfer': 378.10944199562073, '_hotcopy': 0.58285903930664062, '_transfer': 378.10959100723267, 'archive': 257.98302602767944, '_archive': 257.98317098617554} 
[root@belle7 blyth]#

When doing archive, transfer (or extract) separately from the hotcopy specifying the timestamp is required as shown below.

extract dryrun¶

Due to the potential for damage from tampering with the mysql datadir, extraction requres a few options

Extraction of the dybdb1.ihep.ac.cn tarball onto belle7. As no DB called tmp_ligs_offline_db exists on belle7 it is necessary to provide the appropriate datadir for the node with –containerdir /var/lib/mysql

This is a dryrun due to option -n in order to check the paths are as expected:

[root@belle7 ~]# mysqlhotcopy.py -t 20130522_1541 --node dybdb1.ihep.ac.cn --rename tmp_ligs_offline_db_0 --containerdir /var/lib/mysql --ALLOWEXTRACT -n tmp_ligs_offline_db examine extract
2013-05-23 12:01:37,782 env.mysqlhotcopy.mysqlhotcopy INFO     /home/blyth/env/bin/mysqlhotcopy.py -t 20130522_1541 --node dybdb1.ihep.ac.cn --rename tmp_ligs_offline_db_0 --containerdir /var/lib/mysql --ALLOWEXTRACT -n tmp_ligs_offline_db examine extract
2013-05-23 12:01:37,782 env.mysqlhotcopy.mysqlhotcopy INFO     backupdir /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db 
2013-05-23 12:01:37,794 env.mysqlhotcopy.mysqlhotcopy INFO     failed to instanciate connection to database tmp_ligs_offline_db with exception Error 1049: Unknown database 'tmp_ligs_offline_db'  
2013-05-23 12:01:37,794 env.mysqlhotcopy.mysqlhotcopy INFO     ================================== examine 
2013-05-23 12:01:37,794 env.mysqlhotcopy.tar INFO     examining /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130522_1541.tar.gz 
2013-05-23 12:02:13,057 env.mysqlhotcopy.tar INFO     archive contains 7 items with commonprefix "" flattop True 
2013-05-23 12:02:13,057 env.mysqlhotcopy.mysqlhotcopy INFO     seconds {'_examine': 35.262603998184204, 'examine': 35.262594938278198} 
2013-05-23 12:02:13,057 env.mysqlhotcopy.mysqlhotcopy INFO     ================================== extract 
2013-05-23 12:02:13,057 env.mysqlhotcopy.mysqlhotcopy WARNING  no valid db connection using static opts.mb_required 2000 
2013-05-23 12:02:13,058 env.mysqlhotcopy.mysqlhotcopy INFO     sufficient free space,      required 2000 MB less than    free 494499.9375 MB 
2013-05-23 12:02:13,058 env.mysqlhotcopy.mysqlhotcopy INFO     proceeding
2013-05-23 12:02:13,058 env.mysqlhotcopy.mysqlhotcopy INFO     extract Tar /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130522_1541.tar.gz tmp_ligs_offline_db gz  into containerdir /var/lib/mysql   
2013-05-23 12:02:13,058 env.mysqlhotcopy.tar INFO     _flat_extract opening tarfile /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130522_1541.tar.gz 
2013-05-23 12:02:48,471 env.mysqlhotcopy.tar INFO     dryrun: _flat_extract into target /var/lib/mysql/tmp_ligs_offline_db_0 for 7 members with toplevelname tmp_ligs_offline_db_0 
2013-05-23 12:02:48,472 env.mysqlhotcopy.mysqlhotcopy INFO     seconds {'_examine': 35.262603998184204, 'examine': 35.262594938278198, 'extract': 35.413694858551025, '_extract': 35.414549112319946} 
[root@belle7 ~]#

actual extraction¶

An interactive confirmation of YES is required before the extraction is done.

[root@belle7 ~]# mysqlhotcopy.py -t 20130522_1541 --node dybdb1.ihep.ac.cn --rename tmp_ligs_offline_db_0 --containerdir /var/lib/mysql --ALLOWEXTRACT  tmp_ligs_offline_db examine extract
2013-05-23 12:06:33,546 env.mysqlhotcopy.mysqlhotcopy INFO     /home/blyth/env/bin/mysqlhotcopy.py -t 20130522_1541 --node dybdb1.ihep.ac.cn --rename tmp_ligs_offline_db_0 --containerdir /var/lib/mysql --ALLOWEXTRACT tmp_ligs_offline_db examine extract
2013-05-23 12:06:33,546 env.mysqlhotcopy.mysqlhotcopy INFO     backupdir /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db 
2013-05-23 12:06:33,561 env.mysqlhotcopy.mysqlhotcopy INFO     failed to instanciate connection to database tmp_ligs_offline_db with exception Error 1049: Unknown database 'tmp_ligs_offline_db'  
2013-05-23 12:06:33,561 env.mysqlhotcopy.mysqlhotcopy INFO     ================================== examine 
2013-05-23 12:06:33,562 env.mysqlhotcopy.tar INFO     examining /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130522_1541.tar.gz 
2013-05-23 12:07:08,913 env.mysqlhotcopy.tar INFO     archive contains 7 items with commonprefix "" flattop True 
2013-05-23 12:07:08,913 env.mysqlhotcopy.mysqlhotcopy INFO     seconds {'_examine': 35.351444005966187, 'examine': 35.35143518447876} 
2013-05-23 12:07:08,913 env.mysqlhotcopy.mysqlhotcopy INFO     ================================== extract 
2013-05-23 12:07:08,914 env.mysqlhotcopy.mysqlhotcopy WARNING  no valid db connection using static opts.mb_required 2000 
2013-05-23 12:07:08,914 env.mysqlhotcopy.mysqlhotcopy INFO     sufficient free space,      required 2000 MB less than    free 494499.882812 MB 
DO YOU REALLY WANT TO extract Tar /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130522_1541.tar.gz tmp_ligs_offline_db gz  into containerdir /var/lib/mysql    ? ENTER "YES" TO PROCEED : YES
2013-05-23 12:07:48,589 env.mysqlhotcopy.mysqlhotcopy INFO     proceeding
2013-05-23 12:07:48,589 env.mysqlhotcopy.mysqlhotcopy INFO     extract Tar /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130522_1541.tar.gz tmp_ligs_offline_db gz  into containerdir /var/lib/mysql   
2013-05-23 12:07:48,589 env.mysqlhotcopy.tar INFO     _flat_extract opening tarfile /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130522_1541.tar.gz 
2013-05-23 12:08:23,906 env.mysqlhotcopy.tar INFO     _flat_extract into target /var/lib/mysql/tmp_ligs_offline_db_0 for 7 members with toplevelname tmp_ligs_offline_db_0 
2013-05-23 12:09:06,346 env.mysqlhotcopy.tar INFO     total 2429412
-rw-rw---- 1 mysql mysql       8746 Feb  4 16:07 DqChannelStatus.frm
-rw-rw---- 1 mysql mysql 1439608104 May 16 19:15 DqChannelStatus.MYD
-rw-rw---- 1 mysql mysql 1024402432 May 16 19:42 DqChannelStatus.MYI
-rw-rw---- 1 mysql mysql       8908 May 13 13:16 DqChannelStatusVld.frm
-rw-rw---- 1 mysql mysql   17397375 May 20 06:26 DqChannelStatusVld.MYD
-rw-rw---- 1 mysql mysql    3826688 May 20 06:26 DqChannelStatusVld.MYI

2013-05-23 12:09:06,347 env.mysqlhotcopy.mysqlhotcopy INFO     seconds {'_examine': 35.351444005966187, 'examine': 35.35143518447876, 'extract': 77.757769107818604, '_extract': 117.43390297889709} 
[root@belle7 ~]#

mysqlhotcopy options¶

–allowold

Move any existing version of the destination to a backup directory for the duration of the copy. If the copy successfully completes, the backup directory is deleted - unless the –keepold flag is set. If the copy fails, the backup directory is restored.

The backup directory name is the original name with “_old” appended. Any existing versions of the backup directory are deleted.

Size estimation¶

Size of hotcopy directory close to that estimated from DB, tgz is factor of 3 smaller:

[blyth@belle7 DybPython]$ echo "select round(sum((data_length+index_length-data_free)/1024/1024),2) as TOT_MB from information_schema.tables where table_schema = 'tmp_offline_db' " | mysql -t 
+--------+
| TOT_MB |
+--------+
| 152.27 | 
+--------+

[blyth@belle7 mysqlhotcopy]$ sudo du -h 20130514_1832        
154M    20130514_1832/tmp_offline_db
154M    20130514_1832

[blyth@belle7 mysqlhotcopy]$ sudo du -h 20130514_1832.tar.gz 
49M     20130514_1832.tar.gz

Prepare directories on target for the transfer¶

[blyth@cms01 dbbackup]$ sudo mkdir -p /data/var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/
[blyth@cms01 dbbackup]$ sudo mkdir -p /data/var/dbbackup/mysqlhotcopy/dybdb2.ihep.ac.cn/tmp_ligs_offline_db/
[blyth@cms01 dbbackup]$ sudo chown -R dayabayscp /data/var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/
[blyth@cms01 dbbackup]$ sudo chown -R dayabayscp /data/var/dbbackup/mysqlhotcopy/dybdb2.ihep.ac.cn/tmp_ligs_offline_db/

And for testing N to H transfers:

[blyth@hfag data]$ sudo mkdir -p /data/var/dbbackup/mysqlhotcopy/belle7.nuu.edu.tw/tmp_offline_db

Table-by-table hotcopy, to minimise table lock time ?¶

Whilst possible to handle DBI tables in separate hotcopies of payload+validity pairs using –addtodest option this might not be a consistent backup, and there is the LOCALSEQNO too to cause problems.

–addtodest

Don’t rename target directory if it already exists, just add the copied files into it.

This is most useful when backing up a database with many large tables and you don’t want to have all the tables locked for the whole duration.

In this situation, if you are happy for groups of tables to be backed up separately (and thus possibly not be logically consistant with one another) then you can run mysqlhotcopy several times on the same database each with different db_name./table_regex/. All but the first should use the –addtodest option so the tables all end up in the same directory.

Behaviour with crashed tables¶

http://www.databasejournal.com/features/mysql/article.php/3300511/Repairing-Database-Corruption-in-MySQL.htm

[root@dybdb1 mysqlhotcopy]#   ./mysqlhotcopy.py -l debug tmp_ligs_offline_db hotcopy archive  
2013-05-20 11:15:01,291 __main__ INFO     ./mysqlhotcopy.py -l debug tmp_ligs_offline_db hotcopy archive
2013-05-20 11:15:01,294 __main__ INFO     backupdir /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db 
2013-05-20 11:15:01,311 __main__ INFO     db size in MB 3760.22 
2013-05-20 11:15:01,312 __main__ INFO     ================================== hotcopy 
2013-05-20 11:15:01,312 __main__ INFO     sufficient free space,      required 9400.55 MB less than    free 20069.015625 MB 
2013-05-20 11:15:01,312 __main__ INFO     hotcopy of database tmp_ligs_offline_db into outd /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130520_1115 
2013-05-20 11:15:01,333 __main__ INFO     proceed with MySQLHotCopy /usr/bin/mysqlhotcopy  tmp_ligs_offline_db /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130520_1115   
DBD::mysql::db do failed: Table './tmp_ligs_offline_db/DqChannelStatus' is marked as crashed and should be repaired at /usr/bin/mysqlhotcopy line 467.
2013-05-20 11:15:01,828 __main__ INFO     seconds {'_hotcopy': 0.51553797721862793} 
2013-05-20 11:15:01,828 __main__ INFO     ================================== archive 
2013-05-20 11:15:01,828 __main__ INFO     sufficient free space,      required 9400.55 MB less than    free 20069.0078125 MB 
2013-05-20 11:15:01,828 __main__ INFO     tagd /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130520_1115  into Tar /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130520_1115.tar.gz tmp_ligs_offline_db gz  
enter "YES" to confirm deletion of sourcedir /var/dbbackup/mysqlhotcopy/dybdb1.ihep.ac.cn/tmp_ligs_offline_db/20130520_1115 :YES
2013-05-20 11:15:42,169 __main__ INFO     seconds {'_hotcopy': 0.51553797721862793, 'archive': 40.34028697013855, '_archive': 40.340435981750488}

TODO:¶

tarball purging

Links

Content Skeleton

This Page

Previous topic

Next topic

mysqlhotcopy.py script¶

MySQL Hotcopy wrapper¶

Installation¶

Pre-requisites for mysqlhotcopy.py¶

Pre-requisites for mysqlhotcopy¶

Perl module DBD::mysql¶

Features of mysqlhotcopy¶

Commands¶

Usage steps¶

Ownership issue FIXED¶

hotcopy, archive, transfer¶

extract dryrun¶

actual extraction¶

mysqlhotcopy options¶

Size estimation¶

Prepare directories on target for the transfer¶

Table-by-table hotcopy, to minimise table lock time ?¶

Behaviour with crashed tables¶

TODO:¶

Navigation

Quick search

Links

Content Skeleton

This Page

Previous topic

Next topic

mysqlhotcopy.py script¶

MySQL Hotcopy wrapper¶

Installation¶

Pre-requisites for mysqlhotcopy.py¶

Pre-requisites for mysqlhotcopy¶

Perl module DBD::mysql¶

Features of mysqlhotcopy¶

Commands¶

Usage steps¶

Ownership issue FIXED¶

hotcopy, archive, transfer¶

extract dryrun¶

actual extraction¶

mysqlhotcopy options¶

Size estimation¶

Prepare directories on target for the transfer¶

Table-by-table hotcopy, to minimise table lock time ?¶

Behaviour with crashed tables¶

TODO:¶

Navigation