Tuesday, February 7, 2012

XtraBackup Manager - Job Control, Better Debugging and some little fixes...

Hi Everyone,

Just a quick note to let you know that I've just finished up adding some new features to XtraBackup Manager.

You can now get better visibility into what is going on inside XtraBackup Manager with the "xbm status" command.

It will allow you to see which backup jobs are running and also those which may be waiting to start, due to the maximum number of concurrent backup tasks already running.

It looks/works as follows:

[xbm@localhost ~]$ xbm status

XtraBackup Manager v0.8 - Copyright 2011-2012 Marin Software

Currently Running Backups:

+--------+-----------+-------------+---------------------+-------------------+------+
| Job ID |   Host    | Backup Name |     Start Time      |      Status       | PID  |
+--------+-----------+-------------+---------------------+-------------------+------+
| 14     | localhost | test-backup | 2012-02-07 14:19:19 | Performing Backup | 2525 |
+--------+-----------+-------------+---------------------+-------------------+------+

Note: I have to thank a tiny little BSD-licensed project I found on Google Code called PHP text table for saving me the need to reinvent the wheel in providing this very mysql command-line client-styled table output.


In addition to seeing which jobs are running/queued, if there is a backup job you would like to abort for some reason, then you can now simply use the "xbm kill" command with a Job ID taken from the "xbm status" output:

[xbm@localhost ~]$ xbm kill 14

XtraBackup Manager v0.8 - Copyright 2011-2012 Marin Software

Action: Backup Job ID 14 was killed.

The backup job itself will log an event at the ERROR level, like:

2012-02-07 14:19:30 -0800 [ERROR] : [ The backup job was killed by an administrator. Aborting... ]
2012-02-07 14:19:30 -0800 [INFO] : [ Cleaning up files... ]
2012-02-07 14:19:30 -0800 [INFO] : [ Released lock on port 10000. ]
2012-02-07 14:19:31 -0800 [ERROR] : [ Exiting after the backup job was killed... ]

I'm still not 100% on whether an aborted backup message should be considered an "Error" level event or an "Info" level event. My thinking is that I'd prefer to know if a job was aborted, so I figure putting it at the ERROR level will ensure it is always logged.

Now speaking quickly of the log levels -- it is now useful to increase your logging level in config.php from INFO to DEBUG.

You will see the exact commands used for running backups by XtraBackup Manager, which can be useful when troubleshooting XBM-related issues.

It will enable logging like the below -- Note: The password is _actually_ masked when writing the command to the log. You're welcome ;-)

2012-02-07 14:19:19 -0800 [INFO] : [ Staging an INCREMENTAL xtrabackup snapshot of /var/lib/mysql via ssh: mysql@localhost to /tmp/xbm-3592510/deltas... ]
2012-02-07 14:19:19 -0800 [DEBUG] : [ Attempting to run the incremental backup with command:
ssh -o StrictHostKeyChecking=no -p 22 mysql@localhost 'cd /tmp/xbm-3592510 ; innobackupex --ibbackup=xtrabackup --slave-info --incremental-lsn=2325647 /tmp/xbm-3592510/deltas --user=root --safe-slave-backup  --password=XXXXXXX --no-timestamp --incremental --throttle=0 1>&2 ' 
 ]

Aside from the above, some other small fixes were made, including ensuring that all temporary files created on the database host that you're backing up are created in the defined "staging_tmpdir" -- This is a parameter that is set at the host level in XtraBackup Manager.

eg. shell> xbm host edit hostname staging_tmpdir /path/to/use

I encourage you to check out the XtraBackup Manager Project and open issues with any problems you encounter or other feedback.

Cheers,
Lachlan

1 comment:

  1. Nice work Lachlan.

    Yeah, I think it should be an ERROR like you did. It is definitely not normal to kill a backup, and the reasons could be many. Some script gone wrong, or an admin making a mistake.

    ReplyDelete