Showing posts with label Backups. Show all posts
Showing posts with label Backups. Show all posts

Friday, 25 July 2014

Removing hosts from backuppc

Simply remove the host from the web interface and rm -rf the pc/<host> directory, then wait for the next BackupPC_nightly run - it will remove all superfluous files from the pools.

The path for this directory usually is:
/var/lib/Backuppc/pc/<hostname>
If you want to force the clean-up process, you can remove your host like this:

1. Login to the Backuppc server
2. Remove the host in the Backuppc web-interface (under hosts)
3. remove it's directory /var/lib/Backuppc/pc/<hostname>:
rm -rf /var/lib/Backuppc/pc/<hostname>
4. Shutdown backuppc:
service backuppc stop
5. Change into backuppc:
su - backuppc
6. Run the nightly script:
/usr/share/BackupPC/bin/BackupPC_nightly 0 255
7. Go back to root:
exit
8. Start backuppc again:
service backuppc start

Possibly Related Posts

Reclaim free space from Time Machine sparsebundle

You msut run these commands as root:
sudo su - 
Make sure the mount point exists:
mkdir -p /Volumes/TM
Then mount the afp share:
mount_afp 'afp://user:password@afp_server_address/share_name' /Volumes/TM
Now use hdiutil to reclaim the available free space:
hdiutil compact /Volumes/TM/ComputerName.sparsebundle/
unmont the share:
umount /Volumes/TM/
If you get an error message saying:
hdiutil: compact failed - Resource temporarily unavailable
You must make sure you don't have the afp share mounted elsewhere, you can check your mounts with:
df -h
If the output contains a line with your afp server's address or with the string "Time Machine" you have to unmount them.

The following script will do all that for you:
SRV="afp_server_address"
SAVEIFS=$IFS
IFS=$'\n';
for v in $(df -h | grep -E "$SRV|Time\sMachine"  | cut -d"%" -f3 |  sed -e "s/ *\//\//"); do
    umount "$v"
done
IFS=$SAVEIFS
mkdir -p /Volumes/TM
mount_afp "afp://user:password@$SRV/share" /Volumes/TM
hdiutil compact "/Volumes/TM/$(scutil --get ComputerName).sparsebundle"
umount /Volumes/TM/

Possibly Related Posts

Thursday, 24 July 2014

Changing Time Machine Backup Interval

You can use the following command to change the Time Machine backup interval:
sudo defaults write /System/Library/LaunchDaemons/com.apple.backupd-auto StartInterval -int 14400
The time interval is in seconds, so 43200 will start a backup every 12hrs.

Checkout my previous post to learn how to manually delete Time Machine backups.

Possibly Related Posts

Manage time machine backups

Some times you get some errors saying that your time machine's disk is full and Time Machine could not complete the backup so you need to  manually delete old backups.

tmutil provides methods of controlling and interacting with Time Machine, as well as examining and manipulating Time Machine backups. Common abilities include restoring data from backups, editing exclusions, and comparing backups.
tmutil latestbackup
Will output the path to the most recent backup and
tmutil listbackups
will list all existing backups, if you use the same backup disk for multiple machines, you can get just the backups from your machine with:
tmutil listbackups | grep "$(scutil --get ComputerName)"
The following command will delete the backups from a mac named old_mac_name:
sudo tmutil delete /Volumes/drive_name/Backups.backupdb/old_mac_name
If you want to be safe, you can pick one snapshot to delete first to be sure the command works as intended. This is nice since it could take hours to clean up some larger backup sets and you want to leave the Mac confident it's deleting the correct information store.

You can use the tmutil tool to delete backups one by one.
sudo tmutil delete /Volumes/drive_name/Backups.backupdb/mac_name/YYYY-MM-DD-hhmmss
Since tmutil was introduced with Lion, this will not work on earlier OS versions.

The tmutil delete command only removes the backup from the sparse bundle. It doesn’t actually free the disk space. To do that, you have to go a little deeper.

On your Mac is a mount point called /Volumes. You can examine the contents of this mount point with ls:
cd /Volumes
ls -1
Should output something like:
Macintosh HD
Recovery HD
Time Machine Backups
TimeMachine
These are the names of all the mounted disks (or things that look like disks) on your Mac. Notice two likely candidates for your actual TimeMachine volume. Yours may be named slightly differently, but the one you want is the one that actually shows files of type .sparsebundle . In my case, it is the volume TimeMachine:
sudo ls -l TimeMachine/
and you should see something similar to:
...
drwxr-x---@ 1 root wheel 264 Jul 25 08:21 sysadmin’s MacbookPro.sparsebundle
...
Notice that you don’t actually own the file. (Had I not used the sudo command with ls I could not have listed the contents of /Volumes/TimeMachine)

That .sparsebundle file for your Mac is where all your backup sets live. TimeMachine manages the contents of this file, but doesn’t do anything automatically to reduce its size. Luckily there is another tool for that, but you’ll have to be root to run it:
sudo su -
hdiutil compact /Volumes/TimeMachine/YourBackup.sparsebundle
Sample output:
Starting to compact…
Reclaiming free space…
...................................................
Finishing compaction…
Reclaimed 3.1 GB out of 304.1 GB possible.
That’s it! In this example I reclaimed 3.1GB of actual disk space on my TimeMachine volume. 

The following bash script will remove the oldest backup and reclaim the free space:
COMPUTER_NAME=$(/usr/sbin/scutil --get ComputerName)
NBACKUPS=$(/usr/bin/tmutil listbackups | /usr/bin/grep "$COMPUTER_NAME" | /usr/bin/wc -l)
OLDEST_BACKUP=$(/usr/bin/tmutil listbackups | /usr/bin/grep "$COMPUTER_NAME" | /usr/bin/head -n1)
LATEST_BACKUP=$(/usr/bin/tmutil latestbackup)
echo Latest backup: $LATEST_BACKUP
if [[ -n "$LATEST_BACKUP" && "$LATEST_BACKUP" != "$OLDEST_BACKUP" ]]
then
    echo "$NBACKUPS backups. Delete oldest: ${OLDEST_BACKUP##*/} [y/N]? \c"
    read answer
    case $answer in
Image   y*)
ImageImage  echo Running: /usr/bin/sudo /usr/bin/tmutil delete "$OLDEST_BACKUP"
ImageImage  /usr/bin/sudo time /usr/bin/tmutil delete "$OLDEST_BACKUP"
Image    echo "Do you wish to reclaim the free space now? [y/N]? \c"
Image    read answer
Image    case $answer in
ImageImage    y*)
ImageImageImage     mkdir -p /Volumes/TM
    ImageImageImage mount_afp 'afp://user:pass@afp_server_address/share_name' /Volumes/TM
Image    ImageImage hdiutil compact "/Volumes/TM/$(scutil --get ComputerName).sparsebundle"
Image    ImageImage umount /Volumes/TM/
ImageImage    ;;
ImageImage    *)
ImageImageImageImage echo No change
ImageImage    ;;
        esac
Image   ;;
Image   *)
ImageImage  echo No change
Image   ;;
    esac
else
    echo "No backup available for deletion"
fi
In the script above, don't forget to change the afp URL (afp://user:pass@afp_server_address/share_name) to your own.

Possibly Related Posts

Wednesday, 23 July 2014

Ubuntu as Time Machine server

This guide will help you to install and configure the netatalk servise on an Ubuntu server so it can function as a Time Machine backup server for your Mac OS machines.

First install the necessary packages:
sudo apt-get install netatalk avahi-daemon libnss-mdns
Open the netatalk default configuration file:
sudo vi /etc/default/netatalk
Modify the lines:
ATALKD_RUN=yes
PAPD_RUN=no
CNID_METAD_RUN=yes
AFPD_RUN=yes
TIMELORD_RUN=no
A2BOOT_RUN=no
Edit the atalkd.conf file:
sudo vi /etc/netatalk/atalkd.conf
add to the bottom
eth0
Edit the AppleVolumes.default file:
sudo vi /etc/netatalk/AppleVolumes.default
add to the bottom:
/backups/timemachine "Time Machine" allow:@admin cnidscheme:cdb volsizelimit:200000 options:usedots,upriv,tm
The example above also limits the size shown to OS X as 200 GB (the number is given in MiB, so it's 200,000 times 1024 in the real world)

Edit the afpd configuration file:
sudo vi  /etc/netatalk/afpd.conf
add to the bottom:
- -transall -uamlist uams_dhx.so,uams_dhx2.so -nosavepassword -advertise_ssh -mimicmodel TimeCapsule6,106 -setuplog "default log_warn  /var/log/afpd.log"

Create a configuration file for the avahi afpd discovery:
sudo vi  /etc/avahi/services/afpd.service
and enter the following into it:
<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
 <name replace-wildcards="yes">%h</name>
 <service>
 <type>_afpovertcp._tcp</type>
 <port>548</port>
 </service>
 <service>
 <type>_device-info._tcp</type>
 <port>0</port>
 <txt-record>model=MacPro</txt-record>
 </service>
</service-group> 
Restart the services:
sudo service netatalk restart
sudo service avahi-daemon restart

Possibly Related Posts

Monday, 10 September 2012

MySQL Export to CSV

If you need the data from a table or a query in a CSV fiel so that you can open it on any spreadsheet software, like Excel you can use something like the following:
SELECT id, name, email INTO OUTFILE '/tmp/result.csv'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
ESCAPED BY ‘\\’
LINES TERMINATED BY '\n'
FROM users WHERE 1
Or you can use sed:

mysql -u username -ppassword database -B -e "SELECT * FROM table;" | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > filename.csv
Explanation:

username is your mysql username
password is your mysql password
database is your mysql database
table is the table you want to export

The -B option will delimit the data using tabs and each row will appear on a new line.
The -e option denotes the MySQL command to run, in our case the "SELECT" statement.
The "sed" command used here contains three sed scripts:

s/\t/","/g;s/^/"/ - this will search and replace all occurences of 'tabs' and replace them with a ",".

s/$/"/; - this will place a " at the start of the line.

s/\n//g - this will place a " at the end of the line.

You can find the exported CSV file in the current directory. The name of the file is filename.csv.

However if there are a lot of tables that you need to export, you'll need a script like this:
#!/bin/bash
#### Begin Configuration ####
DB="mydb"
MYSQL_USER="root"
MYSQL_PASSWD='mypass'
MYSQL_HOST="127.0.0.1"
MYSQL_PORT="3306"
MYSQL="/usr/bin/mysql"
#### End Configuration ####
MYSQL_CMD="$MYSQL -u $MYSQL_USER -p$MYSQL_PASSWD -P $MYSQL_PORT -h $MYSQL_HOST"
TABLES=`$MYSQL_CMD --batch -N -D $DB -e "show tables"`
for TABLE in $TABLES
do
SQL="SELECT * FROM $TABLE;"
OUTFILE=$TABLE.csv
$MYSQL_CMD --database=$DB --execute="$SQL" | sed 's/\t/","/g;s/^/"/;s/$/"/;s/\n//g' > $OUTFILE
done
Just be sure to change the configuration section to meet your needs.
Name the file something like: export_csv.sh and be sure to make it executable. In Linux, do something like:

chmod +x ./export_csv.sh
If you want to have all of the exported files in a certain directory, you could either modify the script or just make the cirectory, "cd" into it, and then run the script. It assumes you want to create the files in the current working directory.
To change that behavior, you could easily modify the "OUTFILE" variable to something like:
OUTFILE="/my_path/$TABLE.csv"

Possibly Related Posts

Thursday, 22 September 2011

Backuppc and MySQL

The best way to backup a MySql Server using Backuppc is to use a pre-dump script.

you can use $Conf{DumpPreUserCmd} to issue a MysqLDump

Stdout from these commands will be written to the Xfer (or Restore) log file, note that all Cmds are executed directly without a shell, so the prog name needs to be a full path and you can't include shell syntax like redirection and pipes; put that in a script if you need it.

So in our case we would create a script, on the Backuppc client, to dump all databases into a file:
vi /usr/local/sbin/myBkp.sh
and paste the following into it:
#!/bin/bash
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"
DEST="/backup/mysqlDump.sql"
MYSQLUSER="root"
MYSQLPASS="mypassword"
# no need to change anything below...
#####################################################
LOCKFILE=/tmp/myBkup.lock
if [ -f $LOCKFILE ]; then
echo "Lockfile $LOCKFILE exists, exiting!"
exit 1
fi
touch $LOCKFILE
echo "== MySQL Dump Starting $(date) =="
$MYSQLDUMP --single-transaction --user=${MYSQLUSER} --password="${MYSQLPASS}" -A > ${DEST}
echo "== MySQL Dump Ended $(date) =="
rm $LOCKFILE
make the script executable:
chmod +x /usr/local/sbin/myBkp.sh 
and set $Conf{DumpPreUserCmd} with:
$sshPath -q -x -l root $host /usr/local/sbin/myBkp.sh
Now you just have to make shure that Backuppc is getting the /backup folder (or whatever folder you have set in the script) and you can also exclude the /var/lib/mysql folder from backuppc backups.

Possibly Related Posts

Backuppc Got unknown type errors

Type 8 are socket files and type 9 is unknown (Solaris door files).
These are transient files that don't need to be backed up since they can't be restored - they are created by the applications that need them.

The warning messages are benign - they simply mean that those files are being skipped.

Possibly Related Posts

watch backuppc progress


From the Backuppc server you can check witch files backuppc is using at the moment with:
watch "lsof -n -u backuppc | egrep ' (REG|DIR) ' | egrep -v '( (mem|txt|cwd|rtd) |/LOG)' | awk '{print $9}'"
you can also check the running log with:
/usr/share/backuppc/bin/BackupPC_zcat/var/lib/backuppc/pc/desktop2/XferLOG.z
On the client side you can use:
watch "lsof -n | grep rsync | egrep ' (REG|DIR) ' | egrep -v '( (mem|txt|cwd|rtd) |/LOG)' | awk '{print $9}'"

Possibly Related Posts

Friday, 8 July 2011

Set Xenserver VMs Custom Fields with a script

On a previous post I've shown you a script that I use to backup my virtual machines on a XenServer Pool, but I have a lot of VMs so it's not easy to set the custom fields for every VM. So I've made another script that allows you set the custom fields on every VM or in a group of VMs using the tags from XenCenter.

you can use the script like this:

setCustomFields.sh [-t tag] [<template_frequency> <template_retention> <xva_frequency> <xva_retention>]

if you omit the -t flag it sets the custom fields for all VMs

And the script goes like this:


#!/bin/bash
#
# Variables
#

TAG="all"


LOCKFILE=/tmp/snapback.lock

if [ -f $LOCKFILE ]; then
echo "Lockfile $LOCKFILE exists, exiting!"
exit 1
fi

touch $LOCKFILE

#
# Don't modify below this line
#

# using getopts to parse arguments
while getopts 't:' OPTION
do
case $OPTION in
t) TAG="$OPTARG"
;;
?) printf "Usage: %s: [-t tag] [<template_frequency> <template_retention> <xva_frequency> <xva_retention>]\n" $(basename $0) >&2
exit 2
;;
esac
done
shift $(($OPTIND - 1))

TEMPLATE_BACKUP=${1:-weekly}
TEMPLATE_KEEP=${2:-1}

XVA_BACKUP=${3:-weekly}
XVA_KEEP=${4:-1}

# Quick hack to grab the required paramater from the output of the xe command
function xe_param()
{
PARAM=$1
while read DATA; do
LINE=$(echo $DATA | egrep "$PARAM")
if [ $? -eq 0 ]; then
echo "$LINE" | awk 'BEGIN{FS=": "}{print $2}'
fi
done
}

# Get all running VMs
RUNNING_VMS=$(xe vm-list power-state=running is-control-domain=false | xe_param uuid)

for VM in $RUNNING_VMS; do
VM_NAME="$(xe vm-list uuid=$VM | xe_param name-label)"

echo " "
echo "= Retrieving backup paramaters for $VM_NAME - $(date) ="
#echo "= $VM_NAME uuid is $VM ="
#Template backups
SCHEDULE=$(xe vm-param-get uuid=$VM param-name=other-config param-key=XenCenter.CustomFields.backup)
RETAIN=$(xe vm-param-get uuid=$VM param-name=other-config param-key=XenCenter.CustomFields.retain)
#XVA Backups
XVA_SCHEDULE=$(xe vm-param-get uuid=$VM param-name=other-config param-key=XenCenter.CustomFields.xva_backup)
XVA_RETAIN=$(xe vm-param-get uuid=$VM param-name=other-config param-key=XenCenter.CustomFields.xva_retain)

if [[ $TAG == "all" ]]
then
VM_TAGS=$TAG
else
VM_TAGS=$(xe vm-param-get uuid=$VM param-name=tags)
fi

if [[ $VM_TAGS == *$TAG* ]]
then

if [ "$SCHEDULE" != "$TEMPLATE_BACKUP" ]; then
echo "Updating template backup schedule..."
xe vm-param-set uuid=$VM other-config:XenCenter.CustomFields.backup="$TEMPLATE_BACKUP"
fi

if [ "$RETAIN" != "$TEMPLATE_KEEP" ]; then
echo "Updating template backup retention..."
xe vm-param-set uuid=$VM other-config:XenCenter.CustomFields.retain="$TEMPLATE_KEEP"
fi

if [ "$XVA_SCHEDULE" != "$XVA_BACKUP" ]; then
echo "Updating XVA backup schedule..."
xe vm-param-set uuid=$VM other-config:XenCenter.CustomFields.xva_backup="$XVA_BACKUP"
fi

if [ "$XVA_RETAIN" != "$XVA_KEEP" ]; then
echo "Updating template XVA retention..."
xe vm-param-set uuid=$VM other-config:XenCenter.CustomFields.xva_retain="$XVA_KEEP"
fi

fi

done

rm $LOCKFILE

Possibly Related Posts

Friday, 3 June 2011

Xenserver Backup solution

After reviewing some of the available backup solutions (see my previous post) I've opted to use the script provided by Mark Round but I've modified it a bit to allow backing up the VMs to .xva files with an independent schedule from the template backups, I use this to store xva files of my VMs in tapes monthly.

To use it you just have to place the script in /usr/local/bin/snapback.sh in your pool master host and add a line to /etc/crontab:
2 1 * * * root /usr/local/bin/snapback.sh > /var/log/snapback.log 2>&1
Then you can configure the scheduling and retention for each VM, from the XenCenter, by adding the following costume fields:
  • backup - to set the schedule for the template backups (daily, monthly or weekly)
  • retain - to set the number of backup templates to keep
  • xva_backup - to set the schedule for the xva backups (daily, monthly or weekly)
  • xva_retain - to set the number of xva files to keep
Click in the "read more" link to see the script.

Possibly Related Posts

Saturday, 28 May 2011

Xenserver Backup solutions

There are a lot of different backup solutions available to backup your VMs. So I've compiled a comparative list of some of the available solutions, and in my next post I will tell you about the one I chose for my backups.

Generic Backup Solutions:
Backuppc:http://backuppc.sourceforge.net/
  • No agent needed, it uses rsync, rsyncd or smb
  • File level deduplication across all backups
  • Incremental and full backup scheduling
  • Easy to setup
  • Web interface
  • Command Line Interface
  • No support for tape drives
  • disk-based data backup and recovery
  • Free and Open Source
  • No downtime
Bacula:http://www.bacula.org/
  • Free and Open Source
  • No downtime
  • Support for tape drives and tape libraries
  • Uses agents
  • Web interface
  • wxWidgets interface
  • Command Line Interface
  • can encrypt data in transit
  • supports snapshots via Windows VSS
  • Data backed up by Bacula must be recovered by Bacula
  • user postings online indicate that it can be quite complex to set up
Amanda:http://www.amanda.org/
  • Free and Open Source
  • No downtime
  • Free and Open Source
  • can back up Linux, Unix, Mac and Windows clients to tape, disk and storage grids, such as Amazon Simple Storage Service (S3)
  • Write to Multiple Volumes in Parallel
  • Support for tape drives and tape libraries
  • Virtual tapes
  • use of native tools, such as ufsdump, dump and/or GNU tar
  • Ability to read the backup tapes without Amanda
  • Uses agents
  • Commercial version named Zamanda with added features such as:
    • web-based GUI
    • management console
    • one-click restore and reporting application agents (priced additionally) for Microsoft Exchange, SQL Server and SharePoint, and Oracle
    • 24/7 customer support
    • orderly new feature release schedule
  • Commercial Version Pricing (http://network.zmanda.com/shop/home.php?cat=1,3):
    • Basic:
      • Server $400
      • Linux, Solaris and Windows Cients $150
      • Windows Clients for desktops and laptops $200
      • Backup to S3 option $250
    • Standard:
      • Server $500
      • Linux, Solaris and Windows Cients $300
      • Windows Clients for desktops and laptops $300
      • Backup to S3 option $500
      • Oracle agent $300
      • Postgres agent $300
      • VMWare vSphere and ESXi client $300
    • Premium:
      • Server $750
      • Linux, Solaris and Windows Cients $450
      • Windows Clients for desktops and laptops $450
      • Backup to S3 option $750
      • Oracle agent $300
      • Postgres agent $450
      • VMWare vSphere and ESXi client $450
Acronis Backup:http://www.acronis.eu/
  • Uses agents
  • Server runs on Windows
  • Supports Tape drives and tape autoloaders
  • Compress backups to optimize your storage space.
  • Save storage space and time by excluding non-essential files and folders from backups.
  • Store backups into two different locations — backup to a local disk and a copy to a network share.
  • Automatic or manual splitting of backups
  • Bare-metal restore
  • Perform remote restores of your networked machines
  • Restore to dissimilar hardware-optional
  • Convert backup images to virtual machine formats compatible with VMware, Microsoft Hyper-V, Citrix XenServer and Parallels environments.
  • install Agent on unlimited number of virtual machines
  • Automated deletion of outdated backups
  • Backup validation and consolidation by Acronis Storage Node
  • Consolidate incremental and differential backups to save space (Deduplication).
  • Templates for backup rotation schemes
  • Centralized management
  • Reporting and monitoring
  • Command line with scripting support
  • Encrypted network communications
  • Costs 1784€ per license
Xen specific Backup Solutions:
Manual Snapshots:
XenServer supports three types of VM snapshots: regular, quiesced and snapshot with memory. Regular snapshots are crash consistent and can be performed on all VM types. The VM snapshot contains all the storage information and VM configuration, including attached VIFs, allowing them to be exported and restored for backup purposes.

Quiesced snapshots take advantage of the Windows Volume Shadow Copy Service (VSS) to generate application consistent point-in-time snapshots. The VSS framework helps VSS-aware applications like Microsoft Exchange or Microsoft SQL Server to flush data to disk and prepare for the snapshot before it is taken. XenServer supports quiesced snapshots on Windows Server 2003 and Windows Server 2008 for both 32-bit and 64-bit variants. Windows 2000, Windows XP and Windows Vista are not supported.

Snapshot with memory save the VMs state (RAM).This can be useful if you are upgrading or patching software, or want to test a new application, but also want the option to be able to get back to the current, pre-change state (RAM) of the VM. Reverting back to a snapshot with memory, does not require a reboot of the VM.

Backup across multiple external disks:http://couleetechlink.com/xenbackup.php
  • back up to muliple esata hard drive's so if you have a 1.5 tb image and only 2 1tb esata hard drives you can span it between both.
  • This will work with any number of drive’s as long as the total combined disk space is larger then the vm’s you are trying to back up.
  • This also works if you only have one drive
  • The backup will be just a little larger then the used space on the drive, so even if you have a 2tb virtual drive but it is only using 500gb it should fit on a drive with 600gb or more freespace.
  • need to manualy create a list of VMs to backup
Zero-Downtime Limited-Space Backup Script:
http://community.citrix.com/display/xs/Zero-Downtime+Limited-Space+Backup
http://community.citrix.com/display/xs/Zero-Downtime+Limited-Space+Backup+and+Restore
  • Currently, Windows servers are not supported, only Linux VMs and the XenServer instance, itself.
  • based on using the python API and command-line LVM snapshots
  • No downtime
  • Free and Open Source
  • Limited space - "Doing built-in snapshots of VM's was not feasible for us. Currently, there is no way to exclude disks in a snapshot (that we have found). A snapshot will take about double the currently used space for a disk on an SR, and this space cannot be reclaimed until the snapshot is deleted and the machine is shutdown to be coalesced. In one of our VMs we have about 8 TB of user drive space, with no extra space on the SRs where the disks are allocated. We don't have enough room, nor do we care about creating a snapshot with the user data since it is already backed up with netbackup. The script allows us to get no-downtime snapshots of the system disks with only requiring a small and temporary amount of extra space on the SRs".
  • The python API is used to gather metadata about the VM, its disks, and its network interfaces. The metadata is written to plain text files. The data from the disks is imaged by doing a dd on the lvm volumes that correspond to the VDIs for the disks.
  • To restore, a new VM is created and given the memory and CPUs settings stored in the metadata. Then the VIF and disks are restored with the stored images being written to the new lvm volumes.
  • The script is still a work in progress
  • Support for Windows will be added
TINABS (This Is Not Another Backup Script):
http://code.google.com/p/xenserver-vms-live-backup/
  • It is based on using the python API and tested under XenServer 5.6 FP1
  • This library allows you to create simple scripts to backup the skeleton of one or more virtual machines.
  • Data disks are not included and they are recreated empty
  • The core of the library is the backup() function which iterates through a list of user supplied virtual machine and:
    • gets a snapshot of the system disk, attach it to a brand new virtual machine created based on the parameters of the current one in the list,
    • recreates any data disks on a shared SR (I preferly use an NFS SR as destination due to the fact thatt “For file-based VHDs, all nodes consume only as much data as has been written, and the leaf node files grow to accommodate data as it is actively written. If a 100GB VDI is allocated for a new VM and an OS is installed, the VDI file will physically be only the size of the OS data that has been written to the disk, plus some minor metadata overhead” as stated in XenServer Administrator's Guide) and attaches them to the backup one,
    • recreates any VIFs of the original virtual machine and attaches them to the backup one,
    • exports the backup virtual machine in .xva format on a local directory,
    • completely deletes the backup virtual machine.
  • The restoring process simply consists in importing the .xva previously created and restoring any data from a backup!
  • Live backup and export of virtual machines.
  • I don't care about creating a snapshot of the entire virtual machine including even any data disks since their data are already backed up with a backup tool.
  • Run from a remote host (even a Windows machine)
  • Provide a simple GUI (WxPython + XRC)
  • By default (if running through the GUI) all pool's virtual machines tagged with the current day of the week (in the format: Mon, Tue, Wed, Thu, Fri, Sat, Sun) are selected for backup.
  • A single virtual machine can be selected as well
NOTE: In some cases useing the following scritps the disk space won't be freed after deleting the snapshots, if that happens follow this instructions:
http://support.citrix.com/article/CTX123400 but according to http://support.citrix.com/article/CTX127362 this problem is solved in XenServer 5.6 FP1

XenServer Live Backup Script:
http://community.spiceworks.com/scripts/show/161-xenserver-live-backup?page=2

  • runs on windows
  • written in VBScript
  • Requires that XenCenter be installed on Windows machine you run the script from.
  • Beta Stage
Filippo Zanardo's Xenbackup Script:http://pipposan.wordpress.com/2010/06/16/xenserver-final-backup-script-with-or-without-snapshot/http://code.google.com/p/xenbackup/
  • written in Perl
  • skip VMs by adding them to a list
  • Optional use of snapshot: if set to true backup script try to make a snapshot of the vm, else he shutdown the machine, export and power on the machine
  • Mail Notification
  • Optionaly create a subfolder in the store for each backup based on vm name
  • versioning: Set to true to let the script manage to delete the backup older than a certain day or number or hours specified in the $delnumber variable
  • automount: if set to true script try to mount the backupdir specified in mountcommand at start and umount at end, else no action taken and u have to mount dir manually
  • checkspace: if set to true the script check the avaiable space on the backup dir and if less than $spacerequired quit with a message, size is in MB
  • Free and Open Source
  • the author is also working on a web based Xen backup solution (http://pipposan.wordpress.com/2010/12/28/web-xen-server-backup/)
Andy Burton's VM export script:
http://www.andy-burton.co.uk/blog/index.php/2009-08/citrix-xenserver-automated-live-vm-backup-to-windows-cifs-share/
or
http://www.andy-burton.co.uk/blog/index.php/2009-11/updated-citrix-xenserver-5-5-automatic-vm-backup-scripts/
  • Backup of the entire machine
  • Fast recovery in case of disaster
  • Free and Open Source
  • No downtime
  • VDI removal – Run in addition to the standard vm-uninstall command to stop snapshotted VMs allocating all your disk space
  • Backup VM selection – Select between all, running, none and specifically set (by VM uuid) virtual machines.
  • Quiesce snapshots – To take advantage of the quiesce snapshot functionality if the VM supports it.
  • There is an improved version on http://www.8layer8.com/xen/ (posted in: http://www.8layer8.com/?p=260) this version adds:
    • some cleanup scripts to handle disk remounts, removal of older backup images, and some logic to not back up if the backup drive is not present and mounted
    • A plaintext dump of all the info needed to figure out what used to be connected to what and where it used to live, all the SR, VM, VIF, UUID's etc. are here in a reasonably readable format if needed.
    • A sctipt that unmounts and remounts the backup disk, and then cleans it up so that we only have the last two backups on it. Needs some logic to abort if the drive isn't, or can't be, mounted.
    • A script to back up the metadata of the Xen Pool in a restorable format. Backs up the host machines over to the backup drive as well.
  • Back up all the VM's, Xen hosts, and metadata from a single Xen host, so you only need to set this up on one machine
  • Backup destination can be an NFS share, SMB share, USB disk, flash drive, or anything else you can get mounted up.
Markround Script:
http://www.markround.com/archives/61-Xenserver-snapshot-and-template-based-backup-script.html
  • Similar to the previous but more complete
  • Backup and retention policy can be configured from XenCenter
  • Ability to use a different SR for the backups
Alike:http://quorumsoft.com/learnmore.php
  • Agentless backup for XenServer
  • Comes in 3 versions, free, Standard ($899/XenServer Host) and DR ($1189/XenServer Host)
  • Volume licensing
  • Block-level data deduplication across all VMs backed up
  • Friendly UI
  • Versions each snapshot that is backed up
  • Alike is able to backup any or all of the drives in any VM
  • Jobs can be scheduled daily, weekly, or monthly; may be configured for multiple runs per day
  • Alike can run on 64-bit Windows, can back up any guest OS
  • Backup to Any Common Storage Type
  • Alike can fully automate and schedule Citrix's Coalesce tool, dramatically simplifying the reclaim process.
  • Alike can schedule the export, migration or replication of your VMs, providing simple offsite support.
  • Alike installs nothing on the XenServer host operating system (Dom0), and does not require disk from XenServer Storage Repository (SR).
PHD backup:
http://www.phdvirtual.com/server_virtualization_citrix_xenserver
  • Block level deduplication
  • No downtime
  • Backups saved as VHD
  • File level recovery Any OS, Any File System
  • Removes the need to deploy and manage a separate physical server, additional software, scripts or agents for backup and recovery of the virtual environment
  • Simple to Deploy & Easy to Use
  • Integrate management for backup and recovery into XenCenter
  • Data is checked both during the backup and restore processes ensuring data integrity. Self-healing is provided by automatically detecting and repairing corrupt data blocks.
  • Multiple Data Streams for Fast Backup and Restore
  • Job Scheduling
  • Supports Tape Backup Solutions
  • Application Consistent Backup using VSS
  • E-mail notification
  • Support for Thin Provisioned Disks
  • Backup Storage Warnings
  • Distributed Virtual Switch Support
  • Supports all Operating Systems supported by XenServer
  • Licence per host
  • we own the licences with an optional annual subscription.
  • 1395$ until the end of the month, regualrly 2000$ per server. with one year suppor included (email and phone suppot, updates and patches)
  • 280$ per host for the annual subscription 11/5 EST working hours
  • Resellers in Portugal
  • 15 day trial

Possibly Related Posts