Filippo Zanardo's Blog

to blog or not to blog

Archive for the ‘XenServer’ Category

Xenserver Backup Updated 2.2

with 31 comments


Ok guys,
i have some time and i have updated my xen backup script, to improve it and to add some few features.
Basically i add the possibility to compress the backup and the possibility to have more than 1 removable storage, the line changed in the config.txt file are:

#the uudi of the removable storage, at the moment i don’t have an efficent script to localize since name can change
@removableuuid = (‘d7a8fb07-e7d1-9eb4-6d79-dca8680ace89′,’d7a8fb07-e7d1-9eb4-6d79-dca8680ace86’);

#compress option enable/disable compression
$compress = true;

#compress command
$compresscmd = “tar -czvf”;

#compress extension easiser for me to put correct extension
$compressext = “.tar.gz”;

Few other check and modification, the rest is the same.
You can downlaod the new script at:

http://xenbackup.googlecode.com/files/xenbackup2.2.zip

Please comment for feature, bug and so on.

Enjoy! Cheers!

Web Xen Server Backup

with 9 comments


Ok guys,
finally i have some time to write what i have in mind, i’m starting a new project about xenserver backup.
The goal is to make a web interface for the backup and eventually for the restore of the vm.
Some feature i want to add:

  • Multiuser
  • ACL based
  • XMLRPC based
  • Scheduled backup
  • Extended notification
  • Multilanguage
  • Metadata backup
  • Data deduplication with lessfs or opendedup
  • Possibility to manage and take backup on every kind of storage (NAS,USB disk etc etc)
  • XVA can be downloaded everywhere
  • Cool interface
  • the interface can be installed everywhere as long the machine can communicate with xenserver

Ok this some of the feature i have in mind, more will come surely.
I will develop the project in PHP+Mysql+Js.

What do you think about?
What feature you want to be added?

Please comment so we can make some brainstorming and you can give me great idea, and of course if someone wanna help me or partecipate in this project it’s appreciated.

Cheers

Written by pipposan

December 28, 2010 at 12:02

XenServer Final Backup Script with or without snapshot

with 62 comments


Ok guys,
Xenserver 5.6 is out now, and i enhance my backup script i rewrite some part and add some more features/controls, that i will explain later.

UPDATED: 2.2 is now avaiable i rewrite the check avaiable space to fix a minor bug

You can donwload the 2.1 script here

http://xenbackup.googlecode.com/files/xenbackup2.1.zip

It’s simple to set up just modify config.txt and than you can run the script with this simple command line:

perl /path/to/backup.pl /path/to/config.txt

Now i will explain what you can configure and the features:

@skip: this variable contain the uuid of the machine to skip in the backup you can get with the command ‘xe vm-list’, add more uuid separeted by ‘,’

usesnapshot: if set to true backup script try to make a snapshot of the vm, else he shutdown the machine, export and power on the machine

removable: if set to true the script detach the removable storage, else snapshot fail, and then reattached in the correct order, just remember to set up the variable removableuuid with your removable uuid you can get with command ‘xe sr-list’

mailNotification: if set to true script send email to and from the email specified in the config file, i remove the script used before now script use ssmtp that is already installed in xenserver, just configure the file /etc/ssmtp/ssmtp.conf with your data and remember to uncomment this line FromLineOverride=YES

subfolder: Set to true to create a subfolder in the store for each backup based on vm name

versioning: Set to true to let the script manage to delete the backup older than a certain day or number or hours specified in the $delnumber variable

logging: if set to true the script write all the message with data into one log/file specified in the variable $logfile

automount: if set to true script try to mount the backupdir specified in mountcommand at start and umount at end, else no action taken and u have to mount dir manually

checkspace: if set to true the script check the avaiable space on the backup dir and if less than $spacerequired quit with a message, size is in MB

i add also some check to verify that the correct file is been written on the backup folder, if not backup error is notified.

That’s all i hope i explain well and you like the script, comment or post for bugs, feature etc etc.

Cheers!

Nagios Plugin To Check XenServer License

with 4 comments


Hi guys,
since xenserver license is active for 1 year, happens sometimes that i forgot to reactivate the license, and then if u shut down the vm machine they don’t start up if license is expired, i create a nagios plugin to check the days remaining for the license.

here’s the script updated thanks to Martin:

#!/bin/bash
#
# retrieve remaining day for xenserver license
#
# Date: 2010-04-15
# Author: Filippo Zanardo
#

# get arguments

while getopts ‘w:c:hp’ OPT; do
case $OPT in
w) int_warn=$OPTARG;;
c) int_crit=$OPTARG;;
h) hlp=”yes”;;
*) unknown=”yes”;;
esac
done

# usage
HELP=”
usage: $0 [ -w value -c value -h ]

syntax:

-w –> Warning integer value
-c –> Critical integer value
-h –> print this help screen

if [ “$hlp” = “yes” -o $# -lt 1 ]; then
echo “$HELP”
exit 0
fi

# get license expiry date
EX=`xe host-license-view |grep expiry | awk ‘{ print $2; }’`
DEX=${EX:0:8}

today=`date +%Y%m%d`
timeStampToday=`date +%s -d $today`
timeStampOfPast=`date +%s -d $DEX`
secondsInDay=86400
dayDiff=`echo \($timeStampOfPast – $timeStampToday\) / $secondsInDay – 1 | bc`

#dayDiff is the day remaining

OUTPUT=”$dayDiff days remaining”

if [ -n “$int_warn” -a -n “$int_crit” ]; then

err=0

if (( $dayDiff <= $int_warn )); then
err=1
elif (( $dayDiff <= $int_crit )); then
err=2
fi

if (( $err == 0 )); then

echo -n "OK – $OUTPUT"
exit "$err"

elif (( $err == 1 )); then
echo -n "WARNING – $OUTPUT"
exit "$err"

elif (( $err == 2 )); then

echo -n "CRITICAL – $OUTPUT"
exit "$err"

fi

else

echo -n "no output from plugin"
exit 3

fi
exit

just copy it save with some name for example check_xenlicense.sh in the nagios plugins directory and configure to use for example with nrpe.
basic usage is:

check_xenlicense.sh -w 15 -c 5

where “w” is the days remaining for a warning message and “c” is the days remaining for a critical message

if you don’t use nagios you can use as a simple script in a cron job or whatever.

Maybe if someone find this useful i will add the ability to send an email notification.

Cheers
Post for suggestion comment etc etc

Written by pipposan

April 15, 2010 at 12:11

XenServer Backup with Bacula VM director and web interface

with 2 comments


Ok guys now it’s time to put the director on a virtual machine, usually i use debian but u can use whatever distro you like, with mysql-server and apache2 installed.

It’s simply in xenserver create a Debian vm as long you install also linux pack, just make new vm from debian template 🙂

Next install some packages needed for bacula and bweb (web interface that i use with bacula).
Run this command:

apt-get install apache2 mysql-server mysql-client libmysqlclient15-dev php5-mysql php5-mysql

This packages install mysql-server apache2 and php5, make sure you have make and some compiler cause i will compile bacula, or simply install this:

apt-get install build-essential

ok now we have quiet done with prerequisites, we can go to the bacula sourceforge page and copy the direct link from download page, and execute:

wget http://downloads.sourceforge.net/project/bacula/bacula/5.0.0/bacula-5.0.0.tar.gz?use_mirror=sunet

also donwload bacula gui:

wget http://downloads.sourceforge.net/project/bacula/bacula/5.0.0/bacula-gui-5.0.0.tar.gz?use_mirror=sunet

ok now extract bacula with the command:

tar -xzvf bacula-5.0.0.tar.gz

enter the bacula folder and configure for compiling using:

./configure –sysconfdir=/etc/bacula –with-scriptdir=/etc/bacula/scripts –with-smtp-host=localhost –with-mysql –with-tcp-wrappers –enable-smartalloc

after configuration’s finished launch this command:

make
make install

Compile the information needed by bacula database prompted and we are ready.

u can check the state of bacula using bconsole use the command:

/etc/bacula/bconsole

ok now go back 1 folder and extract bacula gui, go into this folder/bweb, and run this command needed by bweb, or check INSTALL file:

apt-get install libgd-graph-perl libhtml-template-perl libexpect-perl \
libdbd-mysql-perl libdbd-pg-perl libdbi-perl \
libdate-calc-perl libtime-modules-perl

now install with this command:

perl Makefile.PL
make install

copy the bweb perl program to your cgi location

mkdir -m 755 /usr/lib/cgi-bin/bweb
install -m 755 -o root -g root bweb/cgi/*.pl /usr/lib/cgi-bin/bweb

get a config file, if you want to use an other place edit lib/Bweb.pm

echo ‘$VAR1 = { template_dir => “/usr/share/bweb/tpl” };’ > /etc/bacula/bweb.conf
chown www-data /etc/bacula/bweb.conf

copy the bweb template file

mkdir -p /usr/share/bweb/tpl/en
install -m 644 -o root -g root bweb/lang/en/tpl/*.tpl /usr/share/bweb/tpl/en

copy the bweb graphics elements (bweb elements must reside in /bweb)

mkdir /var/www/bweb
install -m 644 -o root -g root bweb/html/*.{js,png,css,gif,ico,html} /var/www/bweb

done, reload your apachewith ” /etc/init.d/apache2 reload” and you can finally browse at http://yourip/bweb
and insert information needed by bweb for the configuration file.

Ok now part one is done, we only need to configure our director to talk with the other daemon, and the one we installed on xen server host machine, open the file /etc/bacula/bacula-dir.conf and add this line for the storage daemon:

# Definition of DDS tape storage device
Storage {
Name = Tape
Address = x.x.x.x # N.B. Use a fully qualified name here
SDPort = 9103
Password = “yourpassword” # password for Storage daemon
Device = ULTRIUM # must be same as Device in Storage daemon
Media Type = ULTRIUM # must be same as MediaType in Storage daemon
}

remember to put the same name device, and media type specified in bacula-sd.conf in the xenserver host machine.

For the file daemon:

Client {
Name = f10-64-build-fd
Address = 10.0.0.14
FDPort = 9102
Catalog = MyCatalog
Password = “pippo” # password for FileDaemon
File Retention = 30 days # 30 days
Job Retention = 6 months # six months
AutoPrune = yes # Prune expired Jobs/Files
}

Name must be the same that the one specified in bacula-fd.conf on xenserver host machine.

Restart bacula-director with “/etc/bacula/bacula restart” You can check the status of the connection using bconsole (/etc/bacula/bconsole) type st in the console and type 4 for the status of all components, if no error message show you configure well and u can begin to create a pool and a backup job.

Read the documentation about bacula for setting up the job, or maybe later i will blog about that.

Comments for help!
Cheers

Written by pipposan

February 16, 2010 at 17:57

XenServer 5.5 Tape Backup with Bacula

with 8 comments


Usually some of my customers use tape backup and i don’t want to use the iscsi trick, so i choose Bacula as my backup solution, i use it for about 2 or 3 years and it’s simple and stable and it works for me.

Basically you have to install some of the dependencies to be able to install Bacula, without compiling, first in your file:

/etc/yum.repos.d/Centos-Base.repo

enable all the respository setting enabled=1.
Next install this package with this command:

yum install qt4 qt4-devel qwt qwt-devel mtx mt-st mysql-server

Now start mysql:

service mysqld start

Ok now we have installed dependecies needed by bacula, now go to Sourceforge Bacula page and search for rpms-contrib-fschwarz i use bacula-mysql-3.0.2-1.el5.i386.rpm, go to download and copy the direct link for example:

http://downloads.sourceforge.net/project/bacula/rpms-contrib-fschwarz/3.0.2/bacula-mysql-3.0.2-1.el5.i386.rpm?use_mirror=freefr

Log in on ssh to the xenserver and execute this command:

wget http://downloads.sourceforge.net/project/bacula/rpms-contrib-fschwarz/3.0.2/bacula-mysql-3.0.2-1.el5.i386.rpm?use_mirror=freefr

Now in the current folder u download the bacula’s rpm, install with

rpm -ivh bacula…..

ok now bacula is installed, start the daemon and enable on startup:

service bacula-sd start
service bacula-fd start
chkconfig bacula-sd on
chkconfig bacula-fd on

Xenserver block port needed by bacula so stop iptables or execute this command:

iptables -A INPUT -p tcp –dport 9102 -j ACCEPT
iptables -A OUTPUT -p tcp –sport 9102 -j ACCEPT
iptables -A INPUT -p tcp –dport 9103 -j ACCEPT
iptables -A OUTPUT -p tcp –sport 9103 -j ACCEPT

usually i use director on a vm with webserver enabled so i can check the backup via bweb, so enable and start the director is not necessary. Now you have to modify the bacula’s config file now i put some example, modify to suite your needs.

File /etc/bacula/bacula-sd.conf

Storage { # definition of myself
Name = f10-64-build-sd
SDPort = 9103 # Director’s port
WorkingDirectory = “/var/lib/bacula”
Pid Directory = “/var/run”
Maximum Concurrent Jobs = 20
}

#
# List Directors who are permitted to contact Storage daemon
#
Director {
Name = yourdirector-name
Password = “yourpassword”
}

Device {
Name = ULTRIUM #
Media Type = ULTRIUM
Archive Device = /dev/nst0 #the device where tape is attacched usually /dev/st0
AutomaticMount = yes; # when device opened, read it
AlwaysOpen = yes;
RemovableMedia = yes;
RandomAccess = no;
}

file /etc/bacula/bacula-fd.conf

#
# Restricted Director, used by tray-monitor to get the
# status of the file daemon
#
Director {
Name = yourdirector-name
Password = “yourpassword”
}

#
# “Global” File daemon configuration specifications
#
FileDaemon { # this is me
Name = f10-64-build-fd
FDport = 9102 # where we listen for the director
WorkingDirectory = /var/lib/bacula
Pid Directory = /var/run
Maximum Concurrent Jobs = 20
}

The basic configuration for the storage daemon and the file daemon is done, now add them to your director and create a backup job.

Later i will post some about the director’s configuration and how to install the web interface (bweb).

If you have some question, comments or ask me.
Cheers!

Written by pipposan

February 4, 2010 at 19:02

0 downtime snapshot based XENServer 5.5 backup script

with 19 comments


DEPRECATED
Check this new better version of the script


Ok guys based on this script i made mine with some more features.
Basically i split the configuration file out of the script and added some new features:

automount: if set to true script try to mount the backupdir specified in mountcommand at start and umount at end, else no action taken and u have to mount dir manually

subfolder: Set to true to create a subfolder in the store for each backup based on vm name

versioning: Set to true to let the script manage to delete the backup older than a certain day or number or hours

and i rewrite the message so they are now more “human” readeable.

The usage it’s the same except u have to pass the path to the config file to the script like:

perl /path/to/backup.pl /path/to/config.txt

U can download the file here

http://xenbackup.googlecode.com/files/xenbackup.zip

Unzip in a folder and read and modify the configuration file (basic explanation is written in), i put also in the config file message’s string so it’s easy to translate in your preferred language, i added also the elapsed minute time of each backup.

Next remember to modify the SendEmail script changing the “server” line at line 65, to send notification mail.

## Network
“server” =>’mysmtp.server.com’,
“port” => 25, ## Default port
“alarm” => ”, ## Default timeout for connects and reads, this gets set from $opt{‘timeout’}

and at line 54 hostname:

“hostname” => ‘backup.csinet.ca’, ## Used in printmsg() for all output, and in SMTP EHLO.

Next call in a cron job or manually the script and enjoy.

Remember to put in the skip variable the uuid of the control domain on host and other vm that don’t need to backed up!

Don’t remove the “\” near @ cause it’s needed for the sendEmail script, so the string should looks like “mymail\@mydomain.com”

Comments for bug, wish feature etc etc…

Based on todo list on the main page i will add other feature when i have time!

Cheers!
Filippo

DEPRECATED
Check this new better version of the script

Written by pipposan

December 9, 2009 at 17:38