'PACC: Week I'

Comments

Overview:

1) Started research with working on iTALC installation, for monitoring lab computers. Found the alternative program called "epoptes" http://www.epoptes.org;

Here is the official installation guide http://www.epoptes.org/installation

However I have written some additional guide, which were based on the one from russian forums (Here the link http://forum.ubuntu.ru/index.php?topic=206483.0)

Remark

It is better to install the server part make a reboot and then install clients parts.

For __ubuntu

Server computer

sudo -i

apt-get install epoptes

gpasswd -a username epoptes (change "username" )

reboot

Client computer

sudo -i

sudo nano /etc/hosts

In open window add

192.168.1.3 SERVER

Here the IP address is the address of your SERVER computer

apt-get install epoptes-client

epoptes-client -c

_reboot _

Then launch the epoptes program on server computer, everything should be working.

2) Installed the WebVirtMgr with one of the VM inside, used the Ubuntu image to run, but unsuccessful, probably because of the broken .iso file or . Here are the tutorials that I used to establish the server host and add users to it for creating pools of storage and virtual machines:

https://github.com/retspen/webvirtmgr/blob/master/README.md

http://lost-and-found-narihiro.blogspot.com/2012/11/install-webvirtmgr-on-ubuntu-1210-and.html

http://www.cnblogs.com/panblack/archive/2013/03/04/webvirtmgr-a-kvm-web-management-tool.html

My first host server was done on my laptop: the address is http://192.168.1.8:16509/

There is one connection for user-> student(by using the KVM login and password)

Also tried to connect two computers with different approaches using the created virtual network, computer nodes became connected but the control was not allowed, probably because of the firewall or bad parameters settings.

By using the SSH I have established the remote conenction between student lab computers and own laptop. What I've got is the working test server now it is http://0.0.0.0:16510 there I have 2 Vms and several images, which are not working now. I got error message 500, at home I will work on this, I already found some forums where it was discussed https://github.com/retspen/webvirtmgr/issues/22 However I could establish connection, between local computers through the SSH and can manage some functions of WebVirtMgr Also I have finished the scenario for using WVM basing on the Prof. Norbisrath book.

'PACC: Week 1'

Comments

At the PACC internship, at first, I started working on Italc software. I installed the program but faced some problems with authentication keys.

Then I changed the rights to the keys folder, so that a normal user (e.g. teacher) could launch Italc master's interface.

Italc, however, didn't work properly because, the client (ica) was also requesting root rights in order to be launched. If ica-launcher was implemented by a root, Italc would work well and the major Italc features would be working too.

(On my laptop, I installed and set up Italc too but did not have any difficulties with Italc, ica, avahi or anything else)

The next day I found out that the problem of ica-launcher lies in avahi implementation. Avahi uses .local hostname which is already used by local network (or at least, it says to be so). Changing avahi hostname did not work.

Later, prof. Norbisrath found that problems occur because the client asks for some graphical data which it is not allowed to have access to.

'PACC: week 1 conclusions'

Comments

This past week I have started working with webvirtmgr (http://webvirtmgr.net/) that allows to create and manage VMs based on kvm using libvirt python bindings. I created a host server and several guest VMs with Linux Mint installed on them.

Next step I did was to remotely connect to web server's admin panel so that I can manage VMs from any machine in the Internet. After that I wanted to access VMs from one of the virtual machines. However, I have not succeeded yet. I am experiencing some problem with Internet connection inside VM.

I am intending to solve this by more in-depth examination of networking with virtualization. I assume this will take me a day or two to solve.

Once I solve this, I would like to install one of the remote lab management tools examined by our interns (iTalc or Epoptes) on VMs and try to see if that works.

So, the rough road map would be:

  • Solve networking issue in VMs

  • Install iTalk or Epoptes on VMs and see if they work as intended

  • Currently, webvirtmgr has only admin panel, so I intend to create non-admin users. These users will login via web interface and access their VMs.

CFEngine 3 Overview

Comments

Administrating network of multiple machines running the same type of tasks, sooner or later beginning to think about automatism it's configuration and management. Programs like dssh and rsync in this case will help only partially, forcing to perform the big amount of work by hand. However, there is a tool that can automate most of the administrative functions and make a network of servers to self-adjust to the intellectual infrastructure.

CFEngine (ConFiguration Engine) - one of the oldest and most powerful administration tools that allows you to manage computer network automatically with minimal amount of manual work. Through rules CFEngine administrator can describe the condition in which system should be on one or griup of computers in the network,  at a certain time or under certain conditions. Deviation from this condition will result in corrective action.

CFEngine allows to control many aspects of the system, including the editing of files, starting/stopping services, installation/un-installation of applications, network settings and much, much more.

Installation

CFEngine of version 3 is available in almost all UNIX-like operating systems and Linux-based distributions, so get and install it will not be difficult. For example, to install it on Ubuntu is enough to execute a single command:

                # Sudo apt-get install cfengine3

CFEngine package consists of three key components:

  • Server (cfservd)

  • Client (cfagent)

  • Scheduler (cfexecd)

Server (cfservd) - the central part of the software complex, that takes customer requests and sends them instructions for changing the configuration of machines.

Client  (cfagent) -  installed on all managed machines. It's task - to connect to a server and receive configuration files containing execution code of the instructions (modifying system files, installing software, running servers, etc.).

Scheduler (cfexecd) - responsible for running the agent on the client machine. It's task - to run an agent at regular intervals with sending diagnostic messages to the administrator.

Promises CFEngine agent  management is performed by the rules(promises) described in the configuration files. Modifying it, the administrator can change many aspects of the system, including:

  • Check and change the permissions and ownership of files.

  • Editing files.

  • Compression, deletion, and other file manipulations.

  • Remote execution of commands.

  • Restarting the crashed daemons.

  • Installation of the software, including security and system updates.

  • Configuring network interfaces and routing tables.

  • Configuration files(promises) are a kind of scripts with high level descriptive language, on that agent receives information about what action he should take to the configuration of the target machine. A key feature of this script is that it is focused on getting the same results on all configured machines without the need to draw up a separate script for each of them.

Actually the rules in Cfengine can consist of four components - the type, class, object (promiser) and attributes:

> > type: > class:: > "promiser" -> { "promisee1", "promisee2", ... } > attribute_1 => value_1, > … > attribute_2 => value_n. > >

Not all elements are employed in the rules, some implicit elements comprise instructions that may be omitted. Type field indicates the type of operation, that is usually what needs to be done. Depending on the type of system can be used one of the following classes:

  • any rule - var (variables), class (class, showing the state of the system), reports;

  • only agents - commands, databases, files (the creation and filling of the file, set attributes), interfaces (configuration of network interfaces), packages (package installation), storage (check the mapped drive), methods (treatment of other regulations);

  • other components - access (access to objects in the cf-serverd), measurements (data selection for the report or monitoring in Cfengine Nova), roles (authorization activation of individual classes run remotely via cf-agent cf-serverd), topics (Association named when running cf-know) and occurrences (reference to a resource on the cf-know).

CFEngine make possible of  full control of computer system, spreading the changes, setting updates, producing and receiving monitoring reports. At first look Cfengine seems to be complex and confusing system, but in fact it is not. After experimenting for several days everything can be sorted out with its features and capabilities.

Reference

Learning CFEngine 3 - O'Reilly Media - https://docs.google.com/file/d/0B52Wm9eu8cCGdDlpXzA2QkRkOTA/edit?usp=sharing

Cloning or backing up a system with rsync

Comments

I have used rsync as the tool of my choice for a long time to often clone my system or make a backup, which I can still boot from. Feel free to use this code for your own projects. I would enjoy an attribution and a comment here or an email if you use it, but I will put this into the public domain.

Here is the rsync script I use. You just call it as modersync source-dir remote-dir. Source or remote-dir can be even / (the root dir to copy the running system or overwrite the running system). Be careful, use the switch -n (dry run) the first time you experiment with this. Source and remote-dir can even be remote locations reachable via scp.

[bash]

!/bin/bash

rsync a mobiledesktop

parameters: src, dest

src="$1" dst="$2"

LOGFILE=var/log/modersync.log

echo "Welcome to modersync, written by ulno (http://ulno.net)." echo "" echo "Last calls of modersync:" tail "$src/$LOGFILE" "$dst/$LOGFILE"|grep -v "$LOGFILE"|sort -u|tail echo "" echo "Source will be \"$src/\"" echo "Destination will be \"$dst/\"" echo "Extra parameters will be: \"$3 $4\"" echo "To proceed hit return (to interrupt press ctrl-c)." read

test -d "$src" -a -d "$dst" || \ ( echo "specify source and destination-directory!" exit 1 )

rsync $3 $4 -av -H --delete --progress \ --exclude /dev \ --exclude /proc \ --exclude /sys \ --exclude /tmp \ --exclude /mnt \ --exclude /mnt2 \ --exclude /media \ --exclude /MoDeSync \ --exclude .gvfs \ --exclude /etc/modules \ --exclude /usr/local/etc/ulno-machine-settings \ --exclude /boot/grub/menu.lst \ --exclude /boot/grub/grub.cfg \ --exclude /etc/default/grub \ --exclude /etc/modules \ --exclude /etc/fstab \ --exclude /etc/mtab \ --exclude /etc/X11/xorg.conf \ --exclude /etc/hosts \ --exclude /etc/hostname \ --exclude /etc/resolv.conf \ --exclude /etc/uswsusp.conf \ --exclude /var/run \ --exclude /run \ --exclude /etc/modprobe.d/blacklist-custom.conf \ --exclude "$LOGFILE" \ "$src/" "$dst/" rm "$dst/MoDeSync/dirty.generated" &> /dev/null cp "$src/MoDeSync/dirty.generated" "$dst/MoDeSync/" &> /dev/null

logline="$(date -u +%s) $(date) on $(hostname) from /$src to /$dst." echo $logline >> "$src/$LOGFILE" echo $logline >> "$dst/$LOGFILE"

ldconfig update-grub dpkg-reconfigure initramfs-tools

echo echo "Done."

[/bash]

Be careful, this script demands the folder MoDeSync to be present. The log is saved there and will give you a hint about the copy direction in the beginning.

PACC Project: 1st meeting (May 22)

Comments

Assulan: Worked on Virtual Web Manager (web interface), created a host-server and two virtual machines (2nd is a clone of the 1st one). Faced problems with setting up (resolved). Next step: remote connection (remotely connected virtual machines).

Temirlan: Worked on VWM: created a host-server and one virtual machine. Could not find an appropriate image file. Also worked on Epoptes - an alternative for iTALC. Installation was successful. Problem: works with root rights only. Next step: remote control.

Alexandra: Installed iTALC. Faced a problem with root rights (could only be run by root). Solved by creating group of users and giving the needed rights to them. Next step: 10 lines of text (scenario) of what should be achieved with both desktop sharing programs.

Alexandr: Worked on deployment. Was searching for methods/tools. Found out about rsync and alternatives (mainly CF engine). Installed, did not set up properly yet. Next step: have CF engine working, blog on rsync and CF engine.

Ulrich: Posted a manual, worked on blog. Resolved the issues with internships (payment, credit).

TODO: repository, exchanging files, setting up trac.

Small ssh introduction

Comments

Some of my students, family, or friends still fear the black box with white charecters, sometimes called the shell, the command line, or the terminal. However, if you have embraced the concept and the opportunity of what you can achieve with just some simple commands typed into a box, I will give you here a small manual how to even use this power remotely. The magic command is ssh, the secure shell. Linux and MacOSX usually offer this feature by default (sometimes you have to install openssh-server), however, even windows can easily offer this using cygwin. In debian or ubuntu like system you want to issue sudo apt-get install openssh-server to install the server (to make it possible to display the command line from a different computer). A client on MacOSX and Linux systems you just type ssh into a command line, fo rwindows you either want cygwin or putty.

Let's now imagine, we have installed ssh on our computer named work (let's further assume that it is reachable by the IP-address 192.168.20.15 - try if you can type into a command line on another computer ping work and also ping 192.168.20.15). Let's further assume we have another computer running an non graphic (not putty) ssh client. Let's assume this other computer is called home and accessible with the IP-address 192.168.20.120.

On work you usually log in as the user myuser. Try now to type in a command line on computer work like ssh myuser@work. You will see something like th e following (first time you try, you will be asked to accept the key and have to type yes + return key), then it will ask you for the password of your user at work. After entering this, you will see the command line of the work computer appearing on yout home computer. Fascinating, isn't it?

ulno@home:~$ ssh myuser@work

The authenticity of host 'work (192.168.20.15)' can't be established.

ECDSA key fingerprint is 0f:b7:44:81:a5:f9:a5:04:0b:ad:11:2f:82:5f:1d:b7.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'work,192.168.20.15' (ECDSA) to the list of known hosts.

myuser@work's password:

Welcome to Ubuntu 12.10 (GNU/Linux 3.5.0-17-generic x86_64)

  • Documentation: https://help.ubuntu.com/

145 packages can be updated.

65 updates are security updates.

Last login: Mon Jan 28 19:47:16 2013

myuser@work:~$

However, this is actually only how things start to become interesting. Maybe you don't want to type in your password each time, when you login to the work computer. If you don't want to type your password each time, you can generate a local keypair on your home computer and copy the public key to the remote computer. To generate a key type in the following command on your work computer.

ssh-keygen

Confirm all questions just with return (this will allow you not to type any password later, you can though also protect your key with a password). Depending on your operating systems, you will usually now find your keys in .ssh:

ls .ssh/

id_rsa id_rsa.pub

id_rsa is your private key, id_rsa is your public key.

Type cat id_rsa to show your key (you should get something like the following result).

ulno@home:~$ cat .ssh/id_rsa.pub

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKNYsOJGE7o4pQVvyp9UBFJyogn1C3QsyUA56lRS9Dd3svC33uSKsHEkiJPTCA9+fV8xiPtO6z/VXLk4jddNvBx/WcrrRiQIY6d0iTi40RDmEmNG83t7XiXNZ1Lh+VlG5rPkys1aG/FQbXDYllynEQvjfANxzdBJsUBqdMicYGe3tooHsmnKGIR3MBMTrluNJcaDj8gK2b0P+Mu6Y4M8TsNc/pwFsu+ZJpV9hQrdc1GDxHqpELAHf+npZt/ALtHDVGQFy9ifgjqMo31+YDmqC5ssFrNkiJFWPhjiJrbHjoqsLF/7fy63zDOHLJI8cneO+6EUHi/0s+BakR0R81ZVxN ulno@home

Be careful, if somebody gets their hands on your is_rsa file, they can pretned to be you (at least to another computer). Copy the key starting with ssh_rsa and ending with ulno@home into your clipboard (select and press ctrl-c or on Linux ctrl-shift-c) and execute the following commands (after logging in with ssh) on your work computer:

myuser@work:~$ mkdir .ssh # create a directory for the authenticated key (things starting with # are comments and can be ignored)

myuser@work:~$ chmod 700 .ssh # the stuff in here shoudl kept secret and only be readable by you

myuser@work:~$ echo ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKNYsOJGE7o4pQVvyp9UBFJyogn1C3QsyUA56lRS9Dd3svC33uSKsHEkiJPTCA9+fV8xiPtO6z/VXLk4jddNvBx/WcrrRiQIY6d0iTi40RDmEmNG83t7XiXNZ1Lh+VlG5rPkys1aG/FQbXDYllynEQvjfANxzdBJsUBqdMicYGe3tooHsmnKGIR3MBMTrluNJcaDj8gK2b0P+Mu6Y4M8TsNc/pwFsu+ZJpV9hQrdc1GDxHqpELAHf+npZt/ALtHDVGQFy9ifgjqMo31+YDmqC5ssFrNkiJFWPhjiJrbHjoqsLF/7fy63zDOHLJI8cneO+6EUHi/0s+BakR0R81ZVxN > .ssh/authorized_keys # create the file to check your keys without password, make sure to use your own key and be careful not to have line breaks while copying

myuser@work:~$ ls -l .ssh/authorized_keys # mkae sure that this is also only readable by you

This should be all, if you try now to exit your remote work pc and login again with ssh, it should not ask your anymore for a password.

Also don't miss reading about tunneling with ssh (you can build your own proxy and virtual netwrok with this). Links for this are for exampel the following:

VM management tools

Comments

Projects that might be of interest for PACC:

*

https://www.webvirtmgr.net/ - WebVirtMgr is a libvirt-based Web interface for managing virtual machines. It allows you to create and configure new domains, and adjust a domain's resource allocation. A VNC viewer over a SSH tunnel presents a full graphical console to the guest domain. KVM is currently the only hypervisor supported. It uses Django/Python as web framework, and Python libvirt (http://libvirt.org/python.html) as a tool to manage VMs.

This is small and less mature software, but it is highly customizable or can be used as a basis for our own project.

*

http://opennebula.org/about:about - OpenNebula.org is an open-source project developing the industry standard solution for building and managing virtualized enterprise data centers and enterprise private clouds.

> > The toolkit includes features for integration, management, scalability, security and accounting. It also emphasizes [standardization](http://en.wikipedia.org/wiki/Standardization), [interoperability](http://en.wikipedia.org/wiki/Interoperability) and [portability](http://en.wikipedia.org/wiki/Portability), providing cloud users and administrators with a choice of several cloud interfaces ([EC2](http://en.wikipedia.org/wiki/Amazon_EC2) Query, OGF [OCCI](http://en.wikipedia.org/wiki/Open_Cloud_Computing_Interface) and [vCloud](http://en.wikipedia.org/wiki/VCloud)) and hypervisors ([Xen](http://en.wikipedia.org/wiki/Xen), [KVM](http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine) and [VMware](http://en.wikipedia.org/wiki/VMware)), and a flexible architecture that can accommodate multiple hardware and software combinations in a [data center](http://en.wikipedia.org/wiki/Data_center). (taken from http://en.wikipedia.org/wiki/OpenNebula) > >