Private Academic Cloud Computing in Research and Teaching
Many scientific and industrial problems have the characteristic that they require substantial computing resources, but only on an occasional basis. Such tasks might run once a month, or even once a week, but not every day. Constructing and maintaining large dedicated high-performance computing facilities for such tasks is often cost-prohibitive, in terms of actual financial impact as well as the time required to manage and maintain the systems. An additional concern is that such dedicated facilities are often under-utilized, due to the irregular demands placed upon them. The proposal to establish a Private Academic Cloud Computing (PACC) initiative directly addresses these issues by gathering the unused capacity of existing computers to construct a shared network of “pooled” resources. The otherwise idle resources of the existing machines are then configured and allocated as needed to any application that needs higher-performance computing. The project incorporates recent work in multiple areas of relevance: cycle harvesting, grid networks, Friend-to-Friend computing, Private Cloud systems, the migration of applications to the Cloud framework, and the related issues of cost, performance, and reliability. The PACC initiative will be used for research into Cloud configurability, network topology, performance, credentialing and security, as well as the teaching of Cloud principles to students. The PACC system will serve as an innovative scientific computing experimentation environment to support applications across disciplines. The system is purposed as a low-risk testbed for Cloud methodologies, as a way to run experiments prior to deployment of large-scale national or international projects. The PACC system is designed to be extensible as well, such that if successful, and applications merit additional resources, these resources (new machines, faster processors, more memory) can be added as needed, and linked with other projects of a similar character. The PACC resources will support applications throughout the university, and those of potential partners in both industry and academia (particularly the existing international network of partnering institutions) who would otherwise not have the resources to develop dedicated higher-performance systems.
Open Topics and Work Packages
If we want to make this vision of a useful private academic cloud happen there has to be done a lot of work both in adminstration as in research. If you are a local or external student,researcher, or teacher interesting in using our devlopments or helping us moving forward, please get in touch with me as the principal investigator of this project. If you have ideas for more related projects, please share them with us an dwe will try to make them happen at one point. Currently, we are gethering work packages and ideas. Here is the list of the topics, we have located so far:
Cordinating purchase and installation of ac to cool 30 servers in room 7422
Coordinating purchase and installation of memory for lab computers
Developing fast deployment mechanism for cloning existing lab to new resources (labs or single computers)
Evaluate openstack against native kvm implementation
Have a side look at opennebula
Initial KVM, VNC, and Spice experiments
Find most efficinet way to do remote access
Find way to nicely represent a network of running VMs to one user
Developing method for allocating one VM to one User
Extending previous method to allocating multiple VMS in virtual VDE network to one user
User login integration with LDAP or simpler authentication mechanism
Installation/Development of provisioning framework
Developing crediting system to allow credit based resource reservation and allocation
Testing and performance, run benchmarks for cloud and vde
Build optimization algorithm for automatic ditribution of system parts (i.e. liniar programming)
Building libvirt and ssh based auto deployment environment
Build a web2py based management environment with open interface to libvirt
Evaluate owncloud as student (first only sst and engineering)-wide local dropbox replacement
Find a way, money (evtl. from this grant), and license model to allow NU students to install and maintain their own MS windows machine.
or find a way to do this per seat.
Same for MS office licenses.
experiment with running MS office under wine and serving this on a per-seat base to users.
Setup a ubuntu mirror inside NU.
Setup MS update server inside NU.
Extending and fixing transparent proxy for lab (to allow easy VM setup)
Black and whitelisting IPs/webaddresses for a specific set of student instances.
Integrate this with the project for fixing the transparent proxy. Try eventually to install forwarding proxy - consider also implementing ad-blocking to save bandwidth.
Blocking/allowing access to specific virtual machines for a class time.
Analyze and setup italc (http://italc.sourceforge.net/) or other desktop sharing solutions.
Study webvirtmgr and set-up website to control persoal vms
Implement wake on lan (WOL) for the servers in the class to save energy
fix energy saving funtions in ubuntu in lab
Cybersecurity testbed setups
sandboxing, honeypots and more
Leave the cloud, enter the cloud
Study ways of mass exporting emails from gmail to our own mailserver mixed with our own cloud storage (owncloud)
study gmvault for email export/sync/backup
consider roundcube in combination with owncloud (maybe zimbra or skyrix -- more likely not)
Develop backup-startegy for system services, users, and user's vms
Develop and set up easy remote access to lab (tunnel port over web connection, simulate web-based vpn via ssh-port)
Experimentation and set up of simple trac or git environment for hosting development repositories for students, student groups, and research development projects in virtual environemnts
Write down cases and scenarios about how we can envision using the academic cloud
Ulrich Norbisrath -- Principal Investigator
Michael Lewis -- project member
Bakhytzhan Kallemov -- project member
Assulan Nurkas -- Research Assistant
Islam Sabyrgaliyev -- Research Assistant
Involved interns, summer 2013
Alexandra Kim -- 1
Damir Doszhan -- 1
Raushan Gusmanova -- 1
teaching Parallel Architecture
http://vde.sourceforge.net/ -- network virtualization
google doc we use for coordination (only accessible for members): https://docs.google.com/document/d/1PSHcoEqBW1Hs0GZS5FyjiEe6gyvTZiKCM5n64qrySIM/edit
Georg Singer, Ilja Livenson, Marlon Dumas, Satish N. Srirama, and Ulrich Norbisrath: Towards a model for cloud computing cost estimation with reserved resources. Proceedings of 2nd International ICST Conference on Cloud Computing (CloudComp 2010), Barcelona, Spain.