public marks

PUBLIC MARKS from camel with tags google & software

April 2008

Vdoop - Manage your virtual cluster - Vdoop

Every day, search companies like Google download terabytes of data from the Internet, store it on clusters of thousands of machines, and process it so that it can be easily searched. To make this possible, these companies need sophisticated distributed file system and parallel programing architectures. Have you ever heard of the Map/Reduce distributed parallel programing paradigm? If you are a computer scientist, you should have, because every time you submit a Google search, you are using Map/Reduce. Despite growing demand from companies like Google, Yahoo, and Microsoft, few computer science majors have even heard of Map/Reduce, let alone graduate well versed in its use. Unfortunately, several barriers exist to integrating Map/Reduce into computer science curricula. Obtaining a large cluster, configuring it, and installing complicated distributed file system and parallel programing software is difficult, time consuming, and expensive. In the past, Google's solution to this problem has been to ship entire clusters pre-configured with Map/Reduce software to select universities. In essence, Vdoop does same thing, with exactly the same software, except for our clusters are virtual, and hence free.

December 2007

ganeti - Google Code

by 1 other
Ganeti is a virtual server management software tool built on top of Xen virtual machine monitor and other Open Source software. However, Ganeti requires pre-installed virtualization software on your servers in order to function. Once installed, the tool will take over the management part of the virtual instances (Xen DomU), e.g. disk creation management, operating system installation for these instances (in co-operation with OS-specific install scripts), and startup, shutdown, failover between physical systems. It has been designed to facilitate cluster management of virtual servers and to provide fast and simple recovery after physical failures using commodity hardware.