V3VEE: An open source virtual machine monitor (VMM) framework for modern architectures (http://v3vee.org)
Palacios is a new open-source VMM that enables applications executing in a virtualized environment to achieve scalable high performance on large machines. Palacios functions as a modularized extension to Kitten, a high performance operating system being developed at Sandia National Laboratories to support large-scale supercomputing applications. Together, Palacios and Kitten provide a thin layer over the hardware to support full-featured virtualized environments alongside Kitten’s lightweight native environment. Palacios supports existing, unmodified applications and operating systems by using the hardware virtualization technologies in recent AMD and Intel processors. Additionally, Palacios leverages Kitten’s simple memory management scheme to enable low-overhead pass-through of native devices to a virtualized environment. We describe the design, implementation, and integration of Palacios and Kitten. Our benchmarks show that Palacios provides near native (within 5%), scalable performance for virtualized environments running important parallel applications. This new architecture provides an incremental path for applications to use supercomputers, running specialized lightweight host operating systems, that is not significantly performance-compromised.
J. Lange, K. Pedretti, T. Hudson, P. Dinda, Z. Cui, L. Xia, P. Bridges, A. Gocke, S. Jaconette, M. Levenhagen and R. Brightwell. Palacios and Kitten: New High Performance Operating Systems For Scalable Virtualized and Native Supercomputing. In Proceedings of the 24th IEEE International Parallel and Distributed Processing Symposium (IPDPS'10), Atlanta, GA, April 2010. [paper]
VNET/P: A fast virtual overlay network for high performance computing
Overlay networking with a layer 2 abstraction provides a powerful model for virtualized wide-area distributed computing resources, including for high performance computing (HPC) on collections of virtual machines (VMs). With the emergence of cloud computing, it is now possible to allow VMs hosting tightly-coupled HPC applications to seamlessly migrate between distributed cloud resources and tightly-coupled supercomputing and cluster resources. However, to achieve the application performance that the tightlycoupled resources are capable of, it is important that the overlay network not introduce significant overhead relative to the native hardware.
To support such a model, we designed and implemented a virtual networking system that has negligible latency and bandwidth overheads in 1-10 Gbps networks. Our system, VNET/P, is embedded into our publicly available Palacios virtual machine monitor (VMM). VNET/P achieves native performance on 1 Gbps Ethernet networks and very high performance on 10 Gbps Ethernet networks and InfiniBand. On the latter networks, performance is converging to native speeds as we continue to enhance VNET/P.
Z. Cui, L. Xia, P. Bridges, P. Dinda and J. Lange.
Optimizing Overlay-based Virtual Networking Through Optimistic Interrupts and Cut-through Forwarding.
In the International Conferences for High Performance Computing, Networking, Storage and Analysis (SC'12), Salt Lake City, Utah, Novermber 2012.
[paper]
K. Hale, L. Xia and P. Dinda. Shifting GEARS to Enable Guest-context Virtual Services.
In the 9th International Conference on Autonomic Computing (ICAC'12), San Jose, CA, September 2012.
[paper]
L. Xia, Z. Cui, J. Lange, Y. Tang, P. Dinda and P. Bridges.
VNET/P: Bridging the Cloud and High Performance Computing Through Fast Overlay Networking.
In 21st International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC'12). (Best Paper Nominee)
[paper|talk]
Virtual WiFi: A virtualization approach for wireless network devices
New approach to virtualization that enables wifi virtualization. One phyisical WiFi interface is virtualized and can be used in multiple VMs. Current approach is to virtualize an ethernet device inside the GuestVM. This strips all the wifi functionality. The new approach virtualizes complete wifi functionalities in the VM. The same Intel Wifi driver is used in the GuestVM as is used in the HostVM. Each VM gets its own vMAC, HostVM distributes packets according to vMAC, all other capabilities are directly forwarded to the VMs and can be set by the VMs as well.
L. Xia, S. Kumar, X. Yang, P. Gopalakrishnan, Y. Liu, S. Schoenberg and X. Guo Virtual WiFi: Bring Virtualization from Wired to Wireless. In Proceedings of ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE'11), NewportBeach, CA, March 2011. [paper|talk]
Virtual Passthrough I/O (VPIO): A model-based approach to improve virtualized I/O performance on commodity devices
Virtual passthrough I/O (VPIO) is our proposed IO virtualization approach that uses a model-based method to allow virtual passthrough I/O on commodity devices.
The current approaches to allow high-performance I/O in guest operating systems are limited. In one approach, the virtual machine provides full emulation of the device in order to multiplex it to each guest operating system; however, this requires significant overhead in the VM. To reduce the performance penalty, a guest might bypass the virtual machine altogether in direct-assignment I/O. However, this approach is less secure, since a guest could affect the memory of other guests or the VM itself. Some devices are multiplexed in the hardware and allow each guest to directly access the device while preserving security, but this feature is not available on commodity I/O devices, nor do these devices currently allow migration of guests.
VPIO allows a guest to have direct access to the hardware for most operations and also allows a guest to migrate. VPIO assumes that there is a simple model of the device that can determine (1) whether a device is reusable, (2) whether a DMA is about to be initiated, and (3) what device requests are needed to update the model. VPIO also assumes that the device can be context-switched, that is, that the device can deterministically save or restore the state pertaining to a guest operating system. For the best performance, the goal of VPIO is to have most guest/device interactions complete without an exit into the VM.
Under VPIO, each access to the device must go through a Device Modeling Monitor (DMM). The purpose of DMM is twofold: (1) It saves enough state about the guest and the device that a guest could migrate to a new VM, and (2) it ensures that the VMM enforces proper security. It also keeps track of a hooked I/O list, which is a set of I/O ports that require VM intervention if accessed by a guest. Unhooked I/O ports may be used by the guest directly. The device is multiplexed by performing a context switch on the device (restoring the guest-specific state into the device).
We implemented two example models for NE2000 and RTL8139 network cards and had it running under QEMU. The model was under 1000 lines of code, and only a small fraction of I/Os (about 1 in 30) needed VM intervention.
L. Xia, J. Lange, P. Dinda and C. Bae.
Investigating Virtual Passthrough I/O on Commodity Devices.
ACM Operating Systems Review, Volume 43, Number 3, July, 2009. [paper]
L. Xia, J. Lange and P. Dinda.
Towards Virtual Passthrough I/O on Commodity Devices.
In the First Workshop on I/O Virtualization (WIOV'08), in conjunction with OSDI'08, San Diego, CA, Dec. 2008.
[paper| talk]