The MIKELANGELO project proudly presents the overall MIKELANGELO technology stack.
We plan to release technology updates in six month intervals – this means you know when to expect the new flavours of the MIKELANGELO components and when to schedule internal testing of our releases. As of today, two releases have been published:
MIKELANGELO Stack 2.0:
Over the last six months, MIKELANGELO has improved a large number of components to deliver the MIKELANGELO Stack 2.0. Please find all detailed improvements below.
- sKVM has been improved in particular through IOcm, vRDMA, and SCAM.
- IOcm has been extended by a dynamic IO core manager. The dynamic IO core manager samples I/O statistics, estimates the I/O pressure, and allocates the right amount of I/O cores.
- A second prototype of vDRMA has been implemented. Compared to first prototype, the front-end driver has been improved. It now allows the use of a verbs-based API for improved performance.
- SCAM has been extended by monitoring, profiling, and initial mitigation of attacks.
- In the guest OS layer, OSv and Seastar have been extended.
- OSv has been improved for better support of applications including additional system call and C-lib functions, new build environments, a better NFS client, isolated threads, partial NUMA support, cloud-init support, improved DHCP, file system performance, and assorted bug fixes.
- Seastar improvements cover improved RPC, I/O scheduling, IOtune, CPU scheduling, monitoring, ext4 support, a log-structured memory allocator, and DPDK support.
- The middleware extensions cover the MIKELANGELO Package Manager (MPM), Snap for monitoring, Scotty for system testing, vTorque for virtual HPC, and the MIKELANGELO Cloud Manager (MCM).
- The MPM has been extended with a Docker container to sever the dependency to KVM during VM composition. Furthermore, MPM has been integrated with UniK for a better cloud integration.
- Snap has been extended by ten new collector plugins and two new processor plugins. Most of those plugins relate directly to MIKELANGELO-specific components and some of them have already been further enhanced by the snap community.
- Scotty has been developed from scratch to allow automated experimentation with virtual infrastructures (cloud and HPC). The initial version of Scotty is shipped with Puppet-based deployment scripts.
- For HPC, vTorque has been developed as an extension to Torque to allow the deployment of virtual machines. The current release allows the transparent provisioning of Linux and OSv instances for virtual workloads.
- For the cloud, MCM has been developed as an extension to OpenStack. MCM allows live-scheduling of resources in OpenStack. MCM integrates with OpenStack’s APIs and InfluxDB to collect monitoring data. The main goal of the MCM development is to foster the development of cloud resource management algorithms.
The individual components have been tested in two integrated testbeds. The first testbed uses OpenStack for a cloud deployment. This testbed integrates OSv, IOcm, Snap, MCM, Scotty, and MPM. The second testbed targets HPC and integrates OSv, IOcm, Snap, vRDMA, vTorque, and Scotty. The project’s use cases (link) have used the integrated testbeds to validate the components.
MIKELANGELO Stack 1.0:
This is the first release of the MIKELANGELO project and presents the first step towards the fully integrated and demonstrated MIKELANGELO technology stack. This release is the result of the ramping up period, followed up by intensive development in the first 18 project months. You can see the release as an indication of the directions the project is about to take (as we prepare the ground to introduce new packages, techniques, and more).
We prepared three extensive documents for everyone who wishes to fully check and understand the work on different components, how they fit together and how well they perform.
The first document is about the overall architecture of the MIKELANGELO technology stack and the individual components – The intermediate MIKELANGELO architecture. In short, skimming through the document will show you the components developed, the outlook for new features and also provide you with information on how to run them and what to expect.
Understanding the technology is a good way to start thinking if it is a good fit for your particular use case. In MIKELANGELO, we have four use-cases, ranging from HPC to Cloud – these are all described in the Use Cases Implementation Strategy. You will see what our use-cases are and how we tackled them.
Finally, as MIKELANGELO is about improving the performance in the virtual technology stack, increasing flexibility, unifying the HPC and Cloud software stack, we put these to the test – The Architecture and Implementation Evaluation. This document shows where we are at M18.