vTorque introduces virtualization capabilities to the batch system resource manager Torque to enable cloud like functionality. vTorque enables Torque to deploy and execute job scripts in virtual machines, transparently to the user. vTorque is open source and independent of a specific version of Torque, thanks to its non-invasive nature.
vTorque consists of a set of bash scripts, several templates and a new command line tool for the job submission in virtual environment, called vsub.
The vsub command line interface accepts all standard PBS/Torque arguments, but also introduces several new arguments related to virtual resources, e.g. the amount of vCPUs.
vTorque consists of several wrapper scripts used as hooks for the various sequences in Torque’s job life-cycles, i.e. root and user prologue, to manage virtual guests.
Figure 1: MIKELANGELO software stack for HPC
vTorque integrates various components of the MIKELANGELO stack, outlined in the figure above, and the following blog post: https://www.mikelangelo-project.eu/2017/07/awesome-architecture/
With the various components integrated, vTorque is a complete solution available for the use in HPC batch system infrastructures. The red components in the figure are related to Cloud, only, thus they are not considered for HPC integration.
Achievements and Results
A first proof of concept prototype has been successfully tested at the end of the first year and is presented in the blog post Making Torque ready for Virtual Machines
As of July 2017 vTorque has meanwhile reached a stable and feature rich state, feasible for reliable execution of virtualized and bare-metal bach jobs. vTorque supports various configuration options for virtual resources available at submission time to users. Further, it enables administrators to define optimized defaults for their specific environment and also enables them to disable or enable specific components or behaviour of vTorque.
- D2.19 The first MIKELANGELO architecture – published September 2015
- D2.20 The intermediate MIKELANGELO architecture – published July 2016
- D2.21 The final MIKELANGELO architecture – published July 2017
- D5.2 Intermediate Report on the Integration of sKVM and OSv with Cloud and HPC – published January 2017
- D5.4 First report on the Integration of sKVM and OSv with HPC – published January 2016
- Basic vTorque concept and proof of concept prototype: https://www.mikelangelo-project.eu/2016/06/making-torque-ready-for-virtual-machines/
- The big picture: https://www.mikelangelo-project.eu/2017/07/awesome-architecture/
- vTorque source code can be found on github: https://github.com/mikelangelo-project/vTorque
- log4bsh, formerly part of vTorque, meanwhile available as a separate project: https://github.com/mikelangelo-project/log4bsh