Green IT

Since the mid 90s, clusters are a popular hardware platform to compute-intensive applications, but the power consumption of these machines has already reached an unacceptable amount. In 2007 it was already estimated that the ICT Industry is already responsible for 2% of the global CO2 emission. The High-Performance community is aware of this conflict. As a complement to the list of the 500 fastest machines, the Green500 compares supercomputers by their performance-per-watt since November 2007.


CHERUB - an energy saving daemon for HPC- and SLB-clusters

Compute clusters are often managed by a so-called Resource Management System (RMS) which has load information about the whole system. CHERUB is a centralized daemon which is installed on top of an arbitrary RMS and uses its load information to switch on/off nodes according to the current load situation and load forecasting to save energy in this way. Due to its modular design and its well defined API it can operate with different Resource Management Systems. At the Moment there are modules available for the Portable Batch System (PBS), the Load Sharing Facility (LSF) and the IBM Load Leveler (LL) in the High Performance Compute (HPC) field and for the Linux Virtual Server (LVS) in the Server Load Balancing (SLB) field.
CHERUB uses 5 internal states to manage the nodes of a cluster. The relation between these states can be seen in Fig.1 below.

cherub statediagram
Fig.1: Different Cherub States

Our experimental results shown in Fig. 2 and 3 prove that Cherubs energy management works fine, i.e. it will save energy if possible (Fig. 2) and avoids state-flipping (Fig. 3). Within the experiment node001-node008 where configured to be powered on all the time. While the cluster was doing normal business CHERUB was able to save around 48% of energy (not considering the always on nodes).

cherub savings in one month
Fig.2: Energy Saved with CHERUB
cherub avoids state-flipping
Fig.3: Downtime of nodes when managed with CHERUB

At the moment, Cherub is used on the cluster of the Institute of Computer Science at the University of Potsdam (cluster managed by the Portable Batch System (PBS)) and the German Research Centre for Geosciences at Potsdam (cluster managed by the Load Sharing Facility (LSF)).
Current development efforts are focused on optimizing the Linux Virtual Server module for Server Load Balancing scenarios by using load forecasting and backup features.


The current version (1.3.11) of Cherub can be found here.


If you have any questions regarding CHERUB feel free to contact the developer at simon.kiertscher(at)cs.uni-potsdam.de.


Publications - HPC

Power Consumption Aware Cluster Resource Management
Simon Kiertscher, Jörg Zinke and Bettina Schnor
Bookchapter in Energy-Aware Systems and Networking for Sustainable Initiatives, ISBN-13: 9781466618428
IGI-Global, December 2012


CHERUB: power consumption aware cluster resource management
Simon Kiertscher, Jörg Zinke and Bettina Schnor
Journal of Cluster Computing, ISSN: 1386-7857
Springer Netherlands, September 2011


Cherub: Power Consumption Aware Cluster Resource Management
Simon Kiertscher, Jörg Zinke, Stefan Gasterstädt and Bettina Schnor
IEEE/ACM International Conference on Green Computing and Communications
Hangzhou, China, December 2010


Publications - SLB

Scalability Evaluation of an Energy-Aware Resource Management System for Clusters of Web Servers
Simon Kiertscher and Bettina Schnor
International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS)
Chicago, USA, July 2015


Energy Aware Resource Management for Clusters of Web Servers
Simon Kiertscher and Bettina Schnor
IEEE International Conference on Green Computing and Communications, page 148-156
Beijing, China, August 2013

Intel SCC

The group is member of the Intel Many-Core Research Architecture Community (MARC) and has physical access to a loaned Intel 48-core Single-Chip Cloud Computer (SCC). The SCC is an experimental processor with 48-cores, developed by Intel Labs as a platform for many-core software research. The SCC has special hardware support for message passing and energy management.


The group is working on "Parallel Programming Patterns suited for SCC". We want to identify parallelization patterns and/or communication patterns which are best suited for the SCC and result in scaling applications. In cooperation with the Potsdam Institute of climate impact (Group of Prof. Rahmsdorf) we investigate the scaling of Aeolus on the SCC.


Intel has delivered an implementation of the Message Passing Interface Standard (MPI) called RCKMPI. We have developed an optimization of RCKMPI that improves the communication bandwidth by evaluating the application's communication graph and making optimal use of SCCs Message Passing Buffers. Further, the memory subsystem allows versatile configurations and enables the usage globally shared memory. We investigate an efficient usage of the memory configuration for parallel applications.


Publications

Synchronization of One-Sided MPI Communication on a Non-Cache Coherent Many-Core System
Steffen Christgau and Bettina Schnor
12th Workshop on Parallel Systems and Algorithms (PASA)
Nürnberg, Germany, April 2016


Software-managed Cache Coherence for fast One-Sided Communication
Steffen Christgau and Bettina Schnor
7th International Workshop on Programming Models and Applications for Multicores and Manycores
Barcelona, Spain, March 2016


One-Sided Communication in RCKMPI on the SCC
Steffen Christgau and Bettina Schnor
6th Many-core Applications Research Community Symposium
ONERA Toulouse, France, July 2012


Awareness of MPI Virtual Process Topologies on the Single-Chip Cloud Computer
Steffen Christgau and Bettina Schnor
17th International Workshop on High-Level Parallel Programming Models and Supportive Environments (HIPS) at IPDPS 2012
Shanghai China, May 2012


The Benefit of Topology-Awareness of MPI Applications on the SCC
Steffen Christgau, Bettina Schnor and Simon Kiertscher
3rd Many-core Applications Research Community Symposium
Ettlingen, Germany, July 2011

salbnet: Self-Adapting Load Balancing Network

Server Load Balancing (SLB) is an efficient way to provide scalable and fault tolerant services. The traffic of a site is distributed among a number of servers connected via network.

slb scenario
Fig.1: Server Load Balancing Scenario

Usually, the Load Balancer distributes the load to the back-end servers in a round-robin fashion or according to current server load based on external load parameters. The use of server weights is very popular to improve the load balancing in case of heterogeneous server machines. Correctly determined weights are crucial to the quality of the distribution. While the determination of weights can be done in small and static environments, it can hardly be done in dynamic or heterogeneous environments.

slb scenario
Fig.2: Server Load Balancing Scheme

Often, the load balancing environment is very heterogeneous, since new machines are added at the time the number of servers becomes insufficient. Current approaches use server weights to distinguish the fast from the slow machines. The problem with static weights is that they strongly depend on the application, the kind of requests, the back-end network interconnect, and the server's capabilities. Non-exact weights significantly reduce the quality of the distribution, resulting in increased answer time, more dropped requests, and unbalanced usage of servers.


More sophisticated approaches choose the server on the basis of measured load indicators, but to the price of less scalable algorithms with a complexity of O(n).


We propose a credit-based load balancing scheme that is self-adapting and is basing on an O(1) algorithm, that is able to self-adapt to heterogeneous servers and heterogeneous workloads without the need to specify server weights.


Server Load Balancing for InfiniBand Networks

InfiniBand is an evolving high speed interconnect technology to build high performance computing clusters.


Network interfaces (called host channel adapters) provide transport layer services over connections and datagrams in reliable or unreliable manner. Additionally, InfiniBand supports remote direct memory access (RDMA) primitives that allow for one-sided communication.


For InfiniBand-Cluster, salbnet uses the RDMA-feature to update credits to the dispatcher.


Publications

Self-Adapting Load Balancing for DNS
Jörg Jung, Simon Kiertscher, Sebastian Menski and Bettina Schnor
Journal of Networks
April 2015
Extended journal version of SPECTS 2014


Self-Adapting Load Balancing for DNS
Jörg Jung, Sebastian Menski, Bettina Schnor and Simon Kiertscher
International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS)
Monterey, USA, July 2014
Best Paper


salbnet: A Self-Adapting Load Balancing Network
Jörg Jung, Bettina Schnor and Sebastian Menski
In Proceedings of the 12th Conference on Parallel and Distributed Computing and Networks (PDCN)
Innsbruck, Austria, February 2014


The Impact of Weights on the Performance of Server Load Balancing Systems
Jörg Zinke and Bettina Schnor
International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS)
Toronto, Canada, July 2013


System call tracing overhead
Jörg Zinke, Bettina Schnor and Simon Kiertscher
UpTimes 2/2009 - In Proceedings of the international Linux System Technology Conference and OpenSolaris Developer Conference, ISBN: 3865413587, ISBN-13: 9783865413581
Dresden, Germany, October 2009


Self-Adapting Credit Based Server Load Balancing
Lars Schneidenbach, Bettina Schnor, Jörg Zinke and Janette Lehmann
In Proceedings of the 26th Conference on Parallel and Distributed Computing and Networks (PDCN)
Innsbruck, Austria, February 2008


Self-Adapting Server Load Balancing in InfiniBand Networks
Lars Schneidenbach and Bettina Schnor
Technical Report, Universität Potsdam, ISSN: 0946-7580, TR-2007-2
Potsdam, Germany, 2007


SLIBNet: Server Load Balancing for InfiniBand Networks
Sven Friedrich, Lars Schneidenbach and Bettina Schnor
Technical Report, Universität Potsdam, ISSN: 0946-7580, TR-2005-12
Potsdam, Germany, 2005

servload: Service Benchmarking

servload is a lightweight (∼ 2000 SLOC ANSI C) high performance service benchmark which supports the protocols HTTP and DNS. servload uses requests and responses to determine performance metrics. Furthermore servload loads (and may modify) a given log file and issues (replay) the resulting requests.


The current version of servload can be found here.


See INSTALL file for further hints on installation from source and manual page for usage details.


Publications

servload: Generating Representative Workloads for Web Server Benchmarking
Jörg Zinke, Jan Habenschuß and Bettina Schnor
International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECT)
Genoa, Italy, July 2012

OpenBSD PF Development

The OpenBSD Packet Filter (PF) can also be used as load balancer. We integrated Weighted Round Robin and the Least States algorithms for PF. It is part of the OpenBSD implementation since release 5.0. Further information can be found in Fabian Hahns diploma thesis (in German).