Distributed Hash-tables for Scientific Computing

Distributed Hash-Tables are a common data-store in distributed applications and have proven to be useful in the context of webservices and Big Data applications.


In this project, we investigate how the field of High Performance Computing benefits from this technique. Therefore, we explore two research questions. The first one deals with the efficient implementation of a DHT. Compute-Clusters are equipped with High-performance networks like InfiniBand or RoCE and offer special communication APIs.


The second question is which scientific applications may benefit from DHTs? Therefore, we are adapting existing applications. This is done in cooperation with partners at the GFZ, Potsdam.


Open Source Software


Publications

A fast MPI-based Distributed Hash-Table as Surrogate Model for HPC Applications
Max Lübke, Marco De Lucia, Stefan Petri and Bettina Schnor
Accepted at the 25th International Conference on Computational Science (ICCS), Singapore
July 2025


A fast MPI-based Distributed Hash-Table as Surrogate Model demonstrated in a coupled reactive transport HPC simulation
Max Lübke, Marco De Lucia, Stefan Petri and Bettina Schnor
arXiv
April 2025


POET (v0.1): Speedup of Many-Cores Parallel Reactive Transport Simulations with Fast DHT-Lookups
Marco De Lucia, Michael Kühn, Alexander Lindemann, Max Lübke and Bettina Schnor
Geoscientific Model Development
May 2021


Talks

Combining machine learning and equation based models in reactive transport: POET
Marco De Lucia, Michael Kühn, Max Lübke, Bettina Schnor
Goldschmidt Conference
July 2023


Student Thesis

Einsatz und Bewertung von verschiedenen Key-Value-Stores zur Beschleunigung der gekoppelten hydrodynamisch-geochemischen Simulation POET
Nico Sauerbrei
Bachelor Thesis, University of Potsdam, 2023


Design, implementation and validation of an interpolation-based surrogate model for the POET simulator
Max Lübke
Master Thesis, University of Potsdam, 2023


Einsatz und Bewertung einer verteilten Hash-Table zur Beschleunigung von gekoppelten hydrodynamisch-geochemischen Simulationen
Max Lübke
Bachelor Thesis, University of Potsdam, 2020


Parallelisierung und Modularisierung einer Reaktiven-Transportsimulation
Alexander H. W. Lindemann
Master Thesis, University of Potsdam, 2019


Implementierung einer DistributedHashtable mittels MPI/OSC
Max Lübke
Student Thesis, University of Potsdam, 2018