DSZOOM Home Page
Low Latency Distributed Software-Based Shared Memory
Project Overview
Software-implementations of shared memory are still far behind the performance of hardware-based shared memory implementations and are not viable options for most fine-grain shared-memory applications. The major source for their inefficiency comes from the cost of interrupt-based asynchronous protocol processing, not from the actual network latency. As the raw hardware latency of inter-node communication decreases, the asynchronous overhead in the communication becomes more dominant. Elaborate schemes, involving dedicated hardware and/or dedicated protocol processors, have been suggested to cut the overhead.
This project demonstrates how all the asynchronous overhead can be completely removed by running the entire coherence protocol in the requesting processor. This not only removes the asynchronous overhead, but also makes use of a processor that otherwise would stall. The technique is applicable to both page-based and fine-grain software shared memory.
The DSZOOM project is supported in part by Sun Microsystems, Inc., and the Parallel and Scientific Computing Institute (PSCI).
Project Contributors
- Erik Hagersten (Professor)
- Zoran Radovic (PhD Student)
- Håkan Zeffer (PhD Student)
- Henrik Löf (PhD Student, Div. of Scientific Computing)
- Oskar Grenholm (MS Student, alumni)
- Niklas Ekström (MS Student, alumni)
Conference Publications and Presentations
This paper gives a complete overview of the basic DSZOOM system. It demonstrates how all interrupt- and/or poll-based asynchronous protocol processing can be completely removed by running the entire coherence protocol in the requesting processor.
- Removing the Overhead from Software-Based Shared Memory by Zoran Radovic and Erik Hagersten. In Proceedings of Supercomputing 2001, Denver, Colorado, USA, November 2001.
This paper presents a runtime system concept that enables unmodified POSIX P1003.1c (Pthreads) compliant binaries to run transparently on clustered hardware.
- THROOM - Supporting POSIX Multithreaded Binaries on a Cluster by Henrik Löf, Zoran Radovic, and Erik Hagersten. In Proceedings of the 9th International Euro-Par Conference (Euro-Par 2003), Klagenfurt, Austria, August 2003.
This paper introduces a new write permission cache (WPC) technique that exploits spatial store locality and batches coherence actions at runtime.
- Exploiting Spatial Store Locality through Permission Caching in Software DSMs by Håkan Zeffer, Zoran Radovic, Oskar Grenholm, and Erik Hagersten. In Proceedings of the 10th International Euro-Par Conference (Euro-Par 2004), Pisa, Italy, August 2004.
- Flexibility Implies Performance by Håkan Zeffer, Zoran Radovic, and Erik Hagersten. Appears in Proceedings of the 20th IEEE International Parallel & Distributed Processing Symposium (IPDPS 2006), Rhodes Island, Greece, April 2006.
Workshop Publications and Presentations
- Implementing Low Latency Distributed Software-Based Shared Memory by Zoran Radovic and Erik Hagersten. In Proceedings of the Workshop on Memory Performance Issues (WMPI 2001), held in conjunction with the 28th International Symposium on Computer Architecture (ISCA28), Göteborg, Sweden, June 2001.
Technical Reports
- TMA: A Trap-Based Memory Architecture by Håkan Zeffer, Zoran Radovic, Martin Karlsson, and Erik Hagersten. Technical report 2005-015, Department of Information Technology, Uppsala University, May 2005.
- Flexibility Implies Performance by Håkan Zeffer, Zoran Radovic, and Erik Hagersten. Technical report 2005-013, Department of Information Technology, Uppsala University, April 2005.
- Evaluation, Implementation and Performance of Write Permission Caching in the DSZOOM System by Håkan Zeffer, Zoran Radovic, Oskar Grenholm, and Erik Hagersten. Technical report 2004-005, Department of Information Technology, Uppsala University, February 2004.
- Latency-hiding and Optimizations of the DSZOOM Instrumentation System by Oskar Grenholm, Zoran Radovic, and Erik Hagersten. Technical Report 2003-029, Department of Information Technology, Uppsala University, May 2003.
- THROOM - Running POSIX Multithreaded Binaries on a Cluster by Henrik Löf, Zoran Radovic, and Erik Hagersten. Technical Report 2003-026, Department of Information Technology, Uppsala University, April 2003.
- DSZOOM--Low Latency Software-Based Shared Memory by Zoran Radovic and Erik Hagersten. Technical Report 2001:03, Parallel and Scientific Computing Institute (PSCI), Sweden, April 2001.
Doctoral Thesis
- Software Techniques for Distributed Shared Memory by Zoran Radovic. Doctoral Thesis, Department of Information Technology, Uppsala University, November 2005.
Licentiate Thesis (Swedish 1/2 PhD degree)
- Hardware-Software Tradeoffs in Shared-Memory Implementations by Håkan Zeffer. Licentiate Thesis 2005-002, Department of Information Technology, Uppsala University, May 2005.
- Efficient Synchronization and Coherence for Nonuniform Communication Architectures by Zoran Radovic. Licentiate Thesis 2003-008, Department of Information Technology, Uppsala University, September 2003.
Master's Thesis
- Improving DSZOOM's Run Time System by Niklas Ekström. Master's thesis, UPTEC F03 104, ISSN 1401-5757, School of Engineering, Uppsala University, Sweden, January 2004.
- Simple and Efficient Instrumentation for the DSZOOM System by Oskar Grenholm. Master's thesis, UPTEC F-02-096, ISSN 1401-5757, School of Engineering, Uppsala University, Sweden, December 2002.
- DSZOOM--Low Latency Software-Based Shared Memory by Zoran Radovic. Master's thesis, UPTEC F-00-093, ISSN 1401-5757, School of Engineering, Uppsala University, Sweden, December 2000.