DSZOOM--Low Latency Software-Based Shared Memory

Abstract

Software-implementations of shared memory are still far behind the performance of hardware-based shared memory and are not viable options for most fine-grain shared-memory applications. The major source for their inefficiency comes from the cost of interrupt-based asynchronous protocol processing, not from the actual network latency. As the raw hardware latency of inter-node communication decreases, the asynchronous overhead in the communication becomes more dominant. Elaborate schemes, involving dedicated hardware and/or dedicated protocol processors, have been suggested to cut the overhead. This thesis describes how all the asynchronous overhead can be completely removed by running the entire coherence protocol in the requesting processor. This not only removes the asynchronous overhead, but also makes use of a processor that otherwise most likely would be stalled. The technique is applicable to both page-based and fine-grain software shared memory systems. The proof-of-concept implementation presented in this thesis -- DSZOOM -- is a fine-grain software-based shared memory. It demonstrates a protocol-handling overhead below a microsecond for all the actions involved in a remote load operation, to be compared to the fastest implementation to date of around ten microseconds. The all-software protocol is implemented assuming only some basic lowlevel primitives in the cluster interconnect. The implementation is thread-safe and allows all processors in a node to simultaneously perform remote operations. Based on a remote atomic and simple remote put/get operations the requesting processor can take the role of the directory agent, traditionally assumed by a remote protocol agent in the home node in other implementations.

Available as PDF (314 kB)

BibTeX file entry: Radovic:2000:dec