Global Shared Memory on a Linux cluster
The ultimate goal of this project is to extend the shared memory
semantics of modern operating systems to enable applications on a
cluster to share pages accross machines.
1. The idea
Most modern Unix-like operating system (including Linux)
support shared memory through two different mechanisms: System V shared
memory, and mmap. The limitation of these two mechanism is that the
processes sharing a page must reside on the same machine. In this project we
propose a mechanism to allow processes on different nodes of a cluster to
share a page. One of the main differences between the proposed mechanism and
System V or mmap shared memory is found in the consistency model.
The latter use Processor Consistency (PC), while our system will
provide Lazy Release Consistency (LRC).
2. Design and Implementation
The proposed system should be comprised of two parts:
A set of C/C++ library routines: the routives will
provide applications with a set of functions to allocate, free and
manipulate shared memory segments.
A kernel module that will support Distributed Shared
Memory across a cluster of work stations (using LRC). The module will
make use of the algorithms developed in the Cashmere project.
Our main intention while implementing the system is to minimize
the modifications to the core of the Linux kernel. A second goal is to make
this system available only to applications who need distributed shared
memory. This way not every process in the system will have to pay the
overhead of this extra functionality.
The main additions to the system's kernel will be two data structures:
A per process virtual-to-global address translation
table. This will translate the process specific virtual page address
to a cluster-wide global page address, used by the rest of the system
to identify pages. This extra level of abstraction enables processes
to map the share segments in any virtual address.
A per node page directory. The page directory will be
indexed using the global address and will provide with the necessary
information to implement the consistency model.
Applications will co-operate with the kernel module through the
user-level library and the page fault mechanism. The library routines will be
used to ``register'' themselves with this system and manipulate shared memory
segments. The page fault mechanism will be used to track down page accesses,
and provide applications with the LR consistency model.
Registered applications will use the new page fault path, while
unregistered applications will behave in the standard way. Thus unregistered
applications will not have to pay the potential overhead of this new path.
We expect this system to span the time frame of this course, so
we have to set some short-term goals. The implementation of the two data
structures, the page fault mechanism and the user-level library routines (ie.
all the entry point to the protocol) is imperative so we plan on working on
these first. Next in order is a simple allocator-deallocator process that
will try to allocate and free global shared segments (a simple testbed app).
After this we're going to work on the consistency mechanisms for the shared
segments and implement the full protocol. We expect to finish the first part
of this project in this class, namely implementation of the entry points and
the testbed application.