Notes for CSC 2/458, 24 April 2019 Transactional Memory Background Herlihy and Moss, ISCA'93 Shavit & Touitou, PODC'95 (one of several important theory papers from that era) Harris & Fraser, OOPSLA'03 Herlihy, Luchangco, Moir, and Scherer, PODC'03 Rajwar & Goodman, MICRO'01, ASPLOS'02 Martinez & Torrellas, ASPLOS'02 Problems with locks forget to lock; forget to release deadlock, priority inversion lack of composability * complexity/concurrency tradeoff also preemption, page faults, thread failure, but that's an orthogonal set of issues TM addresses the first bunch of these with higher-level abstraction. A few implementations address the latter as well (by being nonblocking). Basic idea borrowed from DB community. Merger of two key ideas programming model based on high-level atomic blocks implementation based on speculation HW v. SW all things equal, HW is faster, but HW policies are hard to change SW policies can be more complex, more dynamic legacy machines need SW HW impls need SW fall-back hybrids best effort acceleration Representative SW implementations eager, undo, R-W locking, object-based (Ennals, McRT, Bartok) keep time-sorted write log, undo in reverse order eager, NB, cloning, incrementally validating, object-based, obstruction-free, vis or invis readers (DSTM) lazy, redo, time-based, stripe (or object) based, read-post-validating TL2 abort if location has been modified since transaction started TinySTM perform full validation and update of start time instead lazy, redo, non-locking, commit-serializing RingSTM low and high watermarks in (conceptually unbounded) list remember high as of start time; double-check on read; validate if nec. keep read and write filters commit by enqueuing write filter NOrec single global sequence lock value-based validation hashed write set for RAW lookups naturally privatization safe --------------------- Dimensions to the design space (incomplete list) blocking/nonblocking (lock-free v. obstruction-free) OSTM was lock free; several others obstruction-free; most blocking object-based/stripe-based buffering redo/undo/cloning access tracking & conflict resolution eager/lazy mixed: WW eager, RW lazy both reader & writer can commit if reader goes first visible readers/invisible readers/partially-visible readers (SNZI) validation underappreciated by some early systems; implement with incremental open-time checks post-read time-based checks (ala TL2) commit counter heuristic (Spear et al.): skip validation if no one has committed writes sandboxing contention management (maters mainly in the eager case) lots of options, based on random backoff, who started first, who is farther along, who has aborted the larger # of times also who wins, and whether the loser aborts or retries have to deal with starvation -- grab global lock eventually? can force lock-freedom (Guerrouai et al.) --------------------- Hardware lots of proposals six (publicly known) real implementations: Azul, Rock, BG/Q, Haswell, zEC12, p8 Representative HW implementations eager, responder loses (livelock?), with speculative stores kept in Write buffer (Rock) L1 cache (Azul, Haswell) "gathering" cache below L2 (z) Speculation buffer (H&M, ASF, p) L2 cache (L1 pass-through), multiversioning, SW chooses winner (BG/Q) Fully lazy, with central arbiter (TCC) distributed commit protocol (TCC2) continuous conflict monitoring (FlexTM) Interesting wrinkles HLE on Intel constrained txns on z non-txnal loads and stores both, eagerly, on ASF stores, lazily, on z suspend/resume on p ============================ Semantic challenges ---------------------------------------- Privatization: deals with non-txnal access after txnal access. Example: atomic { atomic { if (visible) visible = false if (p != null) } if (p != null) p = p->next p = p->next < abort > Here thread 2 may suffer a memory violation if the code interleaves as shown and it doesn't realize fast enough that the commit in thread 1 is going to force it to abort. This is the DOOMED TRANSACTION manifestation of the privatization problem. Similarly: atomic { if (visible) if (p != null) p = p->next < commit > atomic { visible = false } if (p != null) < clean up > p = p->next } If thread 2 is using a redo log, thread 1 may not see the update of p in time, and may suffer a memory violation. This is the DELAYED CLEANUP manifestation of the privatization problem. There are symmetric versions of both doomed transaction and delayed cleanup for systems with an undo log. ---------------------------------------- Publication: related problem at the _beginning_ of the transaction, but arises only in programs with data races. | Example: | | // data == 42, ready == false, val == 0 | | atomic { | tmp = data | data = 1 | atomic { | ready = true | } if (ready) | val = tmp | } | | With traditional critical sections, val can never end up being 42. | With most current STMs it can, if threads interleave as shown. | | [ Recall that memory models typically require a global | "happens-before" among special synchronization ops, and between | special ops and ordinary ops in the same thread. In release | consistency, the special ops are acquire and release instructions. | In the Java MM, they are synchronized block enter/release and | volatile load/store. A race is a pair of conflicting accesses not | ordered by happens-before. ] | | The program above has a race between txnal and non-txnal code, so | maybe we just say "don't do that"? | | But consider: | | // data == 42, ready == false, val == 0 | | data = 1 atomic { | atomic { if (ready) | ready = true val = data | } } | | Here there is no race in the source code, but the compiler may | _introduce_ a race by speculatively hoisting the read of data out of | the conditional. With traditional memory models, the compiler has | to introduce a WBW fence in thread 1 to prevent the race. What is | the analagous solution for transactions? | | The publication problem can be solved by providing | STRONG ISOLATION (aka strong atomicity), but that requires HW | support or unacceptable overhead on nontransactional accesses. | | An attractive alternative to strong isolation is to say that | race-free programs display single lock atomicity (SLA), and are | publication and privatization safe. Publication-safe means the | implementation respects happens-before between a non-txnal access | and a subsequent txnal access. Privatization-safe means the | implementation respects happens-before between a txnal access and a | subsequent non-txnal access. | | Two questions arise | (1) What sorts of behavior are permitted for racy programs? | Java limits (and defines) the possibilities; C has "catch fire" | semantics (realistically: "thin-air" values). | (2) Must publicizing and privatizing transactions be explicitly | identified by the programmer? If so, the implementation of | non-labeled txns might be cheaper, at least for "catch fire" | languages. | | Menon et al. propose three progressive relaxations of GLA: | DLA (disjoint) -- as if every location had a separate lock, acquired | presciently at beginning of txn | ALA (asymmetric) -- like DLA, but read locks can be acquired lazily | ELA (encounter-time) -- like ALA, but write locks can be acquired lazily | | For race-free programs, DLA and ALA provide the same semantics as GLA. | For racy programs, they introduce new possible (but well-defined) | behaviors. | ELA provides the same semantics as GLA only if the compiler refrains | from speculative hoisting. | | Which of these we should have (if "catch fire" isn't ok) is still an | open question, as is how to implement them efficiently. ---------------------------------------- Fundamental semantics Proposal for C++ is basically an explanation based on locks. We have proposed an (almost but not quite equivalent) model. At the program level: implementation is correct if we have TSC (apparent global serialization order for transactions) for TDRF programs (all conflicting accesses inside transactions). At the runtime level, implementation is correct if: - memory model - conflict function - (optional) arbitration function st - we can prove that every execution has a sequential equivalent (the "fundamental theorem of TM") - we can prove minimal liveness -- specifically, try-commit fails only in the presence of some conflicting txn - a read r in an unsuccessful txn T is inconsistent with previous reads of T only if there exists some other txn S whose prefix prior to r conflicts with T (giving T an excuse to abort) - zombie executions are bounded: if T can make k more steps, it can do so consistently - exceptions (if any) never escape an unsuccessful txn ---------------------------------------- irreversible operations for correctness (e.g. read-after-prompt interactive I/O) for speed (avoid instrumentation in inevitable txn) implement via global read/write lock no concurrency global write lock allows concurrent readers, but inev. txn must acquire have to instrument inev. reads to wait for concurrent writers to abort GWL + fence avoids inev. read instrumentation transactional "drain" irreversible txn as "writer" other writers as "readers" inevitable read locks allows concurrent writers, but forces read instrumentation inevitable read Bloom filter cheaper way to allow concurrent writers commit order == serialization order == cleanup order (RingSTM) ---------------------------------------- condition sync retry ---------------------------------------- nesting subsumption closed semantically necessary if you have cancel() how to tell what level you conflict with? open admits circularity x == y == 0 A B atomic { atomic { open atomic { open atomic { x = 1 y = 1 } } u = y v = x } } u == v == 1 // neither outer txn came first ---------------------------------------- exceptions error v. non-local return ---------------------------------------- debugging how do you see what's inside something indivisible? ---------------------------------------- additional compiler challenges viral character of "txn-safe" elide operations validation memory fences on relaxed order machines instrumentation on provably non-escaping values track irreversible operations require programmer to label? what's the default: reversible or irreversible? if the former, what do you do about legacy libraries? if the latter, you have to label a LOT efficient sandboxing maybe track "transactional state" of references clone subroutines based on states of parameters maybe repair / partial rollback ======================================== Hybrid systems Big challenge: HW txn needs to "subscribe" to SW lock to avoid accidental commit.