Abstract: Resource management on large modern clusters is a growing problem due to the increasing scale of machines and applications, and complexity of hardware. Modern systems offer abundant compute resources to exploit application level parallelism. The multiple compute cores, however, share resources such as caches, interconnects, and memory at some level, with the potential for resource contention resulting in unpredictable and often significantly degraded single thread performance. Moreover, the degree of resources shared between different cores is non-uniform, often depending on the cores involved. Critically, worker threads that share data need to be placed in close proximity with respect to the memory hierarchy in order to streamline communication. At the same time, the benefits of streamlined communication due to proximity should not be negated by additional resource contention.
Modern applications are increasingly dynamic, handle more data, demand higher resources, and seldom exhibit steady characteristics. Emerging data centers and cloud services provide these applications with access to large-scale resources, benefiting from economies of scale. State-of-the-art resource managers are typically pessimistic in nature, requiring per-application upfront resource reservation. While this provides deterministic guarantees for application performance, the approach might leave resources idle, resulting in inefficiencies. In this proposal, we argue that it is possible to provide efficient and deterministic performance on large-scale clusters using a guided optimistic resource management. Optimistic resource management allows users to utilize resources freely as required. While an optimistic approach results in more efficient utilization, interference on and contention for resources may result in overall performance degradation. Guided optimism uses aggregate performance measurements to reduce resource contention while retaining the high resource utilization of the optimistic approach. Using node-local, aggregate bottleneck information obtained from low-level hardware performance counters, guided optimism further improves performance by appropriate data and compute placement (resulting in potential reduction in resource contention), as well as control of resource allocation based on the desired quality of service. We will demonstrate the benefits of the guided optimistic approach on individual compute nodes and in cluster/cloud environments.