Recent years have witnessed the proliferation of very fast devices for networking, I/O, and computing acceleration. Networks operate at speeds of 10 Gbps and transmit millions of packets per second. Commodity solid-state disks (e.g., the Intel Optane DC P4800X and Samsung PM1725a) can perform at or near 1 Million I/O operations per second. Integrated accelerators like GPUs can effectively offload computations that run just a few microseconds a time. Simultaneously, the effective end of single-core performance improvements and the proliferation of multi-core processors has necessitated architectures tuned for parallel I/O across multiple hardware threads. Specifically, it is now common for the hardware/software stack to embody a multi-channel I/O architecture in which each CPU context (hardware thread) owns a dedicated per-CPU command/completion queue pair, giving it independent channels for concurrent, fast I/O from mutually untrusting applications or virtual machines. Examples of this approach include the Windows and Linux NVMe drivers, the Linux multi-queue block layer, and recent SCSI Multi-Queue support. Unfortunately, it is difficult for the operating system to partition the resources among such resource principals (flows) fairly without introducing synchronization overhead that would largely negate the benefits of having multiple queues.
This proposal addresses the issue of resource management for multi-channel devices in a scalable fashion. In our preliminary work, we show how to extend the notion of fair queueing to multi-channel devices. Specifically, we describe a scalable implementation that bounds potential unfairness while minimizing synchronization overhead. A key principle of our design is to localize synchronization (intra-core rather than inter-core; intra-socket rather than inter-socket) as much as possible by forming cohorts of channels that interact locally, hence reducing the frequency of updates to global data.
Our future direction includes enhancing the software stack to utilize per-channel interrupts via user-level interrupt delivery and protecting kernel-bypass stacks from application bug/malice.