Skip to content
Book Open access

Computation Is Fast, Use Threadlet!: Efficient Threading for μs-Scale Computing via OS/Hardware Co-Design

Sep 2026 · Proceedings of the ACM SIGOPS 32nd Symposium on Operating Systems Principles · pp. 430-447 · 0 citations · 107 references

TL;DR

This work proposes threadlet, a new hardware abstraction that is lighter than a software thread by removing uncritical state, and builds a supporting architecture on an FPGA and a research OS for threadlet, outperforming prior works by 3× to 47× on real-world applications.

Abstract

The conventional threading model multiplexes software threads onto hardware cores. This model inherently suffers from the overhead of 1) context switching, 2) scheduling, and 3) system event notifications (e.g., I/O interrupts). As computing enters μs-scale, such overheads become the key bottleneck of a wide range of domains: networking, concurrency control, and microservice scheduling, etc. Existing software solutions, such as cooperative scheduling and polling, sacrifice correctness and/or CPU efficiency, while hardware offers only point solutions with suboptimal performance. To eliminate the threading overheads without introducing difficult tradeoffs, our key insight is that hardware should directly expose schedulable, basic units of execution (i.e., software threads) instead of computational resources (i.e., cores). We propose threadlet, a new hardware abstraction that is lighter than a software thread by removing uncritical state (e.g., floating-point registers). A core caches a few active threadlets in its register file, while spilling the rest to the memory hierarchy. Thus, context switching incurs minimal overhead by simply resuming execution with the next threadlet's state. Moreover, threadlet comes with hardware support that can express diverse OS scheduling policies, enabling fast and flexible scheduling. Finally, as a hardware abstraction, threadlets directly interface with microarchitectural mechanisms (e.g., cache coherence) for efficient event notifications. We build a supporting architecture on an FPGA and a research OS for threadlet, outperforming prior works by 3× to 47× on real-world applications. Our artifact is publicly available at https://github.com/TELOS-syslab/Threadlet.

Read PDF

Similar papers

Open access Oct 2026

Adaptive Thread Scheduling and Optimization Framework for the Multi-Core CPU with Large-Capacity Cache

The continuous growth in processor speed and the persistent memory latency bottleneck make large-capacity cache critical for mitigating the “memory wall”. However, topology-oblivious OS scheduling frequently induces severe cache contention and cross-die migrations, diminishing these hardware benefits. To address this,...

Xin-Ming Song, Mei-Kang Qiu, Hong-Jun Dai · 0 citations
Sep 2026

Exo-GPU: Safe, Imperative, User-schedulable Programming for Tensor Cores

Exo-GPU, an imperative, low-level language that creates minimal abstraction over CUDA, is proposed, to treat parallelism and synchronization as mere annotations on sequential code rather than as fundamental control flow primitives, enabling verification that these constructs do not alter the program semantics.

David Akeley, Yuka Ikarashi, Jonathan Ragan-Kelley · 2 citations
Book Open access Sep 2026

Scheduling Linux Threads under I/O Chiplet Wall Using cSwitch

Chiplet-based processors have become the dominant architecture for modern servers, partitioning functionality across compute chiplets and a centralized I/O chiplet. While this design improves scalability, it introduces a new bottleneck— the I/O Chiplet Wall—where off-chip memory and device requests contend along a shar...

Seunghyun An, Joontaek Oh, Ming Liu · 0 citations
Open access Sep 2026

Compute-Communication Overlap Is Not Free: A Cross-Layer Characterization in GPU LLM Workloads

Distributed large language model (LLM) systems rely on compute-communication (CC) overlap to hide communication overhead. However, scheduling compute and communication concurrently and maximizing overlap opportunities do not guarantee effective overlap. Resource contention can slow concurrent operations, while communic...

Jihwan Oh, Junkyum Kim, Seokjin Go et al. · 0 citations
Open access Sep 2026

Compiler-Assisted Operating-System Scheduler for Simultaneous Multithreading

Simultaneous multithreading (SMT) improves physical-core throughput by allowing multiple hardware threads to share an out-of-order backend, but its benefit depends strongly on which tasks are paired on sibling hardware threads. Existing OS schedulers are topology-aware, yet they lack task-specific microarchitectural in...

Joohyoung Cha, Ji-Ho Lee, Sehyeon Oh et al. · 0 citations
Book Open access Sep 2026

PLINK: A GPU-Initiated I/O Platform Exploiting NVMe Parallelism

PLINK (Parallel-Link), a GPU-initiated I/O platform that addresses this gap through three targeted optimizations: SQ/CQ decoupling to hide device latency, command-ID-indiscriminate polling to eliminate exhaustive CQ search overhead, and warp-level doorbell batching to minimize PCIe MMIO writes and reduce warp divergenc...

Jaewoong Jung, Hyukyul Kang, J. Choi · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.