Pipeline parallelism (PP) is widely adopted for distributed training of neural networks, but its efficiency is limited by pipeline bubbles that leave devices idle. A variety of schedules have been proposed to reduce these bubbles, yet most are hand-crafted, rely on idealised timing assumptions as well as rigid structural constraints, and cannot adjust to memory limitations. Moreover, stage partitioning, task scheduling, and memory management are often optimised in isolation, despite being tightly interdependent. In this paper, we present OptPipe, a unified framework that jointly optimises partitioning and scheduling for pipeline parallelism. We introduce a memory-aware directed acyclic graph (DAG) that captures both task dependencies and the lifetime of intermediate tensors, enabling explicit reasoning about the trade-off between execution efficiency and memory usage. Building on this representation, we present a mathematical formulation for pipeline parallelism that simultaneously determines stage partitioning, device assignment, task ordering, and memory feasibility. Unlike prior methods, our approach imposes no structural constraints on pipeline configurations and adapts naturally to a wide range of memory budgets and workload characteristics. Experiments on GPT-3 and LLaMA-3.2 show that OptPipe improves training throughput by up to 72.5%, 37.8%, and 38.0% over 1F1B, Chimera, and ZB-V, while remaining feasible under memory budgets where these baselines fail.
The findings of the study indicate that a large number of women in Bangladesh are suffering from being overweight or obese, and multiple factors are responsible for this including, older age,being from wealthy households, higher education, being from food-secured households, watching TV at least once a week, and being...
Haribondhu Sarma, N. Saquib, M. Hasan et al.· BMC Obesity· 75 citations· ⚡5
The rapid growth of scientific literature makes it increasingly challenging for researchers to identify novel and impactful ideas, especially across disciplines. Modern artificial intelligence (AI) systems offer new opportunities for scientific ideation, but how compelling are AI-generated ideas, and how can their qual...
Xuemei Gu, Mario Krenn· Proceedings of the National...· 65 citations
O Ongoing pro-poor health-financing strategies should be strengthened with introduction of innovative ways to monitor access, equity, and quality of care in maternal health.
Herfina Y. Nababan, M. Hasan, Tiara Marthias et al.· International Journal of Wom...· 64 citations· ⚡2
Agile methods continue to gain popularity. In particular, the Scrum method appears to be on the verge of becoming a de-facto standard in the industry, leading the so called Agile movement. While there are success stories and recommendations, there is little scientifically valid evidence of the challenges in the adoptio...
A. Marchenko, P. Abrahamsson· Agile Conference· 59 citations· ⚡11
Findings demonstrate a strong positive association between mother’s education and IYCF indicators and suggest strengthening national policies to educate women at least to the secondary level in SACs might be a cost-effective intervention for improving IY CF practices.
M. Tariqujjaman, M. Hasan, M. Mahfuz et al.· Nutrients· 45 citations· ⚡4
A comprehensive taxonomy of the challenges faced when a medium-scale organization decided to adopt software platforms is provided, namely: business challenges, organizational challenges, technical challenges, and people challenges.
Yaser Ghanam, F. Maurer, P. Abrahamsson· Information and Software Tec...· 41 citations· ⚡3
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
MIT News · Artificial Intelligence· news.mit.eduJul 15, 2026
Assistant Professor Pat Pataranutaporn describes a new interface that lets everyday users glimpse inside an AI's neural network before their chatbot ever says a word.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.