Reliabilityand Availability of a Repairable k-Out-of-n System via Event-Triggered Maintenance Semi-Markovian Modelling Reinforcement Q-Learning, and Copula Repair

Abstract

This paper develops a reliability and availability model for a repairable two-tier k-out-of-n computer network system. Four analytical components are combined: event-triggered repair intensification, Gumbel–Hougaard copula-based restoration, a semi-Markovian state-transition formulation, and a reinforcement Q-learning layer for adaptive threshold selection. Subsystem 1 consists of n identical web servers operating under a k-out-of-n: G policy. Subsystem 2 contains three identical database replicas under a one-out-of-three: G policy. The full system is available only when both subsystems and both load balancers are operational. State probabilities are derived via the supplementary variable technique and Laplace transforms. The semi-Markovian extension decouples embedded-chain transition probabilities from state-specific holding-time distributions. Stationary availability is expressed as the weighted ratio of mean up-state sojourn times to total mean sojourn times, accommodating non-exponential failure and repair time laws. The event-triggered policy fires repair intensification when the system availability A(t) falls; specifically, when the degradation indicator Ξ(t)=1−A(t) crosses a threshold subject to a minimum inter-event time. A Q-learning agent selects the threshold–gain action from a finite set, updating a Q-table by the Bellman rule. Numerical results from MATLAB 2026 show baseline availability converging to 0.8581, 0.8537, and 0.8496 for k=5, k=15, and k=25. Without repair, reliability reaches 0.0720, 0.0521, and 0.0398 at t=25. The fixed event-triggered policy generates two trigger events per configuration, achieving area-under-the-curve (AUC) gains between 0.5412% and 0.5652%. The Q-learning policy generates three trigger events and achieves larger AUC gains for the stricter configurations: 0.7154% for k=15 and 1.0066% for k=25. These gains are validated by 30 Monte Carlo replications with non-overlapping 95% confidence intervals for k=15 and k=25. Under Weibull aging with shape βw=1.8, semi-Markovian stationary availability is lower than the exponential Markov baseline for all values of k.