Fuzzy Random Forest: Integrating Fuzzy Set Theory for Enhanced Imbalanced Classification

Abstract

Standard Random Forest algorithms assume crisp class boundaries and precise feature values, limitations that become critical when dealing with ambiguous or overlapping data patterns common in imbalanced datasets. This paper presents Fuzzy Random Forest (FRF), a novel ensemble method that integrates fuzzy set theory into the tree construction process. Unlike existing methods that rely on crisp partitions, FRF extends the classical Gini impurity and Shannon entropy to fuzzy-valued impurity measures. Rather than counting instances as belonging entirely to one class, FRF weights each instance’s contribution to a node by its fuzzy membership degree, so that the resulting fuzzy class probabilities, and the fuzzy impurity and information-gain measures built from them, capture the ambiguity of borderline cases instead of forcing a crisp assignment. We provide complete mathematical derivations, including proofs of boundedness, crisp reduction, and membership conservation properties. Empirical evaluation on eight imbalanced datasets demonstrates that FRF consistently outperforms standard Random Forest, with particularly notable performance on datasets with high uncertainty and ambiguous class boundaries. On a credit risk dataset (80% non-default, 20% default), FRF achieves exceptional precision (0.99) for minority class predictions while maintaining 89.00% overall accuracy. Statistical validation using McNemar’s Test confirms FRF’s superiority over standard Random Forest, with a chi-square statistic of 100.0 (p less than 0.0001), a 20.6% improvement in information gain, and a 42.9% reduction in entropy reduction variance. FRF is particularly suited for applications requiring high precision in minority class identification and interpretable fuzzy decision rules, such as medical diagnosis, fraud detection, and credit risk assessment where regulatory compliance demands transparent decision-making.