Data and code for: Are fuzzy rule-based classifiers calibrated? A cross-family audit and a rule-preserving recalibration

Abstract

Supporting code and results for the article Are fuzzy rule-based classifiers calibrated? A cross-family audit and a rule-preserving recalibration by S. Sahay (School of Computer Science and Engineering, Vellore Institute of Technology, Chennai). The study measures the calibration of the class probabilities of ten classifiers, six of them fuzzy (the type-1 and interval type-2 genetic fuzzy rule bases and the FERL rule learner of ex-fuzzy 3.3.0, FURIA, a zero-order Takagi-Sugeno-Kang classifier and fuzzy k-nearest neighbours) and four standard learners, on 48 OpenML classification datasets with ten repeated train/calibration/test splits (60/20/20). It recalibrates the type-1 rule base with temperature, Platt, isotonic, beta and Dirichlet-style calibration and with rule-preserving recalibration (RPR), a convex ridge-penalised refit of one non-negative weight per rule, and audits calibration rule by rule with resampling floors and a per-rule test. Contents frbc_calibration_code_and_results.zip: complete analysis code (Python; code/run_all.sh, code/requirements.txt, seeds and pipeline in code/README.md), every per-run result (one row per configuration, model, dataset, repeat and method; one row per rule of every rule base; conformal sets; risk-coverage curves; cross-model rule-level audit; per-rule tests), all statistics (Friedman, Iman-Davenport, Nemenyi, Wilcoxon-Holm, rank-biserial, Cliff's delta, bootstrap intervals, Bayesian signed-rank probabilities), every figure (PDF, SVG, PNG) and table (CSV, LaTeX), the decision log and the scripts that build the article's tables, supplement and Fig. 1. per_dataset_main.csv: repeat means per model, method and dataset, the unit of every statistical test in the article. datasets_selected.csv: the 48 datasets with OpenML IDs, sizes, class counts and source suites. DATA_DICTIONARY.csv: definition of every column of the result files and of the model and method codes. README.md: folder guide, map from article figures and tables to files, and reproduction steps. The datasets are public on OpenML and are not redistributed; the code downloads them by ID. Seeds derive from SEED_BASE = 20261009, so every split and model can be regenerated. Data, results, tables and figures are licensed CC BY 4.0; the source code is licensed MIT (see LICENSE.md in the archive).