PFN-Syn achieves the highest average downstream utility among evaluated synthetic generators, with higher generation throughput than training-based baselines, while maintaining competitive distributional similarity and distance-to-closest-record privacy behavior.
Synthetic tabular data is increasingly used to support data sharing and machine learning development, where access to real data is restricted or sensitive. However, existing tabular data generators often require dataset-specific training or fine-tuning, data preprocessing, and extensive hyperparameter tuning to achieve plausible results. In addition, existing models may still struggle with mixed data types, modeling complex feature relationships, and pose a privacy risk for information leakage. With the rise of Large Language Models (LLMs) and their expanding capabilities across many domains, we investigate whether few-shot LLMs can be used in a training-free setting as tabular data generators. We propose FewShotTabLLM, a few-shot, training-free generation framework for tabular data that leverages schema-enriched context, representative sampling, and dataset profiling. We evaluate our approach across five widely used tabular benchmark datasets and compare it against strong training-based baselines from different model families (CTGAN, TVAE, TabDDPM, BeGReaT). A three-axis evaluation protocol has been used that covers representativeness, realism, and privacy. Across all evaluations, FewShotTabLLM outperforms baselines and surpasses them on many datasets, achieving realism, boundary adherence, and privacy preservation across all experiments. Overall, the results suggest that training-free fewshot LLM synthesis can serve as an alternative to existing models and a prominent research direction for improving dependency preservation and privacy-aware generation. Our code is available at: https://github.com/BMW-InnovationLab/FewShotTabLLM
Hadi Koubeissy, Michel El Khoury, Marc Kamradt et al.· Annual International Compute...· 0 citations
Synthetic data, i.e., data simulated from some statistical model, are an important tool for both privacy protection and artificial intelligence pipelines. In the privacy context, synthetic data enable agencies to disseminate record-level information while reducing disclosure risks. In the artificial intelligence context, synthetic data allow analysts to augment training sets, increase coverage of rare cases, and support experimentation when genuine data are scarce. For each usage, we discuss key considerations and methods for generating synthetic data, including sequential modeling, differentially private synthesis, deep generative models, and large language models. Throughout, we highlight key trade-offs between data usefulness, privacy protection, and model reliability. We conclude by outlining some open research challenges and future directions for synthetic data development.
Yinyihong Liu, Jerome P. Reiter· Annual Review of Statistics...· 0 citations
Differential privacy (DP) imposes fundamental trade-offs between privacy and statistical fidelity in synthetic data generation. While access to public data has been shown to improve these trade-offs empirically, existing approaches use public data only indirectly, through pre-processing (e.g., using pre-trained generative models) or post-processing steps (e.g., matching target statistics estimated from public datasets), while relying on domain-agnostic DP mechanisms. In this work, we lay the theoretical framework to study the principled incorporation of public data into DP mechanisms themselves. We consider normalized histograms as distribution estimators and characterize the asymptotically optimal domain-aware privacy mechanism within a specific class of DP mechanisms. We introduce PubMix, a public-data-aware DP mechanism that can be used in histogram-based data synthesis pipelines. Our experiments demonstrate that PubMix significantly improves synthetic data generation quality compared to domain-agnostic privacy mechanisms.
Sajani Vithana, Sangwon Jung, Haoyang Hu et al.· 0 citations
Machine learning pipelines are increasingly deployed in high-stakes domains where robustness against data inconsistencies, bias, and distributional shifts is crucial. However, real-world datasets often suffer from limitations such as data scarcity, class imbalance, and privacy constraints. Synthetic data generation has emerged as a promising strategy to overcome these challenges and enhance pipeline robustness. This paper presents a comprehensive survey and analysis of synthetic data generation techniques, classifying them by data modality, generation method, and application purpose. We explore how synthetic data contributes to the resilience of ML pipelines against failure modes such as concept drift, noise, and adversarial attacks. Through empirical case studies across domains, we demonstrate the practical benefits and limitations of integrating synthetic data into training and evaluation pipelines. Finally, we discuss ethical considerations and outline future research directions toward building more robust, fair, and privacy-preserving ML systems using synthetic data.
Dennis M. Ritchie, Allen Newell· International Journal of Dat...· 0 citations
This work demonstrates that PFNs, which are originally trained for classification, successfully outperform classification baselines on ranking data and introduces a novel sampling and inference scheme to obtain pairwise predictions from PFNs' native pointwise architecture, analogous to pairwise LTR.
David Vos, Samarth Bhargav, M. D. Rijke et al.· International Conference on...· 0 citations
SynPre-FL is proposed, a unified framework combining high-fidelity synthetic EHR generation with synthetic-pretrained FL for robust prediction under non-IID conditions and provides a practical and reproducible framework to enable privacy-aware, interpretable, and robust clinical prediction from distributed tabular EHR data.
Akarsh K. Nair, Muhammad Arifur Rahman, N. Shopland et al.· 0 citations
