This work compares data-driven knowledge distillation (DDKD) against zero-shot inference and fine-tuning on out-of-domain D2T data, and introduces structure-preserving augmentation via structural subsampling and perturbation in cross-domain D2T generation.
Abstract
Structured data exists in many forms (tables, knowledge graphs, charts, and time series), and converting it into text may involve different generation tasks. However, most prior work on data-to-text (D2T) generation has focused on specific tasks and datasets, relying either on task-specific training data or on the zero-shot capabilities of large language models. We study cross-domain D2T generation in a setting where neither in-domain training text nor test references are available, and where domains, generation goals, and input structures vary substantially. We compare data-driven knowledge distillation (DDKD) against zero-shot inference and fine-tuning on out-of-domain D2T data, and introduce structure-preserving augmentation via structural subsampling and perturbation. Experiments on five benchmarks show that, at constant model size (1.7B parameters), DDKD consistently outperforms both fine-tuning and zero-shot inference. Moreover, the resulting small models outperform a much larger finetuned model on two of the five domains, achieving comparable performance on the remaining three. We further construct QUINTD-5, a fivefold extension of QUINTD-1, and show that simply scaling real target-domain inputs yields only modest gains, whereas our augmentation strategy remains more effective and more cost-efficient for cross-domain distillation.
This paper proposes an EAG (En-rich, Aggregate, and Generate) framework, a simple but efficient LLM-based three-stage biomedical D2T approach in low-resource scenarios, and conducts extensive evaluations of closed-source general LLMs, open-source general LLMs, and open-source medical LLMs.
Yupian Lin, Guangya Yu, Yuang Bian et al.· Annual Meeting of the Associ...· 0 citations
Treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail is supported.
Nhat Thanh Vu, M. Rashid, Fariza Sabrina· Electronics· 0 citations
A hybrid approach that combines contextual embeddings from RoBERTa with handcrafted stylometric features to capture both high-level semantic patterns and low-level stylistic cues enables robust generalization across heterogeneous text types is adopted.
Britt van Leeuwen, S. Bhulai, R. V. D. Mei· 0 citations
This work proposes to synthesize alignment sequence pairs and fine-tune an encoder model with span alignment objective and introduces EXP - the first benchmark for explicit evaluation of label projection, thereby reducing confounders and non-determinism in method assessment.
Thang Le, Huy Huu Nguyen, A. Luu et al.· Annual Meeting of the Associ...· 0 citations
This work proposes a novel meta-learning setting that avoids model inversion by jointly leveraging pre-trained models and unlabeled data and introduces a task-weighting mechanism based on task confidence and class distribution balance to ensure effective meta-learning.
Lei Sun, Yusuke Tanaka, Tomoharu Iwata· 0 citations
Synthetic tabular data is increasingly used to support data sharing and machine learning development, where access to real data is restricted or sensitive. However, existing tabular data generators often require dataset-specific training or fine-tuning, data preprocessing, and extensive hyperparameter tuning to achieve plausible results. In addition, existing models may still struggle with mixed data types, modeling complex feature relationships, and pose a privacy risk for information leakage. With the rise of Large Language Models (LLMs) and their expanding capabilities across many domains, we investigate whether few-shot LLMs can be used in a training-free setting as tabular data generators. We propose FewShotTabLLM, a few-shot, training-free generation framework for tabular data that leverages schema-enriched context, representative sampling, and dataset profiling. We evaluate our approach across five widely used tabular benchmark datasets and compare it against strong training-based baselines from different model families (CTGAN, TVAE, TabDDPM, BeGReaT). A three-axis evaluation protocol has been used that covers representativeness, realism, and privacy. Across all evaluations, FewShotTabLLM outperforms baselines and surpasses them on many datasets, achieving realism, boundary adherence, and privacy preservation across all experiments. Overall, the results suggest that training-free fewshot LLM synthesis can serve as an alternative to existing models and a prominent research direction for improving dependency preservation and privacy-aware generation. Our code is available at: https://github.com/BMW-InnovationLab/FewShotTabLLM
Hadi Koubeissy, Michel El Khoury, Marc Kamradt et al.· Annual International Compute...· 0 citations
