2018· International Journal of Artificial Intelligence & Digital Transformation· Vol 1, pp. 01-08· 0 citations
TL;DR
This paper provides a comprehensive framework for developing data infrastructures tailored to the demands of Generative AI applications, and analyzes existing solutions and proposes best practices for designing architectures that ensure efficiency, scalability, and reliability.
Abstract
The rapid advancement of Generative AI models has underscored the necessity for robust and scalable data infrastructures capable of managing vast datasets and complex computational requirements. This paper explores the unique challenges encountered in building such infrastructures, including data acquisition, storage, processing, and real-time access. We analyze existing solutions and propose best practices for designing architectures that ensure efficiency, scalability, and reliability. By examining case studies and current industry practices, the paper provides a comprehensive framework for developing data infrastructures tailored to the demands of Generative AI applications.
The resulting framework provides a vendor-neutral architectural blueprint that systematically connects scalability requirements, platform responsibilities, operational controls, and AI lifecycle needs, enabling engineering teams to evaluate and evolve data and AI platforms incrementally.
Santoshi Maddali· World Journal of Advanced Re...· 0 citations
This paper examines Data Agents from a harness-centric perspective, introducing a taxonomy of Data Agents and associated data environments, and identifying four open reliability problems: inactive semantic calibration, missing clarification, missing experience transfer, and the missing verification-repair repository.
Hua-Chi Zhou, Yu-Jing Zhang, Jia-He Du et al.· 0 citations
The evolution of machine learning has progressively changed where intelligence resides in an AI system. In conventional machine learning the task, data representation, labels and model architecture were tightly coupled, so data preparation was narrow, schema-bound and visible. Generative AI decouples the model from any...
Abbas Ali, Muhammad Ajmal Siddiqui, Moona Zahid· 0 citations
Scientific datasets are most compatible with AI workflows when they are described by rich, machine-readable metadata and provenance information, following the FAIR guiding principles. The FAIR Open-Science Extensible Data Exchange Network (FOXDEN) is a set of cyberinfrastructure building blocks developed at the Cornell...
V. Kuznetsov, Werner Sun, Keara Soloway et al.· 0 citations
The paper argues that multi-tenancy must be treated not merely as a virtualization problem but as a data-governance, workload-management, and responsible-AI problem, and provides a foundation for scalable AI data lakes while identifying limitations related to resource interference, governance complexity, data heterogen...
Arjun Mehta, Priya Sharma· International Journal of Adv...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
