LLM application in digital asset assessments: mixed methods research

Abstract

Digital asset markets are expanding but remain difficult to evaluate, particularly for retail investors who face information overload, unstructured disclosures, and limited regulatory safeguards. Traditional due diligence methods often fail to capture the complexity of blockchain-based projects, exposing investors to misrepresentation and fraud. At the same time, LLMs have shown remarkable results on analyzing large information quantities. Thus, this paper investigates whether large language models (LLMs) can enhance digital asset assessments by providing inspectable, automated, and low-cost decision support. Using a Design Science Research methodology, we developed a framework and a prototype tool to apply LLM-based evaluation to token-offering whitepapers, evaluated across 1,139 generated runs covering 29 token offerings. We compared different content processing approaches, from raw document analysis to aspect-focused summarization, and tested prompting strategies including Zero-Shot, Few-Shot, and Chain-of-Thought reasoning. Our findings show a nuanced pattern. On aggregate quantitative metrics, structured summarization combined with Chain-of-Thought prompting achieves the lowest rating error, although differences in rank correlation between configurations are not statistically resolvable given the number of projects evaluated; aspect-focused summarization structures criterion-specific evidence and candidate red flags explicitly, but its rating error is higher and the interpretability of this representation was not independently evaluated. To address the single-model limitation, we added a lightweight multi-model robustness check using two small open-source models on a matched subset of cases; the results suggest that the observed optimism bias is not specific to the original closed LLM, although performance still varies across model configurations. Overall agreement between model and expert ratings remains limited, which suggests that LLMs are better understood as decision-support tools for screening and explanation than as substitutes for expert judgment. The study therefore contributes a design-oriented due diligence artifact and a dependence-aware evaluation of how summarization and prompting choices affect calibration, output-level inspectability, and validity boundaries in digital asset assessment.