Seeing the knowledge graph: a multimodal vision–language framework with active learning for cold–start course recommendation
TL;DR
ALARM is proposed, a multimodal active–learning framework pairing a frozen large language model (LLM) that encodes course and learner text, a vision transformer (ViT) that “observes” the KG by reading its rendered subgraph image, and a domain–specific acquisition function that selects which labels to query.
Abstract
: Course recommendation on online learning platforms operates under two constraints at once: a chronic coldstart regime that breaks collaborative filtering, and the high cost of reliable preference labels. Existing remedies treat the course knowledge graph (KG) as a symbolic object for graph neural networks or side information, discarding a rich signal: the spatial topology of a KG—which concepts cluster, chain, or sit at hubs—is naturally a visual object that modern vision encoders are built to read. We propose ALARM, a multimodal active–learning framework pairing a frozen large language model (LLM) that encodes course and learner text, a vision transformer (ViT) that “observes” the KG by reading its rendered subgraph image, and a domain–specific acquisition function that selects which labels to query. Fusion aligns ViT patch tokens with LLM hidden states in a shared retrieval space, and only lightweight LoRA adapters and projectors update per active–learning round, at roughly 0.08× the cost of full fine–tuning. On MOOCCube and a XuetangX subset,
