Transfer Learning for Food Calorie Estimation and Health Assessment

Abstract

Although the new vision and multimodal models have enhanced food recognition, accurate calorie estimation is still challenging because it needs to consider more than just the category of food; it also includes factors such as the quantity of food and ingredients. A multi-agent Food Health Assessment framework for food calorie estimation is presented in this paper, and it is investigated whether transfer learning can be used to improve the final calibration without retraining the entire perception backbone. The new system has added visual-language understanding, ingredient-level segmentation, depth-aided portion cues, contextual calorie reasoning, and a trainable calorie regression head. Nutrition5K and MetaFood3D are evaluated with the same multimodal perception pipeline using the same protocol for both the trained model and the internal heuristic baseline. According to the experimental results, Version 6 has reduced the test mean absolute error (MAE) from 394.5 kcal to 74.4 kcal in Nutrition5K and from 411.4 kcal to 117.8 kcal in MetaFood3D. Although light transfer learning has shown some promise, a more systematic regression design is still needed to improve the robustness of cross-domain calorie estimation.