For a project involving diabetes management, we were asked to provide support for the logging of food intake for a long-running study. The project relates to techniques for the dietary assessment of carbs intake in relation to physical activity data.
Background
Overall, we think that existing food logging apps based on textual forms, potentially enriched with food databases have some fundamental limitations in terms of registration burden and registration accuracy. Specifically, even the most mature dedicated apps like MyFitnessPal (1) require a significant number of click- and type interactions and (2) are significantly optimistic on user capabilities to estimate the sizes/weights of their portions.
Our Intended Contribution
Therefore, our data science team decided to investigate a novel approach based on computer vision. In particular, we aim to apply techniques such as the ones delivered by the EU project GoCarb, and related approaches published in the https://madima.org/ workshop series.
Prototype
As of late 2019, we provide GameBus support for:
- uploading a left-to-right video of a meal, which covers the meal from multiple angles in order to facilitate the reconstruction of a 3D model, and
- allowing the user to enter his/her personal “guess” of the total weight of the meal, and
- allowing the user to enter his/her personal “guess” of the weight of the carbs in the meal.
Demo
Instructions for Users
Also also demonstrated in the demo video, we aim to instruct participants to our studies to put a reference card next to their meal such that our computer vision algorithms have a reference for calculating the size of the objects recorded.