How Cal 3D measures
The whole calculation, laid open: why a photo misses the portion, what Cal 3D does instead, where the method stops — and what we honestly do not know yet.
On this page
Why a photo misses the portion
An image has no scale. The same bowl of rice fills the same rectangle at two different distances — and the same mound weighs 120 or 280 grams depending on its height. A model that only sees a flat photo has to guess the missing scale.
How it guesses is well documented. The photo pipeline from which Cal 3D inherited its ingredient recognition was benchmarked on 28 weighed plates. The result: roughly 30 % average calorie error (29.9 % MAPE). What matters, though, is not the number but the pattern behind it.
Where the plates come from: Nutrition5k, a public dataset by Google Research in which every plate was weighed before it was photographed. We did not weigh these plates ourselves — we ran them through the inherited photo pipeline and compared its output against the known weights. The number therefore describes the approach Cal 3D sets out to replace, not Cal 3D itself.
The error is not scatter — it is a pull towards the middle
The model drags every plate towards 300 to 450 calories. Large portions read low, small ones read high:
| Plate | Actual | Estimated from the photo |
|---|---|---|
| large portion | 1,119 kcal | 435 kcal |
| small portion | 195 kcal | 277 kcal |
That this is no coincidence shows in the repeat: across two independent runs, 28 out of 28 plates landed on the same side of the truth. A random error spreads out; this one does not.
Why it follows that more AI does not help
A systematic error cannot be averaged away. Having several models vote, or improving the nutrition database, attacks variance — and variance is not the problem here. As long as the scale is missing, the pull towards the middle stays.
What Cal 3D does instead
Instead of guessing the scale, Cal 3D measures it. While you sweep around the plate for about three seconds, the app reads the camera’s depth data, finds the plane of the table and computes how much space the food occupies above it — the volume in millilitres.
Volume becomes weight through density:
Volume×Density=Weight
Volume — measured on the device, in mlDensity — from the nutrition database, in g/mlWeight — in grams
The model keeps what it is good at
A vision model reads composition well — what is on the plate and in what proportion. Only the absolute scale defeats it. So Cal 3D keeps its ratios and replaces the size alone:
- The model names the ingredients and their proportions.
- The app compares the measured total weight with the estimated one.
- Every ingredient is rescaled by the same factor.
- The corrected grams feed the nutrition table — calories follow from that.
So the part that previously contributed most of the error is not improved but replaced.
Where the densities come from
The calculation stands or falls with density. The nutrition database of the US Department of Agriculture has no density column — but it does carry household measures: “1 cup = 135 g”. And a cup is exactly 236.5882365 ml. Density follows from both.
That gives 2,814 of 8,644 entries a usable density value. Where an ingredient has none, the app does not carry on with an invented figure — it labels the result as estimated.
Spot checks against physical reference values
| Ingredient | Derived from USDA | Physical reference |
|---|---|---|
| Milk | 1.031 g/ml | 1.03 g/ml |
| Honey | 1.433 g/ml | 1.42 g/ml |
| Sugar | 0.845 g/ml | 0.85 g/ml |
| Olive oil | 0.947 g/ml | 0.92 g/ml |
On top of that a plausibility range of 0.05 to 2.0 g/ml applies — from popcorn to honey. Anything outside is discarded rather than used.
The same goes for every other number the app puts on screen — your calorie target, the macro split, the limits for fibre, sugar and salt. Each one has a published source, and they are all listed on the science page, together with the file in the app that implements them.
Measured or estimated — the app tells you
This is the part that matters most. Every entry carries one of two labels:
Measured — a usable depth measurement was available and the ingredients had density values. The number rests on a measurement.
Estimated — something of that was missing. The number then comes from the model’s estimate and carries its uncertainty.
In case of doubt the app always decides for estimated. Every check in the code fails in that direction, never the other: a number presented as measured has to have been measured. Otherwise the label is worthless — and with it the whole method.
Where the method runs out
Measuring beats guessing. It is not the same as weighing. The limits:
- It needs a LiDAR sensor. Fitted to the Pro and Pro Max models from the iPhone 12 Pro onwards — so not to older Pro models such as the iPhone 11 Pro. Without it the app works fully but does not measure — and says so.
- Occlusion. No camera sees what lies under the sauce or under the cheese. With layered dishes the app measures the outer shell.
- Darkness, glass, reflections. Very dark rooms and highly reflective surfaces return too little usable depth.
- Density is an average. The app uses the same density value for a fatty and a lean cut of the same meat. In reality fat tissue is lighter than fat-free mass — roughly 0.9 against roughly 1.1 grams per millilitre. The same volume therefore weighs differently depending on the cut, and the calories differ by more still. This is a source of error that measuring does not remove.
- Misidentified stays wrong. If the model takes couscous for cauliflower rice, the portion is measured exactly and the calorie figure is still off. Measuring fixes the scale, not the recognition. That is why you can correct every ingredient before saving.
Drinks, soups and anything in a container are special cases: the app measures the visible content. With opaque cups, entering the amount by hand is more accurate.
What we do not know yet
We have no accuracy figure of our own — not yet
The 30 % above describes the photo approach, that is, the problem. How well Cal 3D performs across many real plates has not yet been established in a large measurement series. Inventing that number would be the quickest way to lose your trust.
So there is none here. As soon as we have it, it appears on this page — with the date, the number of plates and a description of the setup, so that it can be checked.
To be unmistakable about it: Cal 3D makes no accuracy claim of its own. We describe what is measured (volume, in millilitres, on the device) and what is computed from it (grams via density values, calories via nutrition data) — and we label every entry accordingly. Nowhere do we state a percentage for Cal 3D, and nowhere do we claim an accuracy advantage over anyone else.
What we plan, so you can hold us to it:
- Volume error against objects of known capacity, so the measuring step is verified on its own before the chain hangs off it.
- Calorie error against weighed plates on a kitchen scale — the same setup that produced the 30 % above, so the numbers are comparable.
- Reported separately for measured and estimated, because a blended figure would smear the two cases together.
If you want to take part in a series like that — weigh a plate, scan it, send the data — write to noah.sioly.ug@gmail.com. It genuinely helps.