Small data, narrow data, and what the data cannot see
Biological projects rarely arrive with clean data. What actually shows up is some combination of three things.
Small data. Hand measurements and observations. Stem diameter and truss number once a week. A fish sampled and weighed at stocking, once mid-cycle, and at harvest. Forty rows for a season, taken by someone with a notebook and other work to do. Precious, because a human looked at the plant, and thin, because there are forty of them.
Narrow data. A few sensors at high frequency. Greenhouse temperature and humidity every minute for a year. Oxygen in the tank every thirty seconds. Millions of rows measuring three of the thirty quantities that drive the system.
Missing data. Everything else. Root zone conditions, actual light on the crop after screens and dirt, real feed intake as opposed to feed delivered, what happened during the two weeks the logger was down.
The reflex is to treat this as a data problem and go collect more. Sometimes that is right, and part of our work is saying exactly which extra measurement would pay for itself. More often the sensor you want does not exist, cannot be afforded at scale, or would answer a question you could have derived from what is already known about the biology.
The identifiability problem, stated plainly
Take forty weekly observations and a model with fifteen parameters. Fit it and you will get an excellent fit. You will also get a second parameter set, very different in the numbers a biologist would care about, fitting equally well. A third one too. The optimiser is happy. The model is useless for steering, because the parameters you would use to predict the effect of a change are not determined by the data.
This is not a bug in the fitting method. The information simply is not in those forty numbers. Ignoring that gives confident nonsense, and confident nonsense in an advisory setting is worse than no model at all.
We attack it from two sides.
a) More mechanism, not less
The intuitive move is to simplify: fewer parameters, less to identify. It usually makes things worse, because the simplified parameters become lumped quantities that mean nothing biologically and drift with every dataset.
The move that works is often the opposite. Add mechanism.
It sounds wrong, because more equations means more parameters. The reason it works is that mechanism adds constraints, and constraints are free information:
- A parameter that appears in three places cannot move freely. A maintenance rate that has to explain night-time respiration, the response to a warm week, and the size dependence of growth has far less room than a maintenance rate that only has to fit a growth curve.
- Mass and energy have to balance. Carbon in equals carbon fixed, respired, and stored. Salt going in has to end up in the drain or the root zone. That pins scale without a single extra measurement.
- Units and physical limits bound things before the data gets a vote. Efficiencies live in [0, 1]. Fluxes have signs. Temperature responses have a shape.
- The correlation structure changes. Two parameters that were hopelessly entangled in a black box model are often separated once the process connecting them is written out, because they now leave different fingerprints on different parts of the trajectory.
Concretely, the covariance matrix of a mechanistic fit does not just get smaller. It changes shape. The ridges that made the fit unidentifiable get cut by the constraints, and what remains is often a well-determined subspace plus a few directions we can name and admit we cannot see.
b) Automated sensitivity analysis: autoFocus
The second side is finding out, before fitting anything, what the outputs are actually sensitive to. Not one parameter at a time by hand. Every parameter, across the ranges that are biologically plausible, across the scenarios that matter, thousands of runs, automated.
What comes out is more useful than a ranked list:
- Clusters. Groups of parameters that always move together. The data will never separate them, so they get treated as one thing, and we say so.
- Axes. Directions in parameter space, combinations rather than single parameters, along which the outputs actually move. Fitting happens along these. It is stable, because they are the directions the data can see.
- Blind spots. Directions that change the parameters a lot and the outputs not at all. These get fixed from literature or from mechanism, with the choice written down rather than hidden inside an optimiser.
- Where it matters. Sensitivity is not constant. A parameter can be irrelevant at normal light and dominant in a heat week. That tells you which weeks of the season are worth measuring, and which sensor to buy.
This is the part we have automated hardest, because it is the part that is skipped when time is short, and skipping it is how models end up overconfident. It runs on every model as a matter of course, not as a special study. In our own pipeline it is called autoFocus, and it is the step that turns a big model into a small number of things worth arguing about.
What you get out of it
An honest statement of what the model can and cannot tell you, in the language of the decision you were trying to make. Something like: the effect of this setpoint change on yield is well determined and worth acting on; the split between two loss pathways is not, and here is the one measurement that would settle it.
That statement is the deliverable that survives contact with reality. The pretty curve is not.