Math Forum / Mathematics / Numerical Analysis / January 2010

Hi,

A simple version of my fit problem is as follows:

I have a vector y of 100 measurement values, which I try to model with a set
of template vectors f_i (produced from a model via some complicated
function).
So you would typically calculate:

mu_i = (1/N) sum_j (y_j - f_ij)
sigma^2_i = (1/(N-1)) sum_j (y_j - f_ij - mu_i)^2

and the sigma^2 that results from template f_i is a measure of the fit
quality, right?
It's now easy to find the minimum sigma^2 over all templates and take that
as a best fit. However this procedure always gives a minimum even for pure
noise data which has no relation to the model I'm using.

Question: what should one use to tell, from the fit results sigma^2_i, what
the quality of the fit is? The absolute value of the minimum sigma^2 will
tell something, but that strongly depends on the additive noise on the
measurement (and not specifically on the model used).
The sigma_i could have a nearly flat (or just noisy) distribution, in which
case you wouldn't trust the model. It can also have a deep 'valley', in
which case the fit would be trusted some more. But at the moment I don't
know how to quantify this (or if this approach is even useful).

Any thoughts?

Thanks in advance,
Arthur