Introduction

(1)

The value of c² is minimized with respect to c.  This object is not to find the minimum value of c².  The object is to find the f_A that most closely approximates the function underlying f_i.  The data generated in ArtDat\Welcome.htm has a Gaussian distribution about the underlying value of f_A with a standard deviation given by

  (2)

The data is random, but the error reported is exact, that is it uses f_A as in (2).  There are derivations that indicate that almost all small signals have a Poisson distribution with errors proportional to the size of f_A.  Poisson/Welcome.htm so the value of b_err is most naturally set to be such that b_errf_A is the number of statistically independent counts in each channel. 

This leads to four problems.

1. If equation (2) is inserted into equation (1) which is then minimied with respect to c, there is a term from the denominator that in rare cases can dominate the minimization Loophole.doc.  When the object is to find the best estimator in a Poisson distribution, equation (1) is not the correct expression to minimize [Poisson\Fitting with Poisson random variables.doc], but repeatedly minimizing (1) without including the variation in (2) usually does lead to the correct minimum. – For an example that does not quite find the exact minimum see FermiFit\Welcome.htm.  The best procedure is

a)       Start with a value of f_A in (2) and find all e(x_i) values.

b)       Minimize (1) holding {e} fixed to find a better estimate of f_A.

c)      Use the better f_A in step a)

This will not completely minimize c², but it will find the f_A desired.

2. Zero counts are poorly represented by this distribution and even 1 count does not have a Gaussian distribution. A “solution” to this is to make a_err equal to 1 – This is the strategy used in ..\robfit\Welcome.htm.  This makes the terms in c² à f_A² in the region where there are zero counts leading to a total c² < N by approximately the number of channels with zero counts. 
3. A peak with a few million counts so dominates the value of c² that changes in this peak that may not even be visible when plotted far outweigh the changes of interest at very low count rates stanfit3.doc#Practical_Curve_Fitting.  The solution to this is to make c_err ~ 0.01 for “graphical” accuracy even though the underlying statistical errors are actually Poisson.  This has the same effect as the “solution” above in that counts near large peaks will have values of (f_i-f_A(x_i))/(Ö(b_errf_A(x_i))+c_errf_A(x_i)) smaller than 1 due to the second term in the denominator.
4. Small values e_i in (1) produce local minima for which a single feature has to be preserved while others are optimized.  It is much easier numerically to minimize c² with large values of.  A series of runs with successively smaller values of a_err, b_err, and c_err is much less likely to be stalled by local minima.

 

Folders

01/31/2008  07:03a      <DIR>         ArtDat\Welcome.htm – Considers the generation of artificial data based on a standard direction file.

02/25/2008  07:45a      <DIR>          Poisson/Welcome.htm Poisson – Details about the Poisson distribution

Documents

 

02/27/2008  10:35 AM            35,840 FindingTheErrorConstants.doc

02/04/2008  07:25 AM            79,872 Expectation values.doc

02/04/2008  04:32 PM           199,168 SimErr.doc

02/04/2008  04:39 PM           132,608 Copy of Estimating the error per point.doc

02/08/2008  08:09 AM           224,256 Estimating the error per point.doc

02/25/2008  05:12 PM            30,208 Loophole.doc