S. Teige et al.
Intended for submission to Phys. Lett. B.
postscript version 18 Feb 2000
pdf version, the lousy quality is due to my conversion of their postscript file to pdf. Their original (above) looks better.
postscript version 15 Apr 1999
postscript version 15 Apr 1999
postscript version 5 Jan 2000
| Question | Answer |
| From Jim Napolitano | Figure 3 has been modified to includes an estimate of the acceptance as a function of z. |
| From Hans Willutzki Looking at figure 3 I count 26 data points with the horizontal errorbars of the first and last data point going into unphysical territory. By comparison the Crystal Barrel paper has 25 data points and the errorbars do not go below 0.0 or above 1.0 |
I used different binning than CB and the picture has been remade. |
| From Hans Willutzki Our sample has 87500 events, the Crystal Barrel paper quotes 98000 events. Looking again at figure 3 Scotts statistical errors appear to be somewhat smaller than theirs. |
CB did not do sufficient Monte-Carlo, their MC statistics were less than their data statistics. In our case, the contribution to the error from the MC is negligible as the number of MC events exceeds one million. |
| From Hans Willutzki Our quoted sytematic error is much smaller than theirs. Is this because our acceptance and/or resolution is superior to theirs? |
The sources of systematic errors are discussed in the text of the paper and supporting documentation is given here. I don't know why our systemetic error is smaller but I am confident it is estimated correctly. |
| From Hans Willutzki Looking at figure 1 we disagree with the Crystal Barrel by almost 4 sigma (ours). |
The relevant comparison is to the larger of the error bars (ours vs theirs), hence the statement "probable disagreement (2 sigma)" in the text. |
| From Hans Willutzki How does the acceptance look as a function of z? |
Figure 3 now contains an estimate of the acceptance as a function of z |
| From Hans Willutzki One would like to see a comparison of kinematical variables with Monte Carlo events. This should be shown in additional figures. |
Clearly, the most relevant kinematical quantity is alpha, the quantity being measured. Comparison of input to output is not possible because if an input of x is used in the Monte-Carlo, an output of exactly x is obtained, by construction. Next important in relevance are the confidence level distribution and the photon separation distribution. Please see my postscript or pdf note for comparison of MC and data. Things like beam momentum, apertures etc. are E852 standard. |
|
From Bob Hackenburg The issue of combinatorics is not adequately handled in the third paragraph of page 3 (it starts "eta -> 3 pi0 decays were selected..."). What happens to those events which satisfy two possible combinations, both with confidence greater than 1%? Statistically speaking, it is certainly not correct to simply take the combination with higher confidence and ignore the other. That would introduce a bias; a particular combination is selected to do a particular constrained fit, and the "wrong" combination will occasionally have a better confidence than the "right" combination (which of course would still pass the >1% criterion 99% of the time). The correct thing is to count them both (or all, if more than two combinations work), which of course causes a counting problem. One might decide that the bias is easier to live with than the counting problem, but my point is that the article doesn't state what is done; it only says that the best combination is used to decide whether to keep the event or not. It talks of "combinations" (one or more per event), and then switches the noun to "events" later in the paragraph. I assume that you took the best combination, which introduces the bias mentioned above (ie, some real 3pi0 combinations are discarded in favor of fake combinations which happen to have better confidences). I would prefer to deal with the counting problem than this bias. For example, you could state how many events had this combinatoric ambiguity, and what fraction of the counts in the effective mass plot came from multiple-counted events (I don't think you multiple- count at present, but I think you should). Certainly, if that's small, it's not worth further concern. Multiple-counting of combinatorically ambiguous events is standard, and it introduces the famous combinatoric (and unavoidable) background. This issue is addressed in the Crystal Barrel paper. |
A complicated question, check out my postscript or pdf note for an answer. Executive summary: it does not matter how the choice is made, bottom line is unchanged. A paragraph summarizing this study has been added to the portion of the paper discussing systematic errors. |
|
From Bob Hackenburg Fifth paragraph, page 3, which starts "To determine alpha eqn. 1...", is missing the word "on" between "based" and "a monte-carlo" in the second |
Text has been modified. |
|
From Bob Hackenburg Same paragraph, possibly the word "with" is missing in the sentence: "...any uncertainties in the monte-carlo associated the transverse developement..." |
Text has been modified. |
| From Neal Cason |
I hope the answers to the comments made by others addresses some of your concerns. In addition, Paul Eugenio and Bob Hackenburg will be reviewing my responses to the issues raised by others and will report back to the collaboration. |
| From Kam Seth 1) In the acceptance plot, which you have not included in the draft, but which you have on the web, it is clear that the last point at Z=0.95 is to be rejected as being clearly aberrant. This being so, the points at Z=0.95 and Z=1.0 in Fig. 3 should be removed. If they are removed from the plot and the fit, probably the value of alpha will change a little bit. My eyeball guess is that the slope of |M|**2 versus Z , i.e., alpha, will also be be reduced. It is this result which should be quoted |
The acceptance in the last bin of the old plot is in fact off the scale (vertically) due to an error I made when I plotted the acceptance. This error does not exist when the acceptance is used to calculate alpha. While the error on the acceptance in this bin is larger than typical, I cannot justify eliminating this bin in the fit, hence I have continued to include it. FYI and FWIT, if this bin is neglected, the change in alpha is 0.0005, or about one tenth of alpha. |
| From Kam Seth 2) Because of the direct relationship, I, for one, would like to see the acceptance plot in the paper. |
Figure 3 now includes an estimate of the acceptance. |
| From John Dowd 1. The fit to the eta (solid line) in the insert of fig 2 is never mentioned. Is this an oversight? |
The fit to the bump has beem removed. The dashed line, indicating the background estimate is now mentioned in the caption. |
| From John Dowd 2. In figure 1, the e852 results are shown as 5 different points corresponding to different assumptions in the analysis -- to show the effect on systematic error. However the "extra" data points are not explicitly mentioned in the caption. Maybe add " the four lower data points show the effect of varying the cuts in the data analysis" or some such statement to the caption. |
Text has been modified. |
| From John Dowd 3. The statement that the phase space dependence was removed by dividing by the distribution due to a uniformly populated dalitz plot isn't clear to me. It sounds like a division by one. |
This "correction" is to remove effects of the edges of the Dalitz plot. Examination of this little postscript or pdf figure should demonstrate exactly what this is about. |
| From John LoSecco My comments concern the estimates for systematic error. The number of events fit varies with the cuts used to study the systematic error it might be nice to include the event counts to give the reader an idea of how strongly correlated the different estimates are. For example how many events are lost in moving the photon energy cut from 0.25 to 0.5? |
The estimates are clearly correlated since one fit simply uses a subset of the data used in another fit. (With the exception of the momentum transfer study). I've added text to indicate how the data sample sizes changes with the Emin cut. To be specific, 15,230 events are lost when the Emin cut is boosted to 0.5 GeV. |
| From John LoSecco The CL gt 0.3 cut seems rather strange. It is certainly not an obvious place to make a confidence level cut. What does the confidence level distribution look like? Why is it odd below 0.3? Is this just some simple background, another final state, that got through the trigger. |
Neal Cason has also raised concern over the CL cut I used. Please see the response to his questions below. |
| From John LoSecco The text never mentions what CL cut was used to arrive at the 87500 event sample. |
The section on systematic errors contains the following text: "A similar analysis was performed to determine the value used for the confidence level cut. It was found that $ \alpha $ does not depend on the value used for the confidence level cut provided $ {\rm{CL}} \geq 0.3 $. A contribution to the systematic error of magnitude similar that of the $ \Delta {\rm{r}} $ study was also determined." I believe this indicates the value used. |
| From John LoSecco SInce posting my remarks a more significant source of error occured to me. The distribution is acceptance corrected. Have you looked at how strongly the acceptance correction influences the fit and how well we think we know the acceptance? |
Neal Cason has also raised concern over the acceptance. Please see the response to his questions below. |
| From Neal Cason What Monte Carlo was used? If neither of the standard E852 techniques (SAGE or GEANT), then we need to subject the analysis to some systematic verification. |
Sagen was used with a GEANT generated shower library to simulate the hits in the LGD |
| From Neal Cason How were the corrections applied? Were they done on an event- by-event weighting or was the Dalitz plot itself corrected? (Or was the z distribution corrected directly?) |
The Z-distribution was corrected directly. |
| From Neal Cason Is there any way to check that the corrections are right? |
My approach was to evaluate the effects causing our acceptance to differ from 1. This final state is simpler than many that have been considered to date: Apertures are not a problem and charged particle detection problems do not exist. The only effects that can modify the acceptance are "reconstructability" requirements on the photons. That is, they must be energetic enough to be detected and are sufficiently separated to be resolved. The minimum energy requirement was found not to be an issue. The separation cut is a bigger effect. Please see my postscript or pdf note on this effect and the effect of the CL cut. |
| From Neal Cason Full kinematic fit to n eta final state: |
Yes. |
| From Neal Cason What do the pull distributions look like? What does the confidence level distribution look like? |
Please see my postscript or pdf note on this effect and the effect of the CL cut. |
| From Neal Cason Why do you need such a high (30%) confidence level cut? |
Please see my postscript or pdf note on this effect and the effect of the CL cut. |
| From Neal Cason Were the Monte Carlo events put through the fitter - and do they have a similar confidence level distribution? |
Yes. The "raw MC" events were analyzed by exactly the same analysis chain as the data. In particular, a1_nohtr, sq97 and esrMaker were used. Please see my postscript or pdf note for a comparison. |
| From Neal Cason Basically, do we believe that the fitter has given us an unbiased sample of events? |
The operative assumption is that whatever biases are introduced in the data sample are also introduced in the MC sample and that the selection cuts are adequate to insure the samples behave identically. It is my assertion that I have identified selection criteria meet this requirement. |
| From Neal Cason Why does the value of alpha change when delta r is less than 9 cm? This makes me think that the Monte Carlo corrections don't work as the photons get closer together, and this is worrisome. |
I have long claimed that EM shower Monte-Carlos do not properly reproduce the transverse development of energetic photon-induced showers and find it difficult to believe it could be any other effect. This is not fatal, it is simply required that a cut be applied so that the showers are sufficiently well separated so that this effect does not matter. (After the manner of the "beam hole cut" used in charged particle analyses) The requirement here is that if two showers are separated in the data sample, they are also separated in the MC sample. Please see my postscript or pdf studies for a numerical value for the separation cut and my postscript or pdf note for a comparison of Monte-Carlo and data Delta-r distributions. |
| From Neal Cason Similarly, why does the value of alpha change when the confidence level cut is less than 30 %? This is particularly worrisome since I wouldn't think a cut at 30 % would be necessary. |
Proof is impossible but a plausible argument isn't. Suppose that for events with a "sufficently low" confidence level MC and data behave differently, that is, different things go wrong with the fits in each case. The fitter would place these events onto the Dalitz plot differently causing different biases for the two cases. If this is so, the bias does not cancel when the acceptance correction is applied. As more and more of these "sufficently low" confidence level events are included in the data sample, the uncorrected bias effects the measured value. If the events are excluded, the bias vanishes and the correct value of alpha emerges. The difficulty now is to determine what is meant by "sufficently low". Please see my postscript or pdf studies for numerical values. A comparison of Monte-Carlo and data CL distributions can be found in my postscript or pdf note about this effect. |
| From Gary Adams 1) p. 2 last para, the analysis steps are not clear, in particular, was the first selection a fit to pi0 masses? Also, does "probability greater than 1%" mean C.L. cut? If so it should be spelled out. |
Yes, the first step was to make a DST with all events with any chance at all of fitting the the 3pi0 hypothesis. This was done with a very loose cut on the best chi-squared for the 3pi0 hypothesis from all possible asignments of photons to pions. The text has been modified. |
| From Gary Adams 2)Fig. 1 will not show anything useful when it is shrunk to journal size. It should be replaced with a table. |
Figure 1 has been remade and the caption rewritten. |
| From Gary Adams 3)figs. 2 and 3 have lettering that is much too small for publication. Also, the inset in fig 2 does not show anything that isn't obvious from the larger fig. Perhaps you should change it to a semi-log inset, or make one semi-log plot? |
Figures 2 and 3 have been remade. I include the 3pi effective mass spectrum out to 2 GeV/c^2 because I always find it fishy when a truncated spectrum is shown. The larger range spectrum is now semi-log. |
| From Gary Adams 4)on p. 4 it says the C.L. cut at 0.3 was made because the answer changes if you put it lower. That isn't a very good reason (the correct answer could be the one with all events). I assume this was set based on its effect on the systematic error? This kind of data selection seems circular. Perhaps the CL distribution itself would make the case for you? As it stands it looks odd. |
The CL distributions also justify this cut, particularly when compared to Monte-Carlo. Please see my postscript or pdf note about this effect and the discussion above. |
| From Gary Adams 5)I think someone else already pointed out that the points on fig3 spill out past the limits. |
I used different binning than CB and the picture has been remade. |
The LaTeX files:
Main file
Author list
Previous results/systematic error figure
3 pi0 effective mass figure
dN/dZ with fit that determines alpha