MultiDimFit reports “No valid low-error” despite Minos running succesfully

[desycms107]

Dear Combine experts,

I found that the codes of Combine in CascadeMinimizer and FitterAlgoBase sometimes prevented users from obtaining errors of POI returned from MINOS.

When one executes MultiDimFit, "FitterAlgoBase" used MINOS to calculate the error of the POI.

HiggsAnalysis-CombinedLimit/src/FitterAlgoBase.cc

Line 348 in 62005b0

if (minim.minos(RooArgSet(r))) {

And it only sets the errors of the POI properly in the following lines if the status of minos() is not equal to 1:

HiggsAnalysis-CombinedLimit/src/CascadeMinimizer.cc

Line 273 in 62005b0

return (iret != 1) ? true : false;

Unfortunately, the status of MINOS also depends on HESSE evaluated here:

HiggsAnalysis-CombinedLimit/src/CascadeMinimizer.cc

Line 191 in 62005b0

minimizer_->hesse();

My reasoning is based on what I found in MINUIT2 codes.

I observed many toy data where HESSE has status code = 1 because HESSE found negative eigenvalues of the error matrix, but MINOS had status code = 0, meaning it evaluate the POI uncertainty without errors. Thus, the final status code was 0*10+1 = 1. Following the quoted codes, this lead to the error messages from "MultiDimFit": Warning - No valid low-error found, will report difference to minimum of range for : r. On the other hand, I had a toy data giving MINOS status code = 8, which implied problems in the low error. But MultiDimFit still returned the wrong MINOS error-band to users because the final status code is $8\times10+1 = 81 \neq 1$.

The temporary solution is to do --cminDefaultMinimizerStrategy=0 to skip HESSE or use brute-force likelihood scan via --robustFit=1. The previous one is not ideal because it skipped the HESSE that helped the convergence of the minimiser and hid the warning from it. The latter option is also not optimal because it is much slower than MINOS and may be unnecessary when MINOS is working fine.

I have checked my toy data and workspace using --robustHesse to make sure that there is no negative eigenvalues in the error matrix after the rescaling of the likelihood function is performed. Unfortunately this rescaled workspace is not used by FitterAlgBase when it runs HESSE and MINOS. So it will be ideal if we can make this possible to solve the fundamental issue.

According to a ROOT documentation, the estimated error matrix could become non positive-definite due to large non-linearity of the likelihood function. So I can see why MINOS errors might not be quite right when HESSE finds negative eigenvalues as it is unclear if the minimum found by MIGRAD is a real minimum. But the Combine codes gives little clue to non-expert about why the error-bar of the POI is not reported and about how to fix it. The hard-coded status!=1 can even let wrong errors to be used by users unless they used higher verbose level and comb through the logs. Maybe we can start with only returning MINOS uncertainty of the POI when status==0 and give more informative warning in lower verbose level about what could be wrong in the workspace.

Best,

Kuan, B2G Combine contact