A follow-up to my comment on the Array API: I don't think we can support the Array API here, as scipy.optimize.minimize does not appear to support it.

If I missed anything, please let me know—I'd be happy to investigate further.

Thanks for the PR. Here is a first pass of feedback:

I'm still working on addressing the feedback, but I also wanted to share some findings related to it and provide an update.

I few computational things seem off.

There are some CI failures—I'll fix those shortly.

Also considering adding a verbose parameter to CalibratedClassifierCV to optionally display convergence info when optimising the inverse temperature beta.

At this point, CI is complaining about missing test coverage for tags and for the errors raised in _fit_calibrator and _CalibratedClassifier.

I'm not quite sure how to handle the tags part, but I think the other two cases are fine, since the tests should cover warnings raised by the user-facing API, rather than by private functions.

I'll work on the convergence warning for minimize_scalar later.

Updates on Raising Warnings When scipy-optimize.minimize_scalar Doesn't Converge.

minimize_scalar(method = "bounded") raises the following warnings when optimisation is unsuccessful:

1. "Maximum number of function calls reached". Controled by the "maxiter" in the options dictionary.
2. "Nan result encountered".

(Reference: here)

For (1), the default value is maxiter=500. In my testing (over about a week of trial and error), setting maxiter=40 or lower may trigger non-convergence. However, reducing the number of iterations negatively affects the performance of the calibrator in otherwise converging cases. So I don't think it's worth lowering maxiter just to surface a warning in rare failure cases.

For (2), the docstring of HalfMultinomialLoss says:

        loss_i = log(sum(exp(raw_pred_{i, k}), k=0..n_classes-1))
                - sum(y_true_{i, k} * raw_pred_{i, k}, k=0..n_classes-1)

As far as I can tell, the only two ways loss_i could become NaN are:

• if raw_pred contains np.inf, or is entirely -np.inf.
• if raw_pred contains NaN.

But in either case, scikit-learn's check_array would catch this and raise an error before minimize_scalar even runs.

There's also the edge case where the minimiser lands exactly on a boundary ($\pm 10$), but due to the xtol tolerance in SciPy's optimizer, it ends up returning something like $\pm 9.9999999…$ and still counts as a successful termination.

So overall, I'm leaning towards our current implementation being fine in terms of convergence handling. That said, I might be missing some edge cases or subtle points.

Just to be safe, I've updated the options dictionary in _temperature_scaling to show a warning when convergence fails, and I've also tightened the absolute tolerance xatol for convergence.

Let me know if I've overlooked anything — I'll keep working on it!

Thank you for the review, @Ball-Man!

I can confirm that scikit-learn's check_is_fitted function raises an error for temperature scaling, but not for sigmoid or isotonic calibration, after fitting on a calibration dataset.

I've updated the attribute as you suggested.

I realised that # pragma: no cover can be used to raise errors without affecting CI coverage. So I've added it to both _CalibratedClassifier.predict_proba() and _fit_calibrator.

Specifically, if the method parameter is anything other than "sigmoid", "isotonic", or "temperature", the input validation in CalibratedClassifierCV will raise an error before any of these private functions are even called.

Similarly, I was able to raise an error when minimize_scalar fails to converge, without upsetting CI coverage in the absence of a dedicated test case. So this fix felt reasonable.

Let me know your thoughts, I'm happy to keep working on this!

A few comments otherwise looks good. Thank you for the updates @virchan

CI passed!

LGTM. Thank you @virchan