undefined | Better HN

0 points_0w8t10y ago0 comments

The article calls for an explanation not why some ML method works, but for an explanation of a particular ML result, like why a car drives this way or why a patient got a cancer. While I hopeful for the former, I just do not see the basis for the latter.

If some regulations shall require such explanation, the end result will be fake stories like parents tell to the children that Moon do not fall because it is nailed to the sky.

0 comments

5110910y ago

I don't think the paper asked for that. Relevant quote:

> machine learning is more concerned with making predictions, even if the prediction can not be explained very well (a.k.a. “a blackbox prediction”)

So in your example: an algo may explain that a car slows down, before taking a turn, because else it would likely crash. It may even get to a threshold ("under these weather conditions, anything over 55Mph is unsafe when taking a turn of such and such degree"). Statistics can help with that.

Welling is not asking for deep learning models to explain why a person got a cancer, but to explain its reasoning when it diagnoses a person with cancer ("I am confident, because in a random population of a 1000 other patients, these variables are within ..."). Statistics can help with that. It aligns with their mind set and tool set.

Regulations are cheated even with these kind of explanations, but that is for another story (black box models may provide some plausible deniability).

_0w8tOP10y ago

I am referring to this fragment:

> Thus, for many applications, in order to successfully interact with humans, machines will need to explain their reasoning, including some quantification of confidence, to humans.

No doubtful there are cases when an explanation is easy. Often this is because we have a very solid model like physics of a car. In fact since we know the model, we do not need an explanation, we must demand that the algorithm follows the model or declare it unfit.

But how can we expect an explanation for a behavior in a critical situation on a road that was not explicitly programmed and when the algorithm decided to turn to a particular degree bases on a non-trivial inference? Similarly, when an algorithm decides if a patient needs an emergency operation or if they can wait, why can we expect an simple explanation especially for the patient with rare conditions when algorithm again must perform an inference, not a deduction from 1000 very similar cases?

jupiter9000010y ago

Maybe you couldn't always expect an explanation, but having one would certainly be useful. Methods that could get the job done while also providing enough information to explain or at least shed some kind of light on why the solution was chosen would likely be preferable.

j / k navigate · click thread line to collapse