undefined | Better HN

0 pointsmatthewdgreen3y ago0 comments

"This issue creates an enormous risk for all model deployments in medical imaging: if an AI model relies on its ability to detect racial identity to make medical decisions, but in doing so produced race-specific errors, clinical radiologists (who do not typically have access to racial demographic information) would not be able to tell, thereby possibly leading to errors in health-care decision processes."

0 comments

towaway154633y ago

Without knowing the actual outcome, isn’t there also a possibility of error due to not knowing the race of the individual? They used mammogram images in the study and it is well known that incidence of breast cancer varies by race. Removing that information from the model could result in worse performance.

cameldrv3y ago

Well one thing you wouldn’t want to do is take the output of this model and then apply a correction factor for race on top of it, because the model is already taking that into account.

towaway154633y ago

Is that true or would it help as a tie breaker in cases where the confidence was just at or below the threshold?

1 more reply

ibejoeb3y ago

Typically? It's coded in the standard. There's a DICOM tag for it.

https://dicom.innolitics.com/ciods/procedure-log/patient/001...

matthewdgreenOP3y ago

Unlike the authors of this research paper I am not a trained clinician, so I can't tell you. However I would note that the first exemplary value in the link you gave me is "REMOVED".

ibejoeb3y ago

It doesn't provide example data, but there's still a spot in the standard for it. The values can differ by modality or manufacturer. Sure, it's not required, but certainly it's very important in some situations. Consider dermoscopy.

If interested, searching for "dicom conformance" should yield lots of docs that probably contain specific values for those things.

1 more reply

nradov3y ago

I don't understand that part. All modern EHRs have a field for self-reported race, and clinical radiologists do typically have access to that information. (Whether they actually look at it, or whether it's useful when reading images, are separate issues.)

aulin3y ago

ok, maybe it's an US specific thing, why wouldn't a clinical radiologist have all the information he can gather about his patient including race to help the diagnosis?

codefreeordie3y ago

Because in the US we are required to pretend that there is no such thing as race and no such thing as gender, and all people are exactly and precisely the same and there can be no differences.

Loughla3y ago

Not to get into a flame war, but I want to present an alternate option to yours.

Because in the US some people have a hard time understanding that all races and genders deserve to be treated equally as humans with the same access to goods and services. Further, that there are disparities in care based on race/ethnicity[1][2] and gender[3][4] because of that racism/sexism present in the systems. This then leads to requiring that race/ethnicity and gender data be scrubbed sometimes to keep people from impacting outcomes based on their own biases.

[1] https://www.americanbar.org/groups/crsj/publications/human_r...

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1924616/

[3] https://www.americashealthrankings.org/learn/reports/2019-se...

[4] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2965695/

1 more reply

sandworm1013y ago

>> Because in the US we are required to pretend that there is no such thing as race

Then you are not pretending very well. When I lived in the US I was shocked at how often it was an issue. It permeates nearly every aspect of US culture.

The icing on that cake: A government-run interactive map so you can lookup which races live in which neighborhoods. Some versions allow you to zoom in to see little dots representing clusters of black or white residents. https://www.census.gov/library/visualizations/2021/geo/demog...

nradov3y ago

Actually, the US federal government specifically recommends that healthcare providers record patients' race, ethnicity, assigned sex, and gender identity. Most of those elements are self identified.

https://www.healthit.gov/isa/uscdi-data-class/patient-demogr...

iancmceachern3y ago

Interesting, this is like the dog learning calculus thing. We may create an AI that could perceive things that we aren't able to, or perceive things differently, because we're "limited" in a way that the AI isn't. We wouldn't be able to even tell this is going on, because we don't have the mental model in place to account for it to understand it. We'd be the dog.

j / k navigate · click thread line to collapse

0 comments

towaway154633y ago

cameldrv3y ago

Well one thing you wouldn’t want to do is take the output of this model and then apply a correction factor for race on top of it, because the model is already taking that into account.

towaway154633y ago

Is that true or would it help as a tie breaker in cases where the confidence was just at or below the threshold?

1 more reply

ibejoeb3y ago

Typically? It's coded in the standard. There's a DICOM tag for it.

https://dicom.innolitics.com/ciods/procedure-log/patient/001...

matthewdgreenOP3y ago

Unlike the authors of this research paper I am not a trained clinician, so I can't tell you. However I would note that the first exemplary value in the link you gave me is "REMOVED".

ibejoeb3y ago

If interested, searching for "dicom conformance" should yield lots of docs that probably contain specific values for those things.

1 more reply

nradov3y ago

aulin3y ago

ok, maybe it's an US specific thing, why wouldn't a clinical radiologist have all the information he can gather about his patient including race to help the diagnosis?

codefreeordie3y ago

Because in the US we are required to pretend that there is no such thing as race and no such thing as gender, and all people are exactly and precisely the same and there can be no differences.

Loughla3y ago

Not to get into a flame war, but I want to present an alternate option to yours.

[1] https://www.americanbar.org/groups/crsj/publications/human_r...

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1924616/

[3] https://www.americashealthrankings.org/learn/reports/2019-se...

[4] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2965695/

1 more reply

sandworm1013y ago

>> Because in the US we are required to pretend that there is no such thing as race

Then you are not pretending very well. When I lived in the US I was shocked at how often it was an issue. It permeates nearly every aspect of US culture.

nradov3y ago

Actually, the US federal government specifically recommends that healthcare providers record patients' race, ethnicity, assigned sex, and gender identity. Most of those elements are self identified.

https://www.healthit.gov/isa/uscdi-data-class/patient-demogr...

iancmceachern3y ago

j / k navigate · click thread line to collapse