While it's still going to be a very real number, I think you are overestimating the number of deaths considerably. I appreciate that you provided a link, but I don't think that source includes the information you would need to make the estimate you are looking for.
First, I think that you are implicitly assuming that there are an equal number of people in each age category. In addition to the percentages you quote, you also need to know the percentage of the population per age range. Second, the "case" rate is the not the same as the "infection" rate, as it only includes confirmed cases. It's usually assumed that for each confirmed case, there is some unknown multiple of this number of infected individuals.
Eyeballing the chart here (and realizing that Santa Clara might not match these numbers, and guessing equal number in the relevant 5 year age ranges) a better estimate of the death rate for the 40-60 year old group is probably something around .15%: https://www.acsh.org/news/2020/11/18/covid-infection-fatalit...
