Many factors are involved here. On one hand, you're right about Dijkstra (Is $1000 an option? I never actually got my copy signed (sigh), but....) and Nancy Drew, but on the other I think it's likely that most of the money involved involves "mass market" books: thrillers, Oprah's picks, the stuff on the New York Times list. At some point it does boil down to the marginal situation of "two unknown authors" (or two approximately equivalent ones) that really are semi-commodities. Think of a reader who has read everything that has been suggested or is from their favorite authors, and is looking for something new.

Also keep in mind that the "mass" entertainment market is not competing with speciality books, but is competing with other entertainment options: Netflix, movies, drunken debauchery, etc.

Then there is the price levels we're talking about. Everyone (probably) has some price X below which they will have no problem impulse-buying. Then there's a price Y below which they're willing to take chances, with information, and then there are prices above that, which require actual consideration. And for most people, at least those buying ebooks, we are definitely arguing in this range. For my inner economist, publishers want ebook prices up in the high segment, above Y, while Amazon wants them roughly between X and Y.

"If Amazon has helpful data to add to this, why don't they open it up and let everyone, publishers included, see it to help price the books better?"

The specifics are very likely competitive trade secrets. Don't forget Amazon gets to deal with Apple, B&N, and a host of other retailers. I'm surprised they released the one number they did. On the other hand, judging by negotiations I've been in, I would not find it unplausible that Hachette is aware of those numbers.

"They are trying to squash publishers, gain control of the market and dictate prices, which they cannot currently do."

Yes, just like the publishers are trying to squash Amazon, gain control of the market, and dictate prices. Heck, this isn't even a particularly new situation: I'm sure the negotiations between publishers and distributors that led to the lovely "stripping" practice were just as entertaining.