The last 600 years is a rounding error in the history of humans. Imagine if the different methods of lighting a fire were patented. Those humans also burned witches and imprisoned people for criticizing the church.
We would have a better world if our goal was progress for all over profit for a few.
It is not easy to divide human progress into before and after. Technological progress very much depends on what has gone before. It accelerated when it reached a certain point of development.
I agree with you that patents are not a key cause of progress, and they often impede it by preventing people other than the patent holder further improving anything patented.
I also think they probably work pretty well for mechanical inventions, but they have been applied to everything - drugs, software, electronic devices, even business methods in some places.
It would have implied a better, more developed society.
It would implied that there is a writing system.
It would have implied that there was a way to store records long term.
It would have implied that there was a long distance communication system.
It would have implied the existence of some state and rule of law.
It would have implied some type of justice system where you could get redress.
It would have implied a society with excess production of essential materials that it can support the division of labor involved in supporting this system.
None of that stuff is strictly dependent on IP laws, they just happened to be the obvious solution to the problem “how do researchers and artists protect their investments from folks that steal/copy their ideas and beat them to market?”
Common knowledge of physics for a thousand years was that heavier things fall faster, so spoke Aristotle. Nobody actually "fact checked" that because "duh, of course heavier things fall faster, this feather falls so slowly, everyone knows that" and so knowledge could not advance.