Yup. What frequently happens is that companies in an Oligopoly are able to collude to raise the barriers of entry, keeping out new competitors: sometimes with "help" of government regulations, sometimes because of lack of government regulation. As long as there are only a couple of them (< 4 controlling > 80 percent of the industry), they can keep prices artificially higher than they need to be.
When there's no actual competition and little threat of new competitors entering the market, their incentive to innovate or become more efficient is diminished. Consumers end up paying too much for mediocre goods/services.