But I fully agree with you. Likely what you need is a tiny subset of the capabilities wrapped up by the higher level abstraction, so implement them directly.
Over time you may find you need additional capabilities (although that's far from a given) and, if and when you do, you'll need to make decisions about whether to implement them directly, or wrap everything in a higher level abstraction (or use a third party abstraction).
The point is that if you ever do need these additional capabilities there's a good chance it's because you've been successful enough to enter "good problem to have" territory because you didn't waste time getting distracted by them earlier on and instead chose to focus on work that enabled that success.