If your pointers have to have a bit offset, then now a pointer isn't just 1 hardware word, which would be odd to say the least for a systems language. (It would also be slow, because making a load anything other than a load is something you really don't want to do; you would lose the ability to fold into addressing modes on x86 in many cases for example.)
If some pointers have a bit offset but others don't, OK, I guess, but I'd argue that at that point it'd be a cleaner design to just have a "pointer plus bit offset" be a separate type from "regular pointer". And that would get back to the problem that you would have a type that you couldn't take a "regular" pointer to.
