Yes I agree, depending on the project you might need to dip into some Erlang libraries. But that's a far cry from needing to learn Erlang first.
I remember reading things like "you should learn Erlang before learning Elixir to be really effective" scared me off learning Elixir for a little bit as well.
Also, I find the more familiar you get with Elixir, the easier it becomes to read Erlang code.