Why is that less of a concern than the environment as a whole?
It's no more right to justify that than it would be to justify putting all of the country's debt on a single person or taking all of the pollution in the atmosphere but smothering individual children with it so that everyone else can breathe freely.
So we bring in huge ships with oil from the middle east that often run into trouble and spill into the ocean. The money we send allows dictators to force millions into slavery and prevents women from having basic human rights.
There are many reasons why a pipeline was preferable to the status quo
It would also create millions of American jobs and pour trillions into the economy with the increase in speed and quantity of shipping that can arrive for distribution, and curtail piracy around the Horn of Africa.