If you are allocating memory in your renderloop, your likely doing it wrong. Allocation outside of this critical path, well, chose your poison.
GC will consume more memory, but likely to be faster when it comes to allocating and deallocating large amount of small blocks.
Reference counting, if you do it properly (to avoid thread starvation), is costly. Cheap atomic reference counting, involves a calculated risk that you may have threads hanging.
Manual memory management, well, we all know the cost of that :)
That being said, many many games are today developed with Unit which uses C# as its primary programming language.