Which way does a column go

It is vertical. A way to tell the difference is this:
Member that horizontal comes from the word "horizon" , so that means that vertical is straight up, as in like the wall. 
Hope I helped you understand this concept. :)