Are you going to hang out with your pot smoking friends all the time? That's when it's going to be the hardest, imo. Good luck.
I actually separated myself from them when I started college. Whew, if that wasn't the case, it would be damn near impossible. But since 80% of the people I currently hang out with, including my girlfriend, don't smoke pot, I think this should be easy.