When is the best time to prune tree branches?

Need to prune some tree branches back from the house. They are starting to rub against the siding and brush against the shingles. I'd like to prune them ASAP, but is it okay for the tree if I prune them now? The tree is perfectly healthy, just growing too close to the house (and I didn't plant it -- it was growing there when we bought the house).