A lot of our response to weather (and everything else) depends on our attitude.
My attitude is, It's not Cold, It's Bracing!
Then again, I do dress for the weather. I cover my legs (still wearing bike shorts) whenever it's below 60 degrees, and wear a windbreaker (bright yellow) at that temperature.
I add full gloves and ear coverings if it's below 55 degrees, and add a second layer if it's below 40 degrees, or if I'll be riding into a wind.