When your baby is sick (and it's cold outside), do you limit their outdoor activities?

My son just got over the flu, and I limited his time outside the whole time he was sick. He didn't feel like going out anyway, and I didn't want him out in the cold and wind.