You generally don't want to let a soft compound tire hard freeze repeatedly. It's not the temperature itself that does the damage, it's repeated temperature cycling with the weather going up and down between freezing and warm temps. And any sport tire (on or off the car) should be returned slowly to appropriate temperatures (delay of 24h+) before driving on them.
It may be worth investing in an efficient, low output space heater to keep the garage above ~40deg or ~45deg Fahrenheit year round. As long as the garage temps don't drop below that frequently, you'll be in much better shape with tire storage than in a totally uncontrolled space. The car itself will thank you too, the same sort of rubber aging from freeze cycling happens (albeit more mildly) to seals of various kinds throughout the car.