Increase your miles. Ignore the weather - the wind was a factor of mine years ago when I was first starting out - but it's the "what hurts you, makes you tougher" syndrome. Just keep pushing it. And as you increase your miles and look at that computer, you will be more stoked.
I've been car-free for probably 18 years - and I commute year round (yep - in the winter in upstate NY I run studded tires - and yes on the really bad days I take a bus to work - why be more foolish than I already am, hehe) and I do off road and I do long distance on my roadie (just have to ride out about 20 miles to get out of this darn city to get to the lovely rolling roads in the sticks).
So yes, keep pushing the miles - it will feel good and you will enjoy them. Have fun.