I'd be more worried about the wrinkles caused by squinting all the time! Keep your glasses handy in case you need to look at something in detail, but I don't think it will hurt not to wear them constantly. If you need them for driving, keep a pair visible in the car to remind you.

I have a glasses prescription, but the difference it makes to my general eyesight is so insignificant that I hardly ever wear them. They are useful for reading in poor light, but that's about it. This has been the case for about 30 years, and my eyesight has not suffered because I almost always forget to wear them.