Air borne allergens will effect you if you breathe them in. While you are outside you will breathe in some of the mold. You will also breathe in the mold while inside your house, but in much lower amounts. If you have a good filtration system the mold will be minimal. Allergies are basically an over-reaction of your immune response. As long as you have experienced breathing in the mold before, your body will have antibodies which tag the foreign stuff. Your immune system is suppose to fight the mold without causing you harm. I used to have 70 different allergies, and now I'm down to a couple of environmental allergies that bother me only when I'm stressed. There are many nutritional ways to teach your immune system to calm down and work correctly instead of over-reacting and causing you harm in the process.