I think most of the bad rap comes from the fact that a lot of drug deals go down in buildings with squatters. Granted there are a lot of good people who are homeless, but a few rouge criminals have to ruin it for everyone else.

Unfortunately squatters are stereotyped; they probably get a bad rap from those people who squat in abandoned buildings as a safe haven to use drugs. This is a generalization of course and there are people who "squat" in empty lots by creating gardens for the community. The main reason that people might get in trouble for squatting is probably due to the fact that just because buildings are abandoned does not mean that they are public property; it is still considered trespassing if someone owns the property, even though nobody lives or works there. 

Courts usually pop a squat on squatters simply because they are attempting to get something for nothing.  This is the crux of the decision against many claims by squatters' rights groups for adverse possession of property.  (Adverse Possession is acquiring possession of property over time without compensating and in a way that is "adverse" or "hostile" to the owner.)  The squatters' argument: these buildings were a haven for bad things (drugs, disease, rodents, etc.), and we made it better by living here and improving it.  The courts' argument: regardless of how you improved the property, you had no right to be there in the first place.