Summer camp, school environments, etc., can be quite a tricky thing to understand sometimes. I don't think there's always a clear answer to your problem. ^^ From what I've gathered, I can tell you that it's not always that you've done something "wrong," but rather someone else's judgement of you does not align with his own perspectives, his own mindsets, etc. People like to fit in to be inclusive to a group or a way of behavior/thinking because it gives the appearance of unity and equality and because it masks fear that you will not be accepted or judged in a way that makes you feel uncomfortable. My advice would be to ask yourself why you want to fit in. The desire to be included in a group (through clothing style, physical mannerisms, hobbies, etc.) is not a harmful thing in itself, but I think if you look into the why you want to fit in, as opposed to why you don't fit in, you might find a bit of insight. If you don't fit in because you don't enjoy the things other people do, trying to force yourself to enjoy those things for the sake of gaining someone's respect is only changing yourself for something you think other people want (and it's so difficult to tell what people want, too), which can be destructive emotionally because you may allow someone else to dictate your life without truly wanting to live said lifestyle, dress said way, be like said people. The easy way out is to follow everyone else, but then I think it's difficult-- more so than standing alone-- to simply get out, to simply rekindle your originality and difference. The greatest thing you can walk away from your experiences of feeling isolated and an outcast is not the discovery of how to fit in, but discovering the importance in walking outside of the norm.
What you're going through is not something you've walked alone in-- many other people have felt excluded depreciated, and while your experience is unique, the only little comfort I can offer is the perspective that you are not alone in feeling hurt by those environments. My advice to you would be to not allow the world to make you bitter and resent the people who make you feel different (because the people who are making you hurt may have gone through a similar thing, conformed to a standard by putting forth an abrasive behavior, simply because they don't want to be seen as different). As difficult it is, stay true to what you know you care about, the things you enjoy. People will always find a way to judge you-- by trying to conform to someone else's style, his way of life, or anything about him, you are not evading judgement, rather passing judgement from one group of people to another. If you try to fit in, people will judge you for wanting to sway with the crowd, and if you choose to allow yourself reflect what you believe in and be unique, people will judge you for your differences. Changing yourself for other people will have the greatest effect on you, but do you want to change yourself because you want other people to like/appreciate/respect you, or because you think the change will be beneficial for you as a human being?
Stay true to what you believe in, opinions aside. Your uniqueness is a gem.
If it's getting to the point of something more harmful to you, there are people who can help. I would suggest going beyond the means of the forum and seeking out a counselor (if you're still in school, they often have at least one at the campus) if you feel the necessity. My inbox is open if you ever want to talk. Keep your head up, consider to peek into the why do I think/feel this about said subject (you'll learn something from this) rather than a constant why do I/why do I not, and the path you leave behind you will show your resilience and perserverance through it. <3