Why is it socially acceptable for non-whites to form exclusive groups, but not whites?

There are lots of people who are black and proud, asian and proud, etc. These groups are able to form exclusive fraternities, sororities, clubs, you name it, and it is not frowned upon. However, if white people were to form such exclusive communities, then it would certainly cause some contraversy and be pegged as being racist. Why do you think there is this double standard?