That's what race means, dude. More than one definition. One is, obviously, humans as a whole. The other one is divided culturally, and with cultural differences there obviously comes physical differences. There's a German race, an Italian race, an African American race, etc...etc...
It's not so much about just the color of the skin. Saying that all Germans are jew murderers is just as "racist" as the whole nigger thing.
It comes with the territory, you know?