Because so much of fantasy takes place in settings that in no way resemble the real world, featuring species that in no way resemble human, fantasy writers often have trouble dealing with regular people. This is something that, I think, isn’t as much of a problem for mainstream writers, because they can simply describe the world around them and come up with a reasonably accurate representation of humanity. They can also fall back on the plethora of real-world terms used to describe human beings, racially and otherwise. But using these terms makes no sense if you’re dealing with a world that doesn’t share our political/cultural context. You can’t call someone “African American” if your world has no Africa, no America, and has never gone through a colonial phase in which people of disparate cultures were forcibly brought together, thus necessitating the term in the first place.
That said, it’s equally illogical to populate your fantasy world with only one flavor of human being, which is what far too many fantasy stories default to. Granted, many fantasies take place in confined cultural spaces — a single small kingdom in a Europeanish milieu, maybe a single city or castle within that city. (But how did that castle get its spices for the royal table, or that lady her silks? What enemy are the knights training to fight? Even in the most monochromatic parts of the real Ye Olde Englande, I can guarantee you there were some Asian traders, Sephardic or Ashkenazic Jewish merchants, Spanish diplomats or nobles partly descended from black Moors, and so on.) I get that lots of countries on Earth are racially homogeneous, so it makes perfect sense that some fantasy settings would be too. But whiteness is the default in our thinking for Earth-specific cultural/political reasons. So while it’s logical for fantasy realms to be homogeneous, it’s not logical for so many of them to be homogeneously white. Something besides logic is causing that.
So. It’s a good idea for all fantasy writers to learn how to describe characters of color. And I think it’s a good idea to learn how to describe those characters in subtle ways, since they can’t always rely on Earth terminology. Now, doing subtle description increases the chance that the reader might misidentify the character racially — and to a degree, I think there’s nothing you can do about that. You’re working against a lifetime of baggage in the reader’s mind. But you can still insert enough cues so that when combined, they’ll get the idea across.