So, Atwood's book, which is worth a read if you can handle her style (Some people bounce off it because she doesn't write in chronological order, or rather doesn't present the story that way) has an epilogue.
In that epilogue it is very explicitly stated that the roots of Gilead lie in a collapse of the *Caucasian* birth rate.
Not the human birth rate. The Caucasian birth rate.
The Handmaid's Tale is about white supremacy and it is more than anything else about the way white supremacy treats white women. Which I have personally witnessed. There are cults of white supremacists in this country who...let's just say I know of at least one incident of a woman being manipulated into becoming, essentially, a Handmaid. That actually happened.
Serena Joy is the white woman who becomes complicit in her own oppression, as so many do.
It's about the white fear of being demographically overtaken.
Where are all the Black and Brown people? In the Colonies, fled to Canada, or perhaps Gilead isn't the entire U.S. That's not clear and probably not meant to be. Note that I haven't had chance to watch the show yet, so it could be clarified in the show in a way it isn't in the book.
I do challenge people to read the book, even if it isn't an easy read at multiple levels, before they criticize the show for being racist.