Racist slave masters had sex with black women (rape or not) during the whole period of slavery. That they were black was totally irrelevant and it didn't make them any less racist . In fact, probably made them more so.
The psychology of being intimate with someone you despise is interesting. It must be partially a control issue and something else...

It is: power and control. If you are the dominant side of the exchange, then you are exerting your dominion over that person. If they are the "slave" or sub, then they give up control to the you (sometimes willingly, sometimes not).
If Milo is a bottom (if I had to guess, I'd say he is) then he would be giving up control to a black guy. Although, I see your point: it doesn't matter who is attached to the d!ck if all you care about is your own satisfaction. It becomes quasi masturbatory and not really intimate.