Ask yourself this: why did the ref not take action during the game?
No one heard it. No one saw it. One player claims something has been said. There's no evidence it happened.
I was racially abused/threatened SIX times in college, and each time there were between 2-10 witnesses who confirmed what happened, but did anything happen? No, because I'm white and each time, the principal said it was probably just banter. And here, a player's reputation and job is at stake due to hearsay...