Can't Wake Display from Sleep (HDMI)

Hey guys,

So i built myself a Ryzentosh with a Sapphire Pulse RX580 8GB card.
If I use DisplayPort, sleep and wake works fine. But if I use HDMI, the display won't wake up after sleep. Just says "No Signal". I'd either have to change the HDMI port on the card or restart the computer.

Any idea what could fix this? It's not a big deal but I would like to get it to work cause I use the same monitor with two computers.

Thanks in advance!