Unable to wake display (hdmi) after putting display to sleep unless power off the display and powering it back on

I am not talking about putting the OS to sleep (altho that also doesn't work), I am talking about putting display (and only the display) to sleep (via hot corner for example). Once I do that, I will not be able to awake it via mouse or keyboard, I must first tap on the keyboard and then power off the display and powering it back on to see the graphics.