I have encountered this as well. It doesn't matter if you are playing a game in Demo mode, Regular Mode, Or just listening to the midis through the maker itself, you simply do not hear any music.
I have also encountered something curious about it, if you have a midi playing in RPGMaker and you open up a midi (any midi) file in windows media player you hear both midis. This leads me to think that this problem is caused by windows media player, but I haven't found a solution.