The folders are empty; you add your own things to them, either things you've made or things other people have given permission for you to use. Yes, usually you hafta draw your own stuff, and you HAVE to if you're making a commercial game (and if you're wondering what "commercial" means, it means a game to sell
) unless you have the direct permission of the creator to use the resources.
Gamemaker, like RPGM, comes with its own sprites, but they're just there so you can use them to get started. With GM, it's about YOU making the game, not really like RPGM where you have lots of different monsters and stuff and can make lots of games with what you're given and never really have to make things yourself. GM is mainly for people who want lots of flexibility and have the time (and patience) to learn how to use the program efficiently. RPGM seems geared more to making good games fast with little programming or even knowledge of how the things work, basically a plug&play program. Handy, yes, but I prefer GM's flexibility (though i WISH they'd make a decent all-in-one RPG tutorial/plugin; all the RPG tutorials are either single things like battles, walking, etc or just very basic "how to code this situation" things).
When I need an RPG I use RPGM, when I need any other game I use GM.
GM isn't useless but it takes a while to get used to.