Welcome to MacTalk!
If you're in an app and you press the home button, think of that as putting the app in the "background". When an app is in the background, it could possibly still be doing things, or just be waiting for you to switch back to it.
When you double-tap the home button and tap-and-hold on an app until they start wiggling, you can press the X to remove the app from the background — all that means is the app won't be doing anything in the background anymore, and will have to load all of its resources the next time you launch it.
Force-closing an (unrepsonsive) app is another thing entirely, it's like killing the app while it is still in the "foreground" — and to do this, hold down the power button until the "slide to power off" screen appears when you're still in the app, then hold down the home button until you're dropped back to the homescreen.
Hope this clears a few things up!