I use Audacity and find it very easy to use. It produces reasonable results and has most of the post record "bells and whistles".
This question is about what is best for absolute beginners.
I agree with most of what has been said but some advice forgets the "beginners" part and offers complicated, expensive, or hard to use options for a beginner.

I also use a small mixing desk for eq levels and fx. It is easier to control, more user friendly and produces a better sound than that obtained trying to do these things post record. These are quite expensive but are a necessary piece of hardware and a worthwhile investment especially if you are going to play live or gig.

Re Microphone. This subject and question has appeared on this site time and time again. The simple advice is there are many makes and models to choose from. A decent quality mic is essential for any good recording. Try a Shure SM58 (around $100), or my preference Audio Technica at a similar price.
Do not buy a cheap "karaoke type" mic it will only produce very poor recordings.
Do not buy a condenser mic. Unless you have a phantom powered mixer, it wont work.

http://www.soundclick.com/bands/songInfo.cfm?bandID=687727&songID=5238055