Sorry about the title. I followed these answers as suggested by some users on Ask Different. They've solved my problem.
However, when I opened up Mail.app to inform it I want to use Google Notifier as default, like so,
in the time it took to do this many emails downloaded to my Mac's Mail.app client.
How do I safely remove these emails and 'close' my Mail.app 'account' so to speak. I have no need for a local email client.
Before following these instructions, I suggest disconnecting from the internet so that you don't download any more messages.
Open Mail and go to the Accounts tab in the preferences, then select your account if necessary.
The first thing you should do is ensure the account is set up as IMAP and not POP. If you used the Gmail type to set it up, you should be safe, but I believe Gmail also supports POP access. If it is set up as a POP account, you don't want to simply remove the account from Mail, or you will lose the messages you accidentally downloaded. See below for a way to get those messages back to the server.
Once you have made sure you won't lose your messages, you can simply press the minus button in the bottom-left corner, below the accounts list. As Calum said, this will delete messages in the Inbox, Sent, Trash and Junk folders, as well as folders which are stored on the server instead of your computer. You will have to delete the others manually.
If you want to make sure you don't leave anything on disk, open the HOME/Library folder in Finder. Delete the Mail and Mail Downloads folders from there. This will remove any messages and account data which Mail left behind.
If you have Gmail set up as a POP account, you can get your messages back onto the server by forwarding them to yourself. Before reconnecting to the internet, remove the password from the account information. That way, if Mail tries to check for new messages, it will have to ask for your password and you will be able to cancel it. You will still be able to send messages.
Reconnect to the internet. Find the messages you want to save and click the Forward button in the toolbar. Finally, send them to your Gmail account. They should now be on the server again, although you will need to be careful if you reply to them as it will replay to yourself instead of the person who originall
Delete the account from the Accounts pane of the Mail.app preferences window. This will delete all settings for that account, as well as the local copies of everything in its Inbox, Sent, Trash and Junk folders (so be very sure that this is what you want to do!)
Any emails you'd moved or copied into other local folders (i.e. into the "On my Mac" section of the side pane) will remain, however, so you will have to delete those separately.
