I have this vague memory of a menu item or button in VSD that when clicked on or whatever removed your site from its server.

I cant find it on the current version so I assume it has been removed from VSD.

The host of one of my sites has asked that I back up and remove all files/databases from the site before it is closed.
What would be the preferred way of doing this in VSD?

You should be able to login to your host and manually delete them. If you want the files backed up keeping your VSD site vnu and all the files it uses should be all you need unless you used an external ftp app to upload other files.