You can delete a gallery in the same way, just hover over the gallery row on the Choose Gallery page, and click the red X to delete it. That will remove the gallery and any images it contains. There is no need to manually remove anything from the database.
-justin