Each gallery design is in the galleries folder. If you want to make you own, you can just duplicate one of those folder as a starting point, then edit the files as needed. You will see various php, js, and css files for the gallery. The other thing you should do is edit the admin/design.xml file to give it a custom name. Basically, every folder it finds in galleries can be usable as a design assuming it follows the correct structure.
There should be some other posts in this forum that can give you more details that will help out making a custom design.