There are a couple of ways you can do this. The first is to use the LOAD DATA INFILE statement. This is how you do the importing directly into the db using SQL. There is a command line client tool that helps with this from MySQL called MySQL Import, here is a link to it:
mysqlimport.html
You can also do this completely with some other third party tool, there are many out there. I think one of the most common is Navicat, but there are many others as well. I also found this link to an online utility that helps to do this:
mysqlimport.html
I don't know anything about this site or the app that it has in regards to security or privacy but it seems to be used a bit.