It seems that 25mb file before was a fluke, it's still logging errors, I've had a few memory_limit errors, and since then i've upped it to 256M memory_limit, and i am still getting errors including memory_limit and internal server error..
I've abandoned the whole email attachment idea (why does it require so much memory?).. Instead i'm saving the attachments on the web server and sending a link to the file in the email..