hmmm, where it works sometimes, but not other times, i would suspect a server level issue, perhaps a memory issue.
If you are on a shard hosting server, it may be that one of the other sites on the server is using the servers memory causing the file manager to fail on reading the contents of the directory. perhaps you can request that your host put your site an a different server.