The extension manager should automatically remove any files it installs, so if files are left behind it is a problem with the extension manager.
Jimmy correctly identifies the correct way to ensure all files are removed is to rename the user configuration file. That is the only place we install files, so if the extension manager does have a problem, that will ensure complete removal.