Here’s a trick that I just learnt. You can quickly remove preference panes from System Preferences by right-clicking (or Control-clicking) on them and choosing remove preference pane. Of course this only works on the preference panes that you have installed yourself, i.e. the ones that appear in the “Other” section.
Even though you can just Trash the preference pane file yourself, this is quite a good timesaver if you can’t remember if you installed the preference pane for all users or just your user. Depending on which you chose, the file could be located in the Library folder in Macintosh HD or the Library folder in your user folder. It’s much quicker just to do it straight from System Preferences. Just like if you were Trashing the file, System Preferences will ask you for your administrator password when removing preference panes for all users.