From 4501585:

15) open the native windows file chooser and click on the "My Computer"
    button on the left: the main panel instantly shows a list of the
    various devices avaiable in the system (like A:, C:, etc)
    now click on a local disk, like C: for example, and note that the
    "MY Computer" button on the left is it DESELECTED.
    if you want to see again the devices then click it once again.

    now open JFileChooser and do the same: the "My Computer" button remains
    selected. if you want to see again the devices then you cannot single
    click it, because its already selected: you have to choose another button
    and then click "My Computer" again.

Reproducible w/ 1.4 on Win2k and 1.4.2 on XP.  With 1.4.2 on Win2k, the My Computer sidebar button doesn't stay depressed at all.