PSUs have a +5V Standby rail which is always on even if computer is off, and most mother boards use it for powering USB devices so they can charge on power off. The supply is also usually to the onboard LAN adapter and PS/2 ports to allow wake on LAN and keyboard.
The configuration is various, it may be BIOS controlled, jumpers on the board or power settings if you want to change it, alternatively you could just really turn off the power at the main switch.
