Honestly, your best bet is to just keep the Nikon.
Fuji probably has the best IQ, and the X-M1 will be close to your price, but as of now, I'm not sure there is a really good portrait lens.
Micro 4/3 has the best lens lineup, and acceptable IQ. You could do an E-PM2 from Olympus for the price, but you would need to add the 45mm f1.8 or, at the very least, Sigma 60mm f2.8 to get good portraits.
If you can stand the controls and feel, Sony gives excellent bank for the buck. Not sure on lenses though.