Well, firstly, most rechargeable 1.2V batteries are Li-MH, not lithium-ion.
Assuming you're talking about Li-MH, one issue that leads to manufacturers not recommending substitutions is that alkaline batteries have a gradual voltage falloff when discharged. Li-MH on the other hand are quite stable for a long time, then fall off a cliff. That means that, if the low battery detection in the device is designed for alkaline batteries, it may give you such a short period of warning with Li-MH that you effectively have no warning at all.
As regards PSU voltage, 6V should be fine, assuming a four-cell battery pack. The only danger is potential surge current. Alkaline batteries typically can't deliver much surge current, but a big power supply can. That can result in issues over the long term. But if the device was designed for an external power supply, that shouldn't be an issue. If it wasn't, and you're modifying it, then I'd take steps to limit current when the power supply is first plugged in.