This should work fine, nothing special required. I have 4 different Epson printers (2 3800s,11880,7900) connected to airport Extremes in 2 different locations, I've never had to do any setup on any of the printers. Virtually identical to what you describe.
I assume you are connected via ethernet, not the USB port on the Airport Extreme. When you add your printer, you should have 2 choices show up in the list, Bonjour and Epson TCPIP. I use the latter, however I have used Bonjour successfully as well. You didn't say which OS X version you were using, and I assume you've installed the latest driver.
One thing to try ... disconnect the printers from the network and remove the printers from the list. Use the installer program to uninstall the driver. You may want to repair permissions at this point. Reinstall the driver, reconnect the printers, and add the printers. I have had this cure problems for me with epson printers on a couple of occasions.
It think there are a couple of firmware updates, I know there is a network firmware update for the 3800 that was released late last year. May want to be sure all the firmware is up to date. Not much help I know, other than confirming what you want to do does work quite well.