I was there last week. You don't need a guide, just cash. I went to lower Antelope, $26. I stayed in lower the whole time but as I remember from past trips the price also covered upper. Still it's steep to walk around some rocks. At upper you have to park and they drive you out to it. At lower you pay the money and walk about 50 feet over to where you climb down in it.
Compared to the summer it's not nearly as good. The sun is so low in the sky it doesn't light up many sections and some only get lit up partially. You can still make some great images, some of the areas that do light up work great. I have my doubts about being able to get any sunbeams in upper. If anyone has managed it this time of year please post.
On the plus side the crowds are gone. I was in there 3 hours or so and there was maybe 10 people the whole time. A British photographer that was there told me it was crowded the week before with a busload of Italians, but barring that you will have tons of room. Also the temperature is nice. It was in the 60's, much better than baking in the 90's in the summer. It was completely cloudless. One of my previous trips to Antelope was ruined by big puffy summer clouds, that would float by for 20 minutes just as I got set up and dialed in.
Take a tripod. For great shots if you don't have one don't come. Shoot bracketed exposures. If you have an angle finder this is the time to use it as you're often shooting up at an extreme angle. The dust was much less than in the summer, I guess with less crowds, still if you don't have to switch lenses it's better. I used a 24-70 on a full frame and 70-200 on a 1.6x crop and got some nice stuff with both. More on the 24-70 but the longer reach is nice because you get stuff that not everyone else has shot a thousand times.
Page itself is dead in the winter. I stayed at the Holiday Inn Express and there was only 6 or 7 cars in the parking lot. Many of the restaurants are closed.
If you want to try for the lottery to The Wave, keep in mind that in the winter they move the drawing over to the Kanab, UT field office, about 80 miles from Page. Information for it is on the BLM website.
As you go north out of town, across the bridge, and then 2-3 miles more there's a sign for a scenic overlook, it's a gravel road going uphill to the right. This was really nice at sunset.