In addition to the old walled city (Vieux Quebec). make sure you visit the Citadel (military fortress) just to the west of the old city. This is still a military base, and Quebec home to Canada's governor-general. Lots of history too. The nearby Plains of Abraham is where France lost the battle for what is now Canada.
The docks below the city, and the shores on the opposite side of the river, may offer some interesting photos. Coast guard and naval vessels can usually be found at the dcks.
As has already been noted, most things east, especially the Charelevoix region (a UN biosphere) are "naturally" beautiful.