I had the same kind of diet you did growing up but that doesn't mean much. What works for you, works for you and that's about all one can say.
I wonder if you really did have the same kind of diet, apart from fish and eggs. I was also given lots of fresh vegetables. I recall my mother once beating me with a wooden spoon because I refused to eat the spinach on my plate, because I didn't like the taste of spinach.
White bread was banned from my household. Wholemeal bread, Kellogg's All Bran, Wheat Germ, Full-Cream milk, pure cream, cheese, yoghurt, and of course, unrefined raw cane sugar, were among the staple foods when I was young.
Now in my 70's, my diet is pretty much the same. I always insist on full-cream milk and I still have a mixture of Kellogg's All Bran, Wheat Germ, Whole Grain Oats, and fresh fruit for breakfast.
I eat in moderation. I don't avoid or worry about natural fats, and I'm not overweight. However, I also exercise regularly, taking a long walk on most days and sometimes do intermittent jogging for brief moments up a steep hill, to get myself breathing really heavily.
