A 24-105 lens already covers a very usable range and on top of that, it's just impossible to have the ideal/perfect lens for all the possible situations… That being said, I do understand you as I find myself in the very same situation pretty regularly.
I agree Francois, the 24-105 is in my opinion, is a great lens, especially for a general 'walk about' lens. But I have also had something of an ongoing love affair with my 16-35 and use it a little too much if I am being honest, so why the heck I didn't take it with my I will never know. A senior moment perhaps???
Although in saying this, I have also learnt over the years, that if I take all my lenses out with me (about 8, including a 150-600), because I know I am going to a location where I will be able to work from the back of the car, I find that usually, I never take a single shot, or even take the camera out of the bag - it's like a jinx to me or something when I take everything. Yet if I go out under prepared, then.... well, look at the result above

Dave