I like Kevin's approach: When in doubt, pack everything... ESPECIALLY the wife and he's blessed that she comes with complementary camera gear as well. That just beats everything. Highly recommended. By contrast, sherpas are no better than a 2nd best option.
FWIW, I find when I take what looks like everything, it doesn't change what actually happens: I tend to work a vary narrow palette for "the carry", preferring to put the camera/lens combo in a ThinkTank holster with one other lens in my pocket or a ThinkTank pouch on a belt. There are compartment bags like Kevin shows which are great for the "base station", but awkward on site. So far, I've bought them used fairly cheap. Tripod + Head solution for travel... I'm still working on that. A RRS nut, I've their monopod, tripod, and a head or two, but still not figured the travel thing out to my satisfaction.
Add that I've heard Scott Bourne of Photofocus and formerly of Fuji Mirrorless podcasting has recently switched from Fuji and moved away from Canon as well to Panasonic MFT for the Leica lenses and greater than APS-C compression specifically for birding. That said, back when I was shooting Fuji and had that same Fuji 100-400, I found that a Think Tank lens pouch works fairly well. You can hang it off one of their belts. Did I like the it? No. That is the lens that -great as it is and sharp as a tack- broke this camel's back and sent me off in search of smaller lenses: Olympus Zuiko's first and then Zeiss using Metabones adapters, and ultimately pushed me out of Fuji and into Sony A7II for the IBIS. Think Tank holster might actually carry the 100-400 if you open the zip-up built-in pouch.