You asked about downside. I run a H'blad 500cm and P20 back BTW. Tolerances are tighter for digital as focus is far more critically seen. I was fine I am an amateur the 'blad was owned from new and was serviced. You may be looking at thrashed ex pro bodies and will need to buy wisely. With film every Zeiss lens was perfect. Now some are much better than others such is the analytical quality of digital and pixel peeping. You will need top class glass with the same caveats as above. This is not exclusive to 'blad the same applies to all the MF makers.
Is it worth it ? For whatever intangible reason/s, and regrettably it appears we often cannot prove it by measurement and thus go around in those frustrating forum argument circles from time to time, MF has a different look/feel than a DSLR or digital range-finder. The quality argument is much as it was when Leica owners with the best film and careful processing and a tripod built for a 10x8 could claim to approach MF. Large numbers of pixels in 35mm body with computing power approaching a Mac built in are in that position and as ever the discussion continues. My advice, hire a MF digital outfit for a day, there is no substitute for hands on. A day with a Canon 1DSMk111 showed me conclusively that for my work AF was a complete waste of time. I know without it others could not produce what they want so I don't go around forums saying what rubbish it is. However many opinions you collect, mine included, it's yours that counts. Suck it and see.