That is a lot of math and angles to figure out to make it work right correctly. Looks like you need to know how far away the building is from where you are shooting and the distance between floors of the building you are shooting from, not to mention the vanishing point created behind the lens based off of you perspective (the starting point of your angle) and a good deal of knowledge of trig and maybe calculus.
My degrees are in math and I dont think I would want to put that much effort into shooting. But thanks, maybe one day I will have a job where I absolutely need that.
Just needs a little bit of basic trig...
From the height of the building, the height of the tripod, the max accessible distance from the building and the FOV of the lens you calculate the (max) distance to the Virtual view point, but it gets more complicated when you have a three-dimensional problem creating a perspective/parallax view ¿so that you can stitch without having to scale? ... this would call for accurate surveying.
I think GCSE (16 year old) maths should suffice.