Basically I'm wondering if shooting in NEF will allow me to increase the output size without losing quality
My answer is a qualified "yes". RAW (which includes NEF) aids enlargement in two ways. One is by eliminating JPEG artifacts, which grow increasingly apparent the more you enlarge. The second is by allowing you to get an image file with minimal sharpening into Photoshop. This is important because you don't want to apply sharpening until after you have done your upsampling. Neither of these benefits of the RAW format is likely to make a dramatic difference in the quality of your enlargements, so you will need to do some comparison prints to see if it is worth your while. In general, the advantages of the RAW format lie more in other areas.
I'm changing the file size to what I need with the proportionate DPI
There are some who swear by this method, but I read somewhere that those who understand the math say it will magnify any enlargement artifacts that might appear. What you are talking about is upsampling by a factor of 1.5. It very much depends on the content of the image, but I think most of us would consider this a non-extreme degree of enlargement. I suggest starting with the same image file, duplicating it thrice, then applying your 20 pica method to one copy, Bicubic Sharper to the next, and Bicubic Smoother to the third. Print these identically then evaluate.
Currently I'm getting 40 Picas of height (6.66 inches) at 300 DPI and my final size that I need is 60P6 or 60.5 Picas of height (about 10 inches) at a minimum of 250 DPI
Just to prevent future confusion, most people who do digital imaging use a different convention. We use DPI (dots per inch) only when referring to the actual application of ink by an inkjet printer and use PPI (pixels per inch) when discussing everything else related to the printing process, including enlargement. Also, I am more familiar with inches than picas; I suspect that holds true for others as well, although most Europeans would probably think in centimeters.
Good luck, and don't hesitate to ask if you have further questions.
