export for grading. Color gurus needed

[fredjeang]

Hi,

Questions and more questions...sorry about that.

I'm looking to enhance the export settings for grading in external applications.
I went obssesive about it these days and searching the right info in internet is an incredible time consuming as you know.

What I normally do is once the footage is edited in DNxHD, I transcode to 4.2.2, then import in autodesk, grade and export again. I'm not using Avid DS otherwise it would be straightforward.
It works. I'm doing it because I've been told so. Do this, do that...but now that I'm progressing those questions legitimaly come.
I'm not yet aware of the best practises (if there are such things as best practises because looking in the internet it seems as much messy as always)

I've found many times that DNxHD is used for grading but I don't agree with this choice.

Here is what I do for a 720:
Edit in DNxHD 60.
Export in Uncompressed 10bits in quicktime  
Grade in Autodesk
Export from autodesk in uncompressed 8bits in quicktime


Autodesk can work with almost whatever codec available. I'm on pc and use Avid.

Wich prefered container?

Is there a prefered export setting (or settings) the colorist experts would highly recommend for grading and that can be admited by all kinds of applications for that purpose?

It's not that I don't understand the requirement, it is that I don't get the all picture and possibilities, I'm lacking information-experience and there is too much messy infos, possible choices etc...
There might be a "best practise standart" somewhere, I'm sure.

Is there such a list of what we could call, safe formats for grading safely?
Thanks.



 

[bcooter]

Hi,

Questions and more questions...sorry about that.
 snip

Wich prefered container?

Is there a prefered export setting (or settings) the colorist experts would highly recommend for grading and that can be admited by all kinds of applications for that purpose?



 

Short answer to your question is, the client will define the format.

Long answer is NO.

Not unless you know exactly what your viewing medium is.

In other words if your working in a closed loop episodic TV series you might have a standard codec and size.

Right now 2k regardless of Codec is pretty much the standard.  Mainly because most NLE's work in 2k, also 4k chokes most current systems down.

Remember streaming is the next delivery standard and the CEO of netflix has no doubt that computers will be playing 4k high def in a few more years once most connection if fiber optic.

Regular high def TV probably won't go past 2k for another decade.

Now does that mean you should work only in 4k?   Depends on the market, but as of today it's a 2k world, at least for editing and output.

For shooting file size is less or at least equal in importance to compression and file depth.

Today the only option is to use what you or your client's can afford and try to find out what the final destination of the video is for play.

Most client's don't know as it's an ever changing process and most people just get caught up in file dimensions (just like the still industry).

With still digital capture in the early days most of us were asked for file sizes in the 200 mb range.  Now that the process has matured few people ask or even accept a final image over the standard page size at 350 to 400 ppi.

Now with motion images we're getting the same requests of "make sure it's shot 4k".   Not that any client can actually play 4k.

Still as with everything perception is just as important as reality.

The real answer is do what you can do within the budget. 

Money and time define everything and even if you could economically shoot, cut, color and effect in 4k or 5k to future proof your video, I'm not too sure that any video or motion piece should last for 10 years.

Remember we are now into instant marketing, instant play and rarely is there a reason to keep any image still or motion in circulation for years and years.  I would think 12 months would do, maybe two years and by then there will be new cameras, new faster computers and new ways of producing and delivering and most importantly your clients will be producing new more modern work, not just replaying old content in a high file size.

I wouldn't worry about it, other than find a way to make the content compelling.  That will go further and pay more than any technical advantage, (both in still and motion).

Be careful.  The cutting edge is normally not as profitable as the current standard and usually just gets in the way.  You spend more time learning and testing than actually working.

Also watch the money.   I've seen people buy and buy from still to motion in numbers that will never turn a profit.  Adding $40,000 in computers, another $40,000 in software takes a long time to recoup the investment, especially since the market for still and motion is in flux and seems to pay less per year.

Look at Netflix.  When it started it was $20 per month, then $16,  now it's $8.    That goes back up the chain and effects the budget of content.





IMO

BC

[fredjeang]

Short answer to your question is, the client will define the format.


The real answer is do what you can do within the budget.  

...Money and time define everything and even if you could economically shoot, cut, color and effect in 4k or 5k to future proof your video, I'm not too sure that any video or motion piece should last for 10 years.

Remember we are now into instant marketing, instant play and rarely is there a reason to keep any image still or motion in circulation for years and years.  I would think 12 months would do, maybe two years and by then there will be new cameras, new faster computers and new ways of producing and delivering and most importantly your clients will be producing new more modern work, not just replaying old content in a high file size.

I wouldn't worry about it, other than find a way to make the content compelling.  That will go further and pay more than any technical advantage, (both in still and motion).

Be careful.  The cutting edge is normally not as profitable as the current standard and usually just gets in the way.  You spend more time learning and testing than actually working.

Also watch the money.   I've seen people buy and buy from still to motion in numbers that will never turn a profit.  Adding $40,000 in computers, another $40,000 in software takes a long time to recoup the investment, especially since the market for still and motion is in flux and seems to pay less per year.

Look at Netflix.  When it started it was $20 per month, then $16,  now it's $8.    That goes back up the chain and effects the budget of content.





IMO

BC

Thank you James.

Actually you are absolutly right. Thanks to bring here the reason and rationalism.

I've found a workflow that works for me and actually works well. Mixing different editing platforms in the chain and I'm starting to get used of it. It may not be orthodox, the way it should be and that is why I'm a bit obsessive, but it worked even better in some deliveries (the very few I've made so far) than the alternatives the video gurus from the industry did. We compared the output and the numbers.

In fact, I'm now spending too much time on trying to enhance skills and trying to catch-up with the techniques and the result is that I'm not shooting anything good, even more, I'm not shooting at all recently because I'm too busy with those softwares. This is I think a mistake, a real trap and your post is reminding me to stop for a while, doing things that works for me and future clients because all that is changing at the speed of light.

The only issue is that grading the 5D2 is not as fun as the Red*. You know about that. My aim is to limit the damages of the non Raw files using a codec that when grading does not make the footage falling appart. So far I had good results with the Avid 1:1.

*But I'm still amazed how good the Canon is in lowlight. I think really superior to the much more contrasted and "vulgar" 7D output and the too sensitive 1DMK4.

I need a master to the max possible res but here I've never saw once a technical sheet that wanted more than 720 (tvs included) and then a package of 4 or 5 different other formats.

[bcooter]

In fact, I'm now spending too much time on trying to enhance skills and trying to catch-up with the techniques and the result is that I'm not shooting anything good, even more, I'm not shooting at all recently because I'm too busy with those softwares. This is I think a mistake, a real trap and your post is reminding me to stop for a while, doing things that works for me and future clients because all that is changing at the speed of light.

Fred,

We all face the same thing. 

Time is the enemy and as you know once you start working in motion imagery, post work is the black hole of time.

Though I understand where your coming from and you should be commended because even if you never become a colorist or editor and devote all your time to shooting, understanding the process fully helps you to direct all phases of the project.

I know that learning to cut taught me more about shooting motion than anything I did.  It also taught me that their is no such thing as too much footage.

We're in the middle of cutting 6 videos while in other productions which makes it even more difficult.

Anyway, I'm getting off topic, but I've found the very first thing I ask a client, prior to shooting and in post, what is their goal, what do they want their product to be, what have they seen that they consider a great video?

The last question is important and not to copy or emulate, but I could be thinking louis vuitton and they could be thinking Forever 21.

Big difference.

We also storyboard out everything.  Prior to shooting with drawn images and prior to cutting with a storyboard of the images we plan to use and descriptions about how every image is used, how long, what effects, etc. etc.

Story boards are a life saver.

One thing to keep in mind.  Learning this process just for the sake of learning is like learning how to drive a car in a parking lot.  You can get pretty good, but without a reason to go somewhere, the motivation is hard to find.  At least for me.

But all in all I don't think any still photographer can fully appreciate how difficult shooting and producing a motion piece is until they dive into the complete process, especially with client input.


IMO BC

[fredjeang]

Fred,

We all face the same thing.  

Time is the enemy and as you know once you start working in motion imagery, post work is the black hole of time.

Though I understand where your coming from and you should be commended because even if you never become a colorist or editor and devote all your time to shooting, understanding the process fully helps you to direct all phases of the project.

I know that learning to cut taught me more about shooting motion than anything I did.  It also taught me that their is no such thing as too much footage.

We're in the middle of cutting 6 videos while in other productions which makes it even more difficult.

Anyway, I'm getting off topic, but I've found the very first thing I ask a client, prior to shooting and in post, what is their goal, what do they want their product to be, what have they seen that they consider a great video?

The last question is important and not to copy or emulate, but I could be thinking louis vuitton and they could be thinking Forever 21.

Big difference.

We also storyboard out everything.  Prior to shooting with drawn images and prior to cutting with a storyboard of the images we plan to use and descriptions about how every image is used, how long, what effects, etc. etc.

Story boards are a life saver.

One thing to keep in mind.  Learning this process just for the sake of learning is like learning how to drive a car in a parking lot.  You can get pretty good, but without a reason to go somewhere, the motivation is hard to find.  At least for me.

But all in all I don't think any still photographer can fully appreciate how difficult shooting and producing a motion piece is until they dive into the complete process, especially with client input.


IMO BC

James,

Your situation is a little different (or a lot!!) than mine. Putting yourself into the tasks of editing is indeed IMO necessary or welcome to both enhance motion and being able to lead better your projects at all stages.
And when you spend time in the cutting, you are surely more aware of your own shooting. Editing is probably the best master.
But you have a name, a reputation and a structure that allows you to delegate when time and techniques require so. In other words, you press the shutter, sign the cheques etc...and can surrownd yourself with the creme de la creme technicians, assistants...
So far, I've just been assisting in sometimes high-end structures, other times low-end, exchanged jokes in english with a few international top models and collected a bunch of backstage shots that could fill a  collection of howlers in fashion photography.

I'm not really a leader so far that I know but more a second. I shine with others but when it comes to my personal work, it is a complete disaster. I can't motivate myself if there are no assignments. I'm really amazed how passionate are most of the members in Lu-La. They shoot for shooting, for passion. I can't do that, I just get bored and can't see the meaning.

When people call me, I'm able to display an incredible amount of energy, commitment and I'm damn efficient...but when it comes to my portfolio, I start to turn around in circles, questioning whatever aspect of the reasons why I'm doing it and ultimatly reach such a level of annoyance that I simply end to give up the series. I've been learning about that and sometimes the hard way.
I have any difficulty to find my space and motivation in a structure, in fact I do not know if that is my nature or if I'm simply too shy to lead a personal project, but completly useless to build the same structure, name and reputation for myself. It costs me horrors, specially the business side. I hate those dinners with the clients, those night-club vodka cups where everybody dance and seems to have fun except me etc...
All I see are images, colors, light and hear sound. It is like I'm in an external world and I'd be seeing a movie. Can't be in place without thinking about the possible image but the possible image never appears if I do not have a good reason.

I wish I could have been in this texan race track and did such a good peice of movie like you did without (if I'm right) any assigment.

What's the bridge with the video editing?
Well, I really enjoy it ! In fact I enjoy the motion imagery much more than stills. I think it comes from the fact that I used to draw zillions of comics when I was a kid and I was crazy to put sound on it. In short, I'm way more "at home" when things are moving and with sound than with stills.  Also in my youth in Paris I was very close to the cine plateaux, motion is something more natural for me than stills even if I choosed to do fine arts instead of a Louis Lumiere. But so far I can't relly on any crew technically skilled, I'm almost on my own.

Editing for me has an added sense. A part from the fact that it allows me to self-produce my current horrors under a fake name, It will be sailable and that is probably more important because I have a serious doubt I can make it by myself in the business. I'm not fast enough today in editing software, specially with the short-cuts and lack of certain kind of knowledge, still in the middle of the learning curve. But I'm working hard to catch-up. I draw story boards easily because I know how to draw, made friend with a guy how has a music studio etc...completly in indy configuration but I'm enjoying like never before with video.

So the paradox is that I know I should be back shooting and press the shutter myself, but...shooting what? If the only thing I do is assisting. Editing at least I'm doing something usefull for me and on short term, for others.

It's funny, I'm bombed by work recently, I should be happy because in this crisis it is not easy. Instead, I'm barking because it is not motion work...
 

[smthopr]

I'm not an Avid user or Autodesk user, so I tread lightly here

DNxHD 60 is an 8bit codec that you're editing in. OK.

Why not just send the DNxHD 60 files to Autodesk for color grading?  What matters here is the math that Autodesk uses to calculate the render.  If Autodesk has a setting for this, set it to the best setting that your hardware is capable of supporting.  Something like "16bit floating point".

Exporting to 10bit uncompressed quicktime is just an extra, un-needed render.

You can render out to a 10bit uncompressed codec or DPX files for mastering.

There is no reason to change the size of your format (720P/25) until you know where it will be displayed.  And certainly no reason to ever resize to 2k.

Be also aware:  10bit videos have varying black and white levels depending on the standard used.  Sometimes 0-1023.  Sometimes 64 as black and 940 as peak white, which is standard for 10bit video.

Be again aware:  rendering into and out of quicktime codecs can result in changes to the above black/white levels and/or gamma settings.  Therefore, the quicktime that you render from Autodesk, might have different values than you're expecting.  It's a real rat's nest of formats in video with all kinds of gotchas....

For example.  I recently color graded a feature film shot with the Red camera and rendered to linear DPX files.  The client required the some shots early and we supplied them with the DPX files to send to their FX shop.  The FX shop only works with Apple Pro Res, so they converted to ProRes using a combination of software to get there.

Then they required the entire movie delivered in 10Bit Apple Pro Res.  After a 25 hour render we discovered that a gamma shift had occurred in the conversion to Pro Res using Apple Color software.  This required a re-render to correct to the original gamma.  But the FX shots all came back with a gamma shift and a black/white level shift, so that they no longer matched the graded footage. This all required a different conversion and all needed to be rendered once again...

I hope this is somehow helpful   Best of luck to you!

[fredjeang]

Thank you Bruce

Yes, this gamma shift with Apple is a problem. I was on that these days and I think I have it resolved. I need more transcodings to be sure about my setting.

Avid color space is always broadcast legal, 601-709 so you go from 16 to 235. Autodesk is RGB.
But when you import a footage in Avid you have to choose the colorspace. Choosing 601-709, you simply tell Avid: "that's fine, this footage is legal" even if your file is RGB so Avid won't convert.
That's a good trick to know. Exporting in RGB if not for tv.

By the way, if Chris Sanderson read this thread, post, I'd like to ask him about that fact because I saw that his videos have very little gamma shift (less than normal) between Apple and others.
I did import-export bdsm with one H.264 file from the video tutorial (see the way I treat your works!...) and the shift is hardly noticiable. Are you working in RGB color space?
These shifts always happen when using external softwares like After-effects or Smoke-Combustion etc...and with the Apple codec.

I agree that the 10bit render step is unnecessary but I haven't find a way so far to send directly an DNxHD to Autodesk from the timeline or the bin. There might be a plug-in for that.

I sometimes edit in DNxHD x90 or above, wich is a 10 bits codec and it does not slowdown. Why? because I might grade directly in the editor. Is there a real difference? Indeed. When I do the workflow from the very beginning in 10bits, then import the file in Autodesk, also in 10bits (it can go to 16bits floating) but in rgb, the grading is more stable, the file does not fall easily appart. I really beleive in the bigger the best, even if you downsample a lot. Autodesk also works with DPX but haven't reached that stage so far.
Like in stills, it works that way. More your primary information is huge, better are the results regardless of the size output. I won't of course never upsample above my source file.
It does not bring extra quality, no way, but yes it avoids more dammages.

Anyway, my provisional conclusion is that if you have worked with let's say After-effects and want to reimport the footage in Avid, this time you need to tell Avid to import in RGB. You'll be able to experienced the gamma shift at a non delivery stage because Avid will reduce the blacks and whites range to the broadcast standard.
Then you can re-correct and export in whatever RGB or 709 but you're not going blind.

The problem stands when you work in RGB and not make sure that your gamma is legal. Keeping the values in the scope between those 16-235 extremes except if you plan to do superwhite/black for keying.

Resuming: importing RGB in Avis assures that you are in broadcast space, while importing 709 assures that your file would stay as you did.

What happened to you could have been avoyed, we learn the hard way and sometimes it is the only way. Just before rendering all a movie, exporting 10 seconds of the timeline in different space settings and review the all in different platform. The problems are when many intermediates are involved like in your case the Fx shop. I did a similar mistake 2 months ago and they call me from L.A. My face went red. I sent the same day from ftp the correction and all was fine but...Everything in the hurry, no time to check can sometimes have undesirable consequences. Now I check everything at least 3 times and ask external shops for approval on sample, like in fact I did with printers.

I know that some users here pointed, with some good reasons, that importing Red footage should not mean to grade before the cutting, but in the case of Raw I disagree with this because it is loosing a huge advantage of the raw video. Keep in mind that the practises where built from a time where Raw didn't exist.
I still think that grading the original Raw directly from Avid, and you are correcting real Raw from the source file not jpegs, at the cutting stage is really the best way to extract all the power of those cameras. Yes, it breaks the tradition but the tradition was not made for that kind of files. If your source file is 4K, you are actually correcting 4K in the timeline, even if you are in 2K in the editor.
It is exactly like you had a jpeg, and then you call the raw master when correcting. (linked to AMA). What can be more powerfull than that?

Those gamma issues and the communication between platforms is really interesting and I'd be very please if we could have more imputs here on that matter.

[fredjeang]

I'm not an Avid user or Autodesk user, so I tread lightly here

DNxHD 60 is an 8bit codec that you're editing in. OK.

Why not just send the DNxHD 60 files to Autodesk for color grading?  What matters here is the math that Autodesk uses to calculate the render.  If Autodesk has a setting for this, set it to the best setting that your hardware is capable of supporting.  Something like "16bit floating point".

Exporting to 10bit uncompressed quicktime is just an extra, un-needed render.

You can render out to a 10bit uncompressed codec or DPX files for mastering.

There is no reason to change the size of your format (720P/25) until you know where it will be displayed.  And certainly no reason to ever resize to 2k.

Be also aware:  10bit videos have varying black and white levels depending on the standard used.  Sometimes 0-1023.  Sometimes 64 as black and 940 as peak white, which is standard for 10bit video.

Be again aware:  rendering into and out of quicktime codecs can result in changes to the above black/white levels and/or gamma settings.  Therefore, the quicktime that you render from Autodesk, might have different values than you're expecting.  It's a real rat's nest of formats in video with all kinds of gotchas....

For example.  I recently color graded a feature film shot with the Red camera and rendered to linear DPX files.  The client required the some shots early and we supplied them with the DPX files to send to their FX shop.  The FX shop only works with Apple Pro Res, so they converted to ProRes using a combination of software to get there.

Then they required the entire movie delivered in 10Bit Apple Pro Res.  After a 25 hour render we discovered that a gamma shift had occurred in the conversion to Pro Res using Apple Color software.  This required a re-render to correct to the original gamma.  But the FX shots all came back with a gamma shift and a black/white level shift, so that they no longer matched the graded footage. This all required a different conversion and all needed to be rendered once again...

I hope this is somehow helpful  Best of luck to you!

Bruce,

I've been told about that link about ProRes if you are on windows platform and want the hability to encode in the Apple format: http://www.telestream.net/flipfactory/overview.htm

Really!!!! I should build a political party wich would be called: MUSES = movement for unification and standardization of the editor softwares.

This is the jungle law.

more pornography: http://www.capria.tv/2007/06/dnxhd-vs-prores-422-redux/