Wednesday, December 3, 2014

The Politics of The Green Screen Shoot

"Green Screen" by Peter Pearson
licensed CC BY-SA 2.0
Often vfx professionals are confronted by other film or video professionals who see the world from their perspective without understanding our world, or who have notions about quality that will cause trouble down the line.

Case in point, I read a LinkedIn post today from an editor-animator-camera operator who has a DP wanting to over-crank a camera to get a better chroma key.  More specifics: there is synchronized audio, proposed shoot speed is 59.97, mastering speed is 23.976.  DP believes he will get a crisper, cleaner key.

First, let's tackle the technical question and help this editor with the right information, then we will talk about how to handle the politics of the situation where your voice may not be the loudest in the room.

Over-Cranking the Camera and Conforming Footage
To answer this question, let's consider the over-cranking speed adjustment required for mastering first and then discuss the keying.  Again the specs- synchronized audio, 50fps conformed to 24 fps Drop Frame.  In this situation over-cranking may not help and will most likely will introduce motion discontinuities, also known as a glitch or bump or jump in motion.  

The questioner says the DP wants to "conform" the footage to 23.976.  Conforming footage in editing or compositing parlance means to LIE to the software.  If the footage is shot at 59.97, the native or true speed is 59.97, for simplicity we will say 60 frames per second.  Conforming means to tell the software after importing the clip that the speed is not 60 fps, it is say, 24 fps.  This means that it will play 60 frames of clip in 60 frames at a speed of 24fps.  If it were not conformed, it would play 60 frames of clip in 24 frames of master at a speed of 24fps.  In this case, conforming one second of source footage results in 2.5 seconds of master footage.  This will then cause the footage video to be out of sync (behind) the footage audio.  Usually a problem.  So therefore the footage cannot be conformed, it must be retimed.  

Retiming Footage Methods and Their Artifacts
In doing so there are three ways - dropping frames, blending frames and blending pixels.  In this case every 2.5 frames needs to be merged to produce one frame.  If dropping frames, it means 5 frames become 2, so a jump or glitch will occur every 3 frames.  Depending on motion, this will usually be noticeable.  If frames are blended, it means 2.5 frames will be dissolved together to make a new frame, which means details will be repeated and ghostly artifacts will most likely appear, especially around edges of high contrast.   If pixels are blended, it means a new frame will be attempted based on contextual motion of the pixels.  This can result in swimming pixels.  Because  the footage is being sped up, this is less likely than if it were being slowed, but it still may happen where details are at a high frequency, like in hair or fabrics or other textures.  Anytime footage is retimed there is an opportunity for artifacts to creep in.  The vfx artist or editor may have some control over quality, but in the end, the footage and software will rule the result, not the artist, editor or DP.

Now the question is, will the footage look better if over-cranked.  It depends on your primary delivery market.   If your market is broadcast TV, then shooting 60fps may be desirable if you also master at 60fps, because then you can have smoother, cleaner field rate motion.  But today in HD many shows want to see frame motion and like the 24fps look as more filmic.  As the given specs are a 24 fps (DF) delivery, then that advantage is lost.  If the drop frame method of retiming is used, then absolutely nothing is gained.  The extra frames are just wasted money.  If the blending method is used, the result will not really be sharper.  Instead, contrast edges (true edges and internal edges) will have artifacts that become more notceable as movement is faster.  If the pixel-blending method is used, the resulting edge details are getting softened.

You can test this with any footage.  Pick something with some motion and conform it to 60fps.  Now drop it into a 24fps timeline or sequence, and retime it with each method to a speed of 250%.  The various artifacts will appear. Depending on your footage, it may or may not be acceptable.  But in real production, remember you don't get to reshoot if it is not acceptable, and so the point of this discussion is to provide all the information for an informed discussion.

How will Retiming OverCranked Footage Benefit a Key?
If the pixel blending method can be used without artifacts, the retiming will effectively denoise the edge where the backing color (green screen) meets the subject.  In fact, if drop frame retiming produces minimal glitchy motion, one could simply denoise at 60fps and then drop frames for a slightly better result.  But remember, the key edges are being blended in both methods, which is essentially blurred in the time dimension.  Edges may be less noisy, but the picture itself will not be crisper.

Better Keying Results With Oversized Images.
A better result will be achieved by shooting at a higher resolution.  If instead one shot at 23.976 with 4k or 5k acquisition resolution, and then pulled key, composited and then scaled master to delivery size of 2k or HD, then the resulting key will always look better and there will be zero risk of artifacts.  To keep it simple, suppose we shoot 4k and deliver 2k.  That 2:1 size ratio is a 4:1 detail ratio.  That means our key edges are four times more data and detail.  Fine hairs are now four times the area.  This means that more pixels are used in pulling the key, which gives you a crisper AND a smoother edge at the same time.  If you scale before compositing, the result will be good, but if you scale after compositing, the result will look better because the post-scale is now blending AREA from four times detail to normal detail.  The natural noise in the keyed edge, the natural rounded corners where the key edges are not sharp are 1/4 the size, so the result looks sharper.  The soft edges have 4/1 the pixels, so the blending is smoother.

Cost Implications and Other Options
Both Over-Cranking and Over-Scaling the footage when shot will increase costs.  With over-cranking, your storage and transfer times will increase 2.5x at every transfer point.  To properly watch for artifacts, your system must be able to play full-resolution, uncompressed video real time without skipping any frames.  While you can frame by frame for spatial artifacts, motion glitches often are hidden when stepping through frames due to perceptual issues.  

With Over-Scaling, the larger frames will use more stock or disk space and will consume more handling time and storage at every transfer step.  Even with compression, the rule of thumb is that 2x the size = 4x the detail = 4x the storage space = 4x the time to transfer.  It means a more expensive graphics system is needed to view images in real time, and if not real time, it is 4x the labor to view and check frames.  It will also increase compositing costs in both real labor and rendering cpu time.  

If the cost or schedule implications are an issue, another very simple option would be to reduce the in-camera noise.  Assuming a digital shoot, this should be adjustable in camera settings (outside my scope here).  If film, select negative film with less grain.  

Then of course the more even and better the backing color, the less stress you will need to put on the edges.  So while an acceptable key may be achievable, if the DP is really concerned about a good key, he needs to give you a good plate.  Review the methods of green or blue screen compositing, watch for sharp creases or shading changes in the backing color, and ask for clean plates.  After all, the keying software is just performing mathematical data processing on the acquired footage data.  The rule in data processing is always, garbage in, garbage out.  Fix it in post may be an option if the shoot costs need to be contained or time runs out, but that has its own cost, and should be defended.  

The Politics
Ultimately, how footage is shot is going to be a decision worked out by the Director and Producer, balancing aesthetics and costs.  The DP will have a great deal of influence in this speaking to aesthetics, because there are many considerations beyond just the key.  We touched on temporaral image quality.  The VFX Supervisor, Editor or Compositor needs to be the advocate for the post process.  Your job is to clear away any myths or illusions the others may have about the process and help them understand the potential aesthetic problems and cost implications of the technical method selected.  If the decision effects your budget or schedule, you need to speak up.  

It's not something to go to war over.  So if the DP insists, you just need to make sure your concerns are acknowledged (documented!) and your budget and schedule requirements are specific for the methodology.  Then let the higher-ups make the decision.  But if they leave the decision to you, then you need to weigh their aesthetic goals against the  risks and costs.

At the end of the day, good keys are not really about tricky shoots.  They'e about tried-and true best practices. They're about the skill, knowledge and careful work of the DP, crew and compositor.  Tricky shoots cannot make up for inept work. Great work can only do so much with bad data. 

May all your shots be treasures and all your frames gems.  Best success.

Wednesday, December 22, 2010

#0051 Ten Essential Rules of File Naming -. Summarized

Previous:      Asset Management 101 –  Ten Essential Rules of File Naming - 10 Include Extra Data

Asset Management 101  

Ten Rules To File By
So about now you're wondering if you want to modify your naming conventions or not.  Doing so can be a big undertaking, involving sometimes getting the consensus of other supervisors and producers.  It should certainly be initiated only for new projects, and you'll need to prepare a good document explaining what the changes are, why they are important and possibly how the system will be implemented.

When implementing the system, you'll need to consider how you will train your staff.  A memo is fine, but better is a briefing paper or possibly a slide show illustrating how the new system works.  You may find it necessary to explain to the staff why the changes are important and how it will make things more secure and improve everyone's understanding of file status and relationsips and revision level.   You need to explain the new system because people tend to resist and question change; explaining it to people is a show of respect.

Mostly, you'll need to get your intermediate supervisors and coordinators fully committed to the changes and get them to understand it is part of their job description to knoow and enforce naming conventions.

A final thing to consider is automation.  For one company, I made a spreadsheet in Open Office.  Artists in each work unit can fill in the information for their shot and get back a properly formatted name for their scripts and renders.  Similar utilities can be developed in MEL, Python or Javascript for users.  In NUKE a gizmo could be put together.  Remember, artists are busy and will ignore or make mistakes in file naming just because they are rushing or tired.  Make it easy for them.  

Here's an outline of this series

10 Rules To File By

1.Let it Speak (Make it meaningful)
2.Make it Short
3.Keep it Simple
4.Avoid Special Characters and Control Punctuation
Use command-line compatible punctuation
Avoid Spaces
Two periods maximum
The hypen – if you dare
Use postScript Notation
Special Emphasis
5.Protect Sort Order
Maintain Chronology
Variations and Passes after take or version
6.Rendered Files Must Refer to Their Script
7.There can be only ONE revision 1
Call Composited Shots "takes"
Put Modifiers After Revision Number
Tie Render Passes To A Revision
Use Takes and Versions Together
Make sure your revision modifier policy is understood
Use Codes and Punctuation Sparingly
8.Abbreviate Consistently
9.Use Project Identifiers
Separate internal or external work
Client company and sometimes division or department
Project: film, show, series, campaign,
Sub-project: TV episode, spot or reel number.
10.Include extra data
Artist Names
Variation vs Option Descriptions
Rights Managed Files
Camera Angle

Tuesday, December 21, 2010

#0050 Ten Essential Rules of File Naming 10-. Include Extra Data

Previous:      Asset Management 101 –  Ten Essential Rules of File Naming - 8 Abbreviate Consistently

Asset Management 101

Include Extra Data
In the post, #0043 AM101: Part 2 - Understanding File Classification , I list more than 15 ways to classify files. In your naming system, incorporating this meta-data into a file name can often be useful, avoiding the need for secondary data files with data about the data. Let's look again at some of the extra data you might include:

Artist Names

This should go without saying, but sometimes it is important or helpful to embed the artist names or initials in a filename. Case in point: I currently am supervising in a mixed-OS environment, with Linux, Mac and Windows machines. The server is Linux, and my Windows machine doesn't convert for me the user and group ID numbers into names. For most files, we don't care enough to make this an issue. But for some files, with shared authoring, it is easier if we know who authored the most recent revision. Likewise, artists may be working in a common folder generating multiple assets.  In both these examples, knowing who made the file can save time.

Variation vs Option Descriptions

Often we are producing more than one option. For all files, this also belongs after the revision control number, unless the options and variants represent unique elements that can or should be used.
To clarify, suppose we are making a single element in Photoshop of a gaseous cloud for compositing. We've designed three Layer Comps and named them, “red”, “blue” and “mixed”. If only one of these should be used we would name these:
example 1
But suppose we are asked to provide three unique gas cloud variations for a shot. These might be named:
example 2
This implies, of course, a requirement for explicit and clear instructions to the artist. However, giving explicit instructions will strengthen your command and control system. When in doubt, the artist should ask if they are presenting options or variations.

Other File Classification Data

Rights Managed Files – One company I worked for found it important, when collecting stock files, to include rights usage information in the filename. They developed some simple abbreviations to indicate Royalty Free and Rights Managed assets. When collecting or accepting assets from the client for reference or use, it might be useful to encode the file with information to indicate whether the file is only for reference, client use in a particular project, or general use for all client projects.

Camera Data – Sometimes it might be useful to file footage with information about the camera angle (wide, close-up) or a camera number. Suppose you have stereo footage, then you need a consistent way to name your left and right view footage and asset files. Likewise, suppose you're assembling work from a multi-camera shoot, you may want to identify which camera was used in the source footage filename.

Use your imagination, and remember, that the most important aspect of asset filenaming is to facilitate control, yet accommodate unpredictable needs. Allow in your convention for modifiers and additional data, don't try to predetermine every possible situation, and update your specifications and review them with your staff often.
Just remember to protect your sort order, Usually the most important aspect of sort order will be the asset name and the next most important will be the revision control system numbering. After that come your modifiers and other data.

Asset Management 101 – part 9: Ten Essential Rules of File Naming-Summarized