 |
| "Green Screen" by Peter Pearson licensed CC BY-SA 2.0 |
Case in point, I read a LinkedIn post today from an editor-animator-camera operator who has a DP wanting to over-crank a camera to get a better chroma key. More specifics: there is synchronized audio, proposed shoot speed is 59.97, mastering speed is 23.976. DP believes he will get a crisper, cleaner key.
First, let's tackle the technical question and help this editor with the right information, then we will talk about how to handle the politics of the situation where your voice may not be the loudest in the room.
Over-Cranking the Camera and Conforming Footage
To answer this question, let's consider the over-cranking speed adjustment required for mastering first and then discuss the keying. Again the specs- synchronized audio, 50fps conformed to 24 fps Drop Frame. In this situation over-cranking may not help and will most likely will introduce motion discontinuities, also known as a glitch or bump or jump in motion.
The questioner says the DP wants to "conform" the footage to 23.976. Conforming footage in editing or compositing parlance means to LIE to the software. If the footage is shot at 59.97, the native or true speed is 59.97, for simplicity we will say 60 frames per second. Conforming means to tell the software after importing the clip that the speed is not 60 fps, it is say, 24 fps. This means that it will play 60 frames of clip in 60 frames at a speed of 24fps. If it were not conformed, it would play 60 frames of clip in 24 frames of master at a speed of 24fps. In this case, conforming one second of source footage results in 2.5 seconds of master footage. This will then cause the footage video to be out of sync (behind) the footage audio. Usually a problem. So therefore the footage cannot be conformed, it must be retimed.
Retiming Footage Methods and Their Artifacts
In doing so there are three ways - dropping frames, blending frames and blending pixels. In this case every 2.5 frames needs to be merged to produce one frame. If dropping frames, it means 5 frames become 2, so a jump or glitch will occur every 3 frames. Depending on motion, this will usually be noticeable. If frames are blended, it means 2.5 frames will be dissolved together to make a new frame, which means details will be repeated and ghostly artifacts will most likely appear, especially around edges of high contrast. If pixels are blended, it means a new frame will be attempted based on contextual motion of the pixels. This can result in swimming pixels. Because the footage is being sped up, this is less likely than if it were being slowed, but it still may happen where details are at a high frequency, like in hair or fabrics or other textures. Anytime footage is retimed there is an opportunity for artifacts to creep in. The vfx artist or editor may have some control over quality, but in the end, the footage and software will rule the result, not the artist, editor or DP.
Now the question is, will the footage look better if over-cranked. It depends on your primary delivery market. If your market is broadcast TV, then shooting 60fps may be desirable if you also master at 60fps, because then you can have smoother, cleaner field rate motion. But today in HD many shows want to see frame motion and like the 24fps look as more filmic. As the given specs are a 24 fps (DF) delivery, then that advantage is lost. If the drop frame method of retiming is used, then absolutely nothing is gained. The extra frames are just wasted money. If the blending method is used, the result will not really be sharper. Instead, contrast edges (true edges and internal edges) will have artifacts that become more notceable as movement is faster. If the pixel-blending method is used, the resulting edge details are getting softened.
You can test this with any footage. Pick something with some motion and conform it to 60fps. Now drop it into a 24fps timeline or sequence, and retime it with each method to a speed of 250%. The various artifacts will appear. Depending on your footage, it may or may not be acceptable. But in real production, remember you don't get to reshoot if it is not acceptable, and so the point of this discussion is to provide all the information for an informed discussion.
How will Retiming OverCranked Footage Benefit a Key?
If the pixel blending method can be used without artifacts, the retiming will effectively denoise the edge where the backing color (green screen) meets the subject. In fact, if drop frame retiming produces minimal glitchy motion, one could simply denoise at 60fps and then drop frames for a slightly better result. But remember, the key edges are being blended in both methods, which is essentially blurred in the time dimension. Edges may be less noisy, but the picture itself will not be crisper.
Better Keying Results With Oversized Images.
A better result will be achieved by shooting at a higher resolution. If instead one shot at 23.976 with 4k or 5k acquisition resolution, and then pulled key, composited and then scaled master to delivery size of 2k or HD, then the resulting key will always look better and there will be zero risk of artifacts. To keep it simple, suppose we shoot 4k and deliver 2k. That 2:1 size ratio is a 4:1 detail ratio. That means our key edges are four times more data and detail. Fine hairs are now four times the area. This means that more pixels are used in pulling the key, which gives you a crisper AND a smoother edge at the same time. If you scale before compositing, the result will be good, but if you scale after compositing, the result will look better because the post-scale is now blending AREA from four times detail to normal detail. The natural noise in the keyed edge, the natural rounded corners where the key edges are not sharp are 1/4 the size, so the result looks sharper. The soft edges have 4/1 the pixels, so the blending is smoother.
Cost Implications and Other Options
Both Over-Cranking and Over-Scaling the footage when shot will increase costs. With over-cranking, your storage and transfer times will increase 2.5x at every transfer point. To properly watch for artifacts, your system must be able to play full-resolution, uncompressed video real time without skipping any frames. While you can frame by frame for spatial artifacts, motion glitches often are hidden when stepping through frames due to perceptual issues.
With Over-Scaling, the larger frames will use more stock or disk space and will consume more handling time and storage at every transfer step. Even with compression, the rule of thumb is that 2x the size = 4x the detail = 4x the storage space = 4x the time to transfer. It means a more expensive graphics system is needed to view images in real time, and if not real time, it is 4x the labor to view and check frames. It will also increase compositing costs in both real labor and rendering cpu time.
If the cost or schedule implications are an issue, another very simple option would be to reduce the in-camera noise. Assuming a digital shoot, this should be adjustable in camera settings (outside my scope here). If film, select negative film with less grain.
Then of course the more even and better the backing color, the less stress you will need to put on the edges. So while an acceptable key may be achievable, if the DP is really concerned about a good key, he needs to give you a good plate. Review the methods of green or blue screen compositing, watch for sharp creases or shading changes in the backing color, and ask for clean plates. After all, the keying software is just performing mathematical data processing on the acquired footage data. The rule in data processing is always, garbage in, garbage out. Fix it in post may be an option if the shoot costs need to be contained or time runs out, but that has its own cost, and should be defended.
The Politics
Ultimately, how footage is shot is going to be a decision worked out by the Director and Producer, balancing aesthetics and costs. The DP will have a great deal of influence in this speaking to aesthetics, because there are many considerations beyond just the key. We touched on temporaral image quality. The VFX Supervisor, Editor or Compositor needs to be the advocate for the post process. Your job is to clear away any myths or illusions the others may have about the process and help them understand the potential aesthetic problems and cost implications of the technical method selected. If the decision effects your budget or schedule, you need to speak up.
It's not something to go to war over. So if the DP insists, you just need to make sure your concerns are acknowledged (documented!) and your budget and schedule requirements are specific for the methodology. Then let the higher-ups make the decision. But if they leave the decision to you, then you need to weigh their aesthetic goals against the risks and costs.
At the end of the day, good keys are not really about tricky shoots. They'e about tried-and true best practices. They're about the skill, knowledge and careful work of the DP, crew and compositor. Tricky shoots cannot make up for inept work. Great work can only do so much with bad data.
May all your shots be treasures and all your frames gems. Best success.
Posts
