#0019 The Pipeline -Form Follows Function

Image via Wikipedia

Pipeline Design 101

Anyone who has taken any sort of introductory design class, whether in art, graphics, architecture or what have you, knows that "form follows function". When designing a CG pipeline one needs to consider the function of the pipeline before designing the structure let alone the mechanics.

In our last post, "#0018 Understanding Pipelines" we discussed that the term pipeline is broadly used for many different things, from software and hardware component design to include various types of CG work pipelines. In an epiphany of sorts, I came to realize that a CG pipeline can fall in any of three classes:

1. the production pipeline: the labor processes and steps
2. the material pipeline: the data assets acquired, produced and delivered
3. the approval pipeline: the direction and redirection mechanism and loop

Another way of looking at this is process, data and metadata; to get totally geek and introduce some new jargon, we could speak of the primary, secondary and tertiary pipelines.

Case Study 1:

Small Shop, Big Job, Varied Jobs

I recall on

e day after working with a fellow for six months he stood up and declared to me, "we need a pipeline!"

At the time we were producing about 35 shots a week with a crew of about seven 3d artists and five compositors. Our production pipeline was deliberately simple. While we were all working on the same television series with a fairly defined theme, the content of each episode could vary widely, and hence the nature of demands on our small 3d team was constantly shifting.

Primary Pipeline: Production

I therefore preferred to staff with a number of generalists at different skill levels, but I included one generalist who was a strong modeler and one who was strong in dynamics. A 3d supervisor provided muscle to round out any gaps in skill sets, and I would step in to demonstrate "advanced" 3d techniques in anything from models to animation to shaders.

From our 3d unit the work would move to our compositing unit, where again we had little specialization, other than skill levels. Again the unit was lead by a highly experienced and capable comp supervisor. The use on our team of compositors at a wide range of skill levels, with a couple of old salts who knew green screen and shot aesthetics well, rounded out by a young motion graphic designer, allowed us to assign shots based on difficulty, design needs and so on. And again I could step in to advise or teach or do.

So, we had a simple production pipeline with a form suited to it's function: produce a large volume of material on a weekly basis with highly variant visual effects demands.

Secondary Pipeline: Data

Our coordinator managed the materials pipeline, which at our level of operation involved checking in assets from editorial and production and delivering product to editorial. I set-up a system using GOOGLE Docs spreadsheets to log assets needed, assets received and shots delivered and take status notes.

Between 3d and 2d we'd established a materials pipeline that wasn't very sophisticated: artists would log a shot ready for compositing and specify a database location code and compositors would pick up the work and mark it with a code indicating it had been picked up. One artist was assigned to A simple effective. And again this was communicated using the same GOOGLE docs database.

In addition, our "generalist-modeler" was tasked to collect and manage a database of re-usable models. At the same time, we tasked a couple of compositors to maintain our library of 2d assets.

Again I was intimately involved with this (and later we hired an I.T. Coordinator to help). So our data pipeline was in place.

Tertiary Pipeline: Direction and Approvals (or the data about the job)

Direction and approvals went through me. I set-up the GOOGLE docs system for tracking every show, and in this I logged all the shots, gave them a number and included the director's notes and script snippets. This was set-up so that a week or two ahead of need our coordinator would make a new show document and inform the director. After the first time working with us, about 90% of the directors were able to fill out this form before my first meeting, and would modify it during pre-production.

I would add my shot direction and my 3d and comp supervisors would add theirs. As shots were submitted, we used a tight naming system to code the internal approval level into the shot name. This system was designed so that the take delivered always referred back to a specific composition in a specific project file, and each 3d element referred back to the originating Maya file, to allow rapid identification of files for revision. Enforcing this naming convention was a continual effort of education and cajolery on my part assisted by the coordinators and other supervisors.

Change notes were communicated in the spreadsheet as well. While not foolproof, we made an effort to get all notes written down and communicated them orally as well. The pace and volume of work made formal dailies impossible, but on a daily basis shots were reviewed by the 3d supervisor, comp supervisor and myself. We encouraged directors to come in and would call and ask them to come in, but were generally too busy to chase them. Happily, almost no shot that made it past the other supervisors' and my eyes was ever bounced. So we had an approval pipeline that functioned.

Case Study 2:

Big shop, Long Job, Volume Business

Another shop I worked for was very different. I want to get on to other things, so here is a summary:

Production Pipeline: Specialists in roto, wire removal, cleanup and tracking; specialists in modeling, animation, characters and dynamics; specialists in scripting and special programming needs; specialists in data management and film outs; specialists in green screen extraction, 2d tracking and generalist compositors and 3d animators. I forgot to mention specialists in storyboarding, design, character modeling, character rigging, texture UV layout, texture painting, researchers, lighting and shaders. Plus coordinators, CG supervisor, Roto supervisor, etc. About 25-30 job descriptions and about a staff of 150 working on two or three films.

This shop was geared up to handle about a dozen or two shot sequences, comprising anywhere from eight to 40 shots, with similar shot needs in each sequence. Production turn-around for a film with 300 or so shots ordered was about eight months.

Data Pipeline: tell a coordinator where things are

Approval Pipeline: dailies

The Two Pipelines

We need a pipeline!

Image via Wikipedia

Getting back to the story: so he says to me, "We need a pipeline!" Of course I dismissed the suggestion, which wasn't the best way to handle it, but that's a story for another day. The point is, we already had a fully functional pipeline.

My shop #1 is engineered for a low budget operation with flexibility in mind. Another small shop that specializes in a particular kind of work might find a different pipeline more to their advantage.

For example, a shop producing over a four month period 500 or so shots with 3d vfx of WWII aircraft and warships in the Pacific theater will need and be able to get advantages out of a more specialized pipeline than we employed. The mission of shop #1 is to support a science variety show, and the range of topics is immense. One week they are animating black holes, another constellations and another depicting exploration of Mars. They have hypothetical animals one week, and volcanoes the next. The work needs were unpredictable; the shop making airplanes had predictable needs.

Likewise, shop #2, making feature films, was able to achieve specialization because the volume of work included a significant amount of essentilly repetitious shots. At the same time, the volume of work allowed them to have at least one and ususally several persons expert in almost any specialization. And, at the same time, the length of projects and the overlap of several films, allowed them to maintain a pool of talent with a large variety of skills and experience, providing indirectly generalization and directly labor economy. Further, the shop was big enough to have more coordinators than I could count, so the details of their material and approval pipelines were hidden from artists. (Today, more reliance is made on technology to coordinate these pipelines.)

Form follows Function

Your job, as a CG Supervisor, is to understand the function first and design a pipeline to fit: remember form follows function. The idea that some people have that a pipeline comprised of specialists is not the best solution in every situation. Just as the giraffe has a long neck to reach the highest leaves, your CG pipeline must follow the form that gives you the most efficient and effective solution to the work you need to do. In some situations, the best solution is a team with diverse capabilities and a leader able to create ad hoc pipelines using those talents to get special problems solved. In other situations, an assembly line of specialists will be more effective.

One size does not fit all.

---Happy Pixels!