Mar 212016
 

I’m trying to build a system right now that can automatically substitute environment variables for paths on a per-scene basis. When a user opens a Maya file, I want to parse that file and set temporary environment variables called $SHOW, $SEQ and $SHOT that point to the show’s root folder, the sequence’s folder in that show, or the shot in that sequence. The variable names and paths I’m trying to get are pretty specific to the current pipeline I’m working on, but this essentially lets me use relative paths in Maya but WITHOUT using Maya’s outdated and inflexible workspace system. I don’t want an individual Maya project for every single shot and asset in the entire show, and I don’t want all of my Maya scenes to be forced to live within a single workspace, either.

I’ve solved this problem before by using symbolic links to basically force all of the Maya projects generated for every single shot and asset to link to the same place. This makes for a pretty busy-looking file system, though, and symlinks only work on Unix-based servers (i.e. not Windows). This system I’m building now looks a lot more like Houdini’s global variables… as long as I define $SHOW or $SHOT or whatever, I can path a reference or a texture to $SHOT/assets/whatever.xyz and it doesn’t necessarily have to be within a Maya workspace, and it’s also not an absolute path.

Read more below… Continue reading »

Mar 112016
 

A while ago I posted about how to manage viewports in Maya in order to prevent the screen from redrawing while doing a bake or Alembic export, which slows things down considerably. It turns out there’s a way easier way to handle this, thanks to a hidden (of course) MEL variable that Maya docs don’t seem to mention anywhere.

The command is:

paneLayout -e -manage false $gMainPane

The variable $gMainPane is a shortcut that just refers to Maya’s main viewport window. You can set the “managed” state of the entire pane to False, which simply disables refreshing it until you set it back to True. Just one line of code!

Here’s another, even easier method, that’s actually part of the Python cmds module:

cmds.refresh(suspend=True)

There’s a catch, though. From the docs on the refresh() command:

Suspends or resumes Maya’s handling of refresh events. Specify “on” to suspend refreshing, and “off” to resume refreshing. Note that resuming refresh does not itself cause a refresh — the next natural refresh event in Maya after “refresh -suspend off” is issued will cause the refresh to occur. Use this flag with caution: although it provides opportunities to enhance performance, much of Maya’s dependency graph evaluation in interactive mode is refresh driven, thus use of this flag may lead to slight solve differences when you have a complex dependency graph with interrelations.

So make sure you test each method with different kinds of bakes before you commit to any one solution.

Jun 242015
 

A quick script here to convert Gizmos in Nuke into Groups. Gizmos are nice in theory in that they provide reusable groups of nodes for users to interact with, but they’re often more trouble than they’re worth, especially if you need to send your Nuke scripts to someone else. Here’s a script that just converts all the Gizmos in your scene into groups seamlessly:

import nuke

def isGizmo(node):
    return 'gizmo_file' in node.knobs()
    
def getGizmos():
    allNodes = nuke.allNodes()
    if allNodes:
        gizmos = [f for f in allNodes if isGizmo(f)]
        if gizmos:
            return gizmos
    return False

def deselectAll():
    # deselect all nodes.
    for i in nuke.allNodes():
        i.knob('selected').setValue(False)
        
def convertGizmoToGroup(gizmo):
    # convert a gizmo to an identical group.
    # delete the original, reconnect the group in the chain.
    # rename the group to match the original.
    inputs = []
    for x in range(0, gizmo.maximumInputs()):
        if gizmo.input(x):
            # print 'input: %s' % (gizmo.input(x).knob('name').value())
            inputs.append(gizmo.input(x))
        else:
            inputs.append(False)
    origName = gizmo.knob('name').value()
    origPosX = gizmo.xpos()
    origPosY = gizmo.ypos()
    deselectAll()
    # select knob, then run gizmo.makeGroup()
    gizmo.knob('selected').setValue(True)
    newGroup = gizmo.makeGroup()
    deselectAll()
    # delete original
    nuke.delete(gizmo)
    newGroup.knob('name').setValue(origName)
    newGroup['xpos'].setValue(origPosX)
    newGroup['ypos'].setValue(origPosY)
    # disconnect old inputs
    # reconnect inputs
    for x in range(0, newGroup.maximumInputs()):
        newGroup.setInput(x,None) 
        if inputs[x]:
            newGroup.connectInput(x, inputs[x])
        # print 'connecting output: %s to input: %s' % (inputs[x].knob('name').value(), newGroup.input(x).name())
    return newGroup
    
def convertGizmosToGroups():
    gizmos = getGizmos()
    if gizmos:
        for i in gizmos:
            convertGizmoToGroup(i)

There’s four methods here: first, a method to determine whether or not a node is a gizmo (by looking for a “gizmo_file” knob). Second, a method to find all gizmos by running isGizmo on every node in nuke.allNodes(). Third, the big function that does most of the heavy lifting. There’s a built-in API call that can run on gizmos called makeGroup() but by itself it’s not incredibly useful; it just wedges a converted copy of your Gizmo into the flow without doing anything to the original node, and it doesn’t name the group or anything. So this script is just adding onto that API call.

The first few lines of convertGizmoToGroup are creating an array of inputs so that we know what nodes are connected to the inputs of the gizmo. We’re just grabbing references to each input and stuffing it into the array. Next we record the name and position in the node graph of the original gizmo. We deselect everything and then set the “selected” knob of the gizmo to “True” (why we don’t have a real selection API call in Nuke, I’ll never know). Then we run the magic gizmo.makeGroup() command, which converts the gizmo into a group, but potentially leaves inputs disconnected, doesn’t delete the original gizmo, and doesn’t rename the new group. We do that in the next few lines; first deleting the original, then renaming our newGroup to match the original name, then setting the node graph position of the new group to match the original’s so it doesn’t mess up your organization. Finally, we loop through our array of inputs and connect them (in order) to the new group node. Pretty simple!

The last function is just a loop that converts every gizmo in the scene. You could import this module using menu.py and turn it into a button on a toolbar to quickly convert the whole scene from gizmos to groups by calling convertGizmosToGroups.

Hope it’s useful!

May 182015
 

I’ve implemented most of my planned pipeline, finally. Just want to do a brief walkthrough to brag about features to show what this set of tools is capable of doing.

The pipeline uses symlinks to allow each asset or shot to exist inside its own self-contained Maya project workspace, while linking together key directories in those workspaces to all point towards the same repository. This way, textures can all be found in a single location, cache data can be shared between departments (Animation, Lighting, etc), and renders can all be easily found in a single place, without confusing Maya by rendering to paths outside of the workspace.

All project-level commands can either be run from a Linux terminal or from the web interface mentioned in the previous post. New project structures are generated using a master XML file that defines folder names/hierarchies and permissions bits for each folder, to prevent regular users from writing files where they shouldn’t.

The File Manager handles assets and shots. Assets can be “tagged” in order to categorize them in the menu on the left. These tags are arbitrary, can be changed at any time, and don’t affect the file structure at all, so the way assets are organized can evolve fluidly without screwing up references. Asset versions are organized using a public/private system, so users do whatever they need to do inside their own work folders, then “publish” the asset, which automatically is assigned a name and version number based on the department it’s being published to (such as Modeling or Rigging). Artists using these assets can automatically receive updates to their references without dealing with a “master” asset constantly being overwritten and screwing up in-progress renders or simulations. Each version has notes associated with it, and for published versions an automatic note tracing the file back to the work file it was originally saved from.

assetManager

The Assets tab of the File Manager.

The Animation I/O tool handles the import/export of animation data, including cameras. Animation can be exported either as baked curve data (Maya ASCII), or as Alembic caches. The hybrid approach is to maintain a light server presence and flexibility, since blindly caching Alembic data for everything can seriously bog down a server, not to mention the time it takes to actually write an Alembic cache in Maya. Animation is written out to shared cache folders named after the shot they were exported from, and animation exported this way can be automatically updated using the import tools. Lighters don’t even have to know what assets are supposed to be in their scenes; the Match Exported Scene button will automatically pull in references as needed when importing data (even when the data is not Alembic). A single click will make sure that a lighter’s scene exactly matches the animator’s scene (minus all the junk that animators like to leave around).

The Look manager tool handles shading and render layers. A user can shade, apply render layers, per-layer overrides, displacement sets, etc., and then save those customizations to a Look file. This Look can then be applied to an identical (or nearly-identical) asset anywhere else in the project. Materials are saved to a separate file that is then referenced into other scenes by the Look Manager. If you have multiple objects sharing the same Look, they will shared the shader reference, helping the Hypershade stay as clean as possible. Shading and layer assignments are handled by object name, so if your receiving asset is named slightly differently (Asset_A vs. Asset_B) you can use a find/replace string to substitute object names. Shading in this pipeline is completely independent of lighting and can be adjusted and modified (aside from UV adjustments) anywhere, then saved as a Look for use by others. When Looks are removed or replaced, the Look Manager automatically removes unused render layers and displacement sets, as well as referenced materials files that are no longer in use.

The Look Manager interface.

The Look Manager interface.

The Preflight system helps keep assets and scenes conforming to pipeline structure. A series of checks is run, with presets for checks available based on department (Modeling, Rigging, Lighting, etc). Each check will be highlighted if there is a problem, which can then be fixed with an automated function. Each check maintains its own viewable log, so users can see exactly what the problems are and what is fixed if the function is run. The checks are dynamically loaded from a folder, and each check is based on a template. A pipeline operator or power user could add their own checks to this folder as long as they fit the template, and they will be automatically loaded into Preflight at next launch.

Renders, once finished, can be quickly loaded into an Auto-Comp generated in Nuke or NukeAssist. The ReadGroup script creates a special group that will take all single-channel passes in a folder and combine the channels together into a single stream. The entire group of Read nodes can be updated simultaneously by picking a new folder on the ReadGroup, rather than having to manually replace each individual channel. Another button on the ReadGroup automatically replaces each main sequence type (Beauty, Tech, and Deep pass groups) with the latest available version. Another click quickly generates a composite out of the channels within, adding or multiplying channels depending on the channel names (this function assumes a VRay pipeline but could be expanded to work with Arnold or mental ray) and creating a fast comp along with a contact sheet to check passes quickly. Lighters can use this tool to make sure their passes rendered correctly, and then use the “Approve Renders” button to automatically move footage from the “incoming” renders directory to the “approved” renders directory, where compositors can use them. Compositors can keep groups of renders linked together easily for version control, and use the auto-comp as a starting point for breaking out channels in their comps.

A generated Auto-Comp, with the interface panel on the right.

A generated Auto-Comp, with the interface panel on the right.

The pipeline is built to be as modular as possible, so each piece doesn’t necessarily need to know what the other pieces are doing in order to function. A few core scripts automatically set the user’s workspace when files are opened or saved, and a couple optionVars maintain the current user and project folder. Everything else can be derived from the workspace, so the tools can be used piecemeal in a pipeline (for the most part) if necessary. Most configuration is done in a single settings.py file, which can be edited to configure the pipeline for just about any server setup. The goal was to make a pipeline that could be entirely feature-complete if necessary, or to have the tools operate individually as a layer on top of an existing pipeline.

Sorry about the big long bragpost, just wanted to have a place to document what I’ve been spending all these months working on since it’s hard to put something like this in a demo reel!

Feb 112013
 

Many operations in Maya will run faster if Maya doesn’t have to refresh the viewport while running them. For example, if you switch the viewport to only show the Graph Editor before baking animation, or caching particles or Alembic geometry, the operation will happen much faster than if Maya had to actually display the geometry for each frame that it’s being baked. There is probably a way via the API to tell Maya’s viewport not to refresh, but since I don’t know shit about the API, here’s a workaround using a few of Maya’s less-documented MEL commands and some Python.

I wrote this method assuming that artists would want to cache out information frequently without it disrupting their workflow. That means that I needed to first store the user’s current panel layout, then switch it to something that doesn’t require a refresh on every frame (like the Graph Editor), and then restore the previously restored layout.

I prefer coding in Python, but some of the procedures I’m running are MEL functions that are found in scripts/startup and so they’re not documented and they don’t have Python equivalents. A hybrid approach is the best way to handle it, since MEL is terrible.

Click for some Python code… Continue reading »

Jun 202012
 

I just ran into a hideous problem when I was trying to bake out cameras and animation from one scene, and get the animation into another. I always keep my lighting scenes separate from the animators’ scenes so they can’t mess up my shots. Unfortunately, the animator had used a Scene Time Warp, and then baked out the animation, which of course never quite lined up correctly and we couldn’t figure out what was going on. Turns out that keys baked under a Time Warp will not actually bake the Time Warp itself.

Here’s a script, then, to bake the effects of the time warp on all selected objects. It then (optional, but recommended) deletes the time warp from the scene entirely:

Continue reading »

Feb 222012
 

I’ve been working on some tools recently that open groups of Maya scenes in batch mode, running Python or MEL operations on each file (for example, to quickly add a render layer and apply materials to a large group of objects at once, or to get the names of all the references in a scene) and then either returning data or saving the file. This sort of thing could be done in the usual windowed Maya interface, but this is slow and means the user has to sit there and watch Maya open a bunch of files, run operations and all that. Running scripts on lots of files at once is much faster if you’re running Maya in standalone (“batch”) mode.

There are other, more informative posts on how to get Python running in standalone mode (I learned a lot from this one in particular), but the gist of it is that you start by running /bin/mayapy.exe, and then initialize the Maya Python interpreter by calling the following function:

import maya.standalone as standalone
standalone.initialize(name='python')

Once that’s done you can start scripting like normal, starting with import maya.cmds as cmds and working from there. Some commands are not available in standalone mode, especially anything to do with UI.

Anyways, while working on a script that is meant to process huge heaps of unsorted Maya files, I realized that crashes were going to be a frequent problem when opening ancient, broken scenes, so I couldn’t just load an instance of Maya in standalone mode from a script and let it run through everything in a loop. I needed to be able to process one file at a time, and then either return some useful information if the process is successful, or let me know if Maya crashes so I can avoid that file later on or at least mark it.

To do this, I used Python’s subprocess module. Again, I’m not going to go into a ton of detail about the module, it’s pretty complicated, but I’ll give a quick example of how I’m using it to return information from a Maya process when I am calling the script from Maya (or any other program, really).

Let’s say I want this script to add a new render layer to several scenes, and add all meshes in each scene to that new layer. First, I’ll write the script that actually creates the layer and assigns objects.

import sys
import maya.standalone as std
std.initialize(name='python')
import maya.cmds as cmds
filename = sys.argv[1]
layername = sys.argv[2]

The sys.argv[1] and sys.argv[2] are easy ways to pass variables to an external script. sys.argv[n] just means the nth command line argument, so sys.argv[1] is the first command-line argument when running the script (after calling for the script name, for example, by running mayapy.exe yourScript.py). When we call this script, we’ll pass it a file to open, and a layer name to add.

def addRenderLayer(filename,layername):
    try:
        cmds.file(filename,o=1,f=1)
        newLyr = cmds.createRenderLayer(n=layername,empty=1,makeCurrent=1)
        meshes = cmds.ls(type='mesh')
        xforms = []
        for i in meshes:
            xf = cmds.listRelatives(i,p=1)[0]
            xforms.append(xf)
            cmds.editRenderLayerMembers(layername,xforms)
        cmds.file(s=1,f=1)
        sys.stdout.write(newLyr)
        return newLyr
    except Exception, e:
        sys.stderr.write(str(e))
        sys.exit(-1)

addRenderLayer(filename,layername)

Here we’re creating an empty render layer based on the “layername” argument, then we’re getting all the meshes in the scene. In most scenes, people add transforms to render layers, not their individual shape nodes, so we’ll do this by creating an empty list “xforms,” then going through each mesh in a loop and finding their first related transform node (hence the listTransforms(i,p=1)[0] since listTransforms returns a list) and appending it to “xforms,” then adding “xforms” to the new render layer. Then we just save the file and return the name of the layer (I just like having return values for my functions).

We also write the name of the new layer to stdout. I’ll go into more detail about this later, but it lets us send information back to the program that calls this script later on. There’s also an exception handler that will write any error encountered to the stderr stream, and will exit with code -1, which is sort of a convention for “this program exited abnormally.”

Save this file as makeNewLayer.py somewhere in your default scripts directory.

Next, we’re going to make the process that we run from Maya…

import maya.cmds as cmds
import subprocess
# replace mayaPath with the path on your system to mayapy.exe
mayaPath = 'c:/program files/autodesk/maya2011/bin/mayapy.exe'
# replace scriptPath with the path to the script you just saved
scriptPath = 'c:/users/henry/desktop/addRenderLayer.py'
def massAddRenderLayer(filenames,layername):
    for filename in filenames:
        maya = subprocess.Popen(mayaPath+' '+scriptPath+' '+filename+' '+layername,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
        out,err = maya.communicate()
        exitcode = maya.returncode
        if str(exitcode) != '0':
            print(err)
            print 'error opening file: %s' % (filename)
        else:
            print 'added new layer %s to %s' % (out,filename)

Okay, a lot going on here. The subprocess.Popen function is calling mayapy.exe with three parameters: first, the script we want to run (makeNewLayer.py) followed by the two arguments that script wants (the filename and the layer name), then the input stream and the output stream which are both set to subprocess.PIPE. I’m not quite knowledgeable enough about the subprocess module to go into specifics, but basically when you run a process there are three “streams”: the input, output and error streams, usually called stdin,stdout and stderr. When you set the input and output streams to subprocess.PIPE when calling Popen(), you can be sure that the process will receive any input stream you give it (which we are not doing here; instead we’re using the sys.argv method shown above) and also communicate back any output it gives.
The next procedure, communicate(), returns two variables: stdout and stderr. We can use this for debugging or monitoring progress. Normally in the Maya script editor, if your script fails for whatever reason, you will get some kind of (possibly) useful information back from an error. However, you won’t have any idea what goes wrong with a subprocess unless you are listening to the error stream.
communicate() does a couple other awesome things. First, and most importantly, it waits for the subprocess to complete before continuing with your script. This is REALLY IMPORTANT, as you definitely don’t want a script like this to continue running until the process is finished. Second, it sets a few variables on the Popen() object, one of which is Popen.returncode. If you have ever slogged through Maya render logs and noticed lines like “Maya exited with status: 210” or “Maya exited with status: 0,” those numbers are exit codes. Status 0 means the program exited normally; any other exit code means something went wrong. Since I want to be notified if something goes wrong when opening or editing a scene, I check to see if Popen.returncode is anything other than zero, and if it is, I print out the error stream and leave a notice that says something went wrong. If everything worked, I just want to know that it worked, and so I print a string. Since I wrote the output layer to sys.stdout, when I read stdout from the subprocess, I get the name of the newly created layer. I do this since the layer might not come out with exactly the name I wanted if another node in the scene had the same name as the new layer!

Now, to actually run the script. If you were to run massAddRenderLayer() from Maya, as in the example, you’d call it like this:

# define a list of filenames to iterate through
filenames = ['file1','file2','file3']
renderLayerToAdd = 'someNewLayer'

# run procedure, assuming you've already defined it
massAddRenderLayer(filenames, renderLayerToAdd)

I’ve been using the subprocess module along with Maya standalone more and more as I make more complex applications. It’s incredibly useful to be able to process many files at a time… if you get any good ideas, you know where to post them.

Aug 102011
 

I’ve been working on a project that incorporates a lot of models that were made by duplicating or importing things into a scene over and over again. When this happens, you have lots of nodes that will have the same name, and Maya will differentiate between them by tacking the name of the parent onto the object using “|” as a separator (pCube1 becomes group1|pCube1, etc). For typical use, this isn’t a huge problem, but seeing as hardly any project I’ve worked on has been “typical” Maya use, duplicate object names end up breaking things more often than they don’t. This is especially apparent if you use a lot of custom scripts, which I do all the time.

I wrote a quick script to address this so that objects are always uniquely named. It works by analyzing all the transform nodes in your scene and looking for any names that contain ‘|’ which signifies that Maya found another object with the same name and is trying to differentiate them. After that, it renames any object it finds by tacking on a ‘_###’ string to the end, looking for the lowest possible number to use for ### in a while loop.

The biggest trick here was to make sure that I didn’t rename parent objects before I renamed their children. If I renamed ‘group1’ before I renamed ‘group1|pCube1’ then the latter object wouldn’t exist anymore by the time I got to it. To prevent this from happening, I built a dictionary where the key is the object’s full name, and the value is the number of times ‘|’ appears in the name. Then I sorted the dictionary by value (not by key) in reverse order, so that objects lowest in the hierarchy (the ones with the most ‘|’ characters) are renamed first.

The Python for that sorting looks like this:

badXforms = [f for f in cmds.ls(tr=1) if '|' in f]
    count = 0
    # we need to somehow sort this list by the number of '|' appearing in each name. this way we can edit names from the bottom of the hierarchy up,
    # and not worry about losing child objects from the list.
    countDict = {}
    for f in badXforms:
        countDict[f] = f.count('|')
    # now sort the dictionary by value, in reverse, and start renaming.
    for key,value in sorted(countDict.iteritems(),reverse=True, key=lambda (key,value): (value,key)):</pre>

The first line makes the list of all transforms that have ‘|’ in them. Then in the next few lines I build the dictionary so that when I sort the names by the ‘|’ count I know I’ll have all the original names in the right order.

The last line is the trickiest. You can get all the keys and values of a dictionary by saying this:

 for key,value in someDictionary.iteritems():

print key, value 

But I need this dictionary sorted by value, not by key, and I need it sorted backwards (in descending order). So I use the sorted() function on countDict.iteritems(), and add the flag reverse=True to sort descending. The last part just swaps keys and values of the dictionary for the purposes of the loop, without actually modifying the dictionary. Lambda functions are great for this sort of thing. lambda (key,value): (value,key)  is a very simple function that just reverses the two inputs.

I only just learned this trick, so I’m sorry if that didn’t make as much sense as it should have. Anyways, this script is really coming in handy on the job I’m working on right now, so I’m posting it up here (right click > Save As) for everybody else. To run the script, first initialize it in your Python window:

 import hf_renameDuplicates as hfRD 

Then to run the script:

 hfRD.renameDuplicates()

Inside the parentheses you can use any number to determine the padding of the extension added to duplicate names. The default is 3.

Hope it’s useful!

Jun 062011
 

There are plenty of situations in motion graphics where you have tons of repeating elements. Designers love that kind of thing. You’ll also often be expected to make slight variations of these repeating elements… like a bunch of dots or squares or something that have slightly different textures to them, but are essentially the same thing. You could make a unique shader or a separate referenced asset for each one of these variations, if, say, you really hated yourself. Instead you could build the texture swap right into a rig, so the animator or designer or whoever could just pick a color or a texture for each object while animating the scene. This makes it way easier to direct, and you can just reference the same object over and over again instead of making multiple assets. I was just showing somebody how to do this the other day and figured it’d be a good thing to post about. Here’s how the setup works…

Continue reading »