Toadstorm Nerdblog

I’ve been working on some tools recently that open groups of Maya scenes in batch mode, running Python or MEL operations on each file (for example, to quickly add a render layer and apply materials to a large group of objects at once, or to get the names of all the references in a scene) and then either returning data or saving the file. This sort of thing could be done in the usual windowed Maya interface, but this is slow and means the user has to sit there and watch Maya open a bunch of files, run operations and all that. Running scripts on lots of files at once is much faster if you’re running Maya in standalone (“batch”) mode.

There are other, more informative posts on how to get Python running in standalone mode (I learned a lot from this one in particular), but the gist of it is that you start by running /bin/mayapy.exe, and then initialize the Maya Python interpreter by calling the following function:

import maya.standalone as standalone
standalone.initialize(name='python')

Once that’s done you can start scripting like normal, starting with import maya.cmds as cmds and working from there. Some commands are not available in standalone mode, especially anything to do with UI.

Anyways, while working on a script that is meant to process huge heaps of unsorted Maya files, I realized that crashes were going to be a frequent problem when opening ancient, broken scenes, so I couldn’t just load an instance of Maya in standalone mode from a script and let it run through everything in a loop. I needed to be able to process one file at a time, and then either return some useful information if the process is successful, or let me know if Maya crashes so I can avoid that file later on or at least mark it.

To do this, I used Python’s subprocess module. Again, I’m not going to go into a ton of detail about the module, it’s pretty complicated, but I’ll give a quick example of how I’m using it to return information from a Maya process when I am calling the script from Maya (or any other program, really).

Let’s say I want this script to add a new render layer to several scenes, and add all meshes in each scene to that new layer. First, I’ll write the script that actually creates the layer and assigns objects.

import sys
import maya.standalone as std
std.initialize(name='python')
import maya.cmds as cmds
filename = sys.argv[1]
layername = sys.argv[2]

The sys.argv[1] and sys.argv[2] are easy ways to pass variables to an external script. sys.argv[n] just means the nth command line argument, so sys.argv[1] is the first command-line argument when running the script (after calling for the script name, for example, by running mayapy.exe yourScript.py). When we call this script, we’ll pass it a file to open, and a layer name to add.

def addRenderLayer(filename,layername):
    cmds.file(filename,o=1)
    newLyr = cmds.createRenderLayer(name=layername,empty=1,makeCurrent=1)
    meshes = cmds.ls(type='mesh')
    xforms = []
    for i in meshes:
        xf = cmds.listRelatives(i,p=1)[0]
        xforms.append(xf)
        cmds.editRenderLayerMembers(layername,xforms)
        cmds.file(s=1,f=1)
    sys.stdout.write(newLyr)
    return newLyr

addRenderLayer(filename,layername)

Here we’re creating an empty render layer based on the “layername” argument, then we’re getting all the meshes in the scene. In most scenes, people add transforms to render layers, not their individual shape nodes, so we’ll do this by creating an empty list “xforms,” then going through each mesh in a loop and finding their first related transform node (hence the listTransforms(i,p=1)[0] since listTransforms returns a list) and appending it to “xforms,” then adding “xforms” to the new render layer. Then we just save the file and return the name of the layer (I just like having return values for my functions).

We also write the name of the new layer to stdout. I’ll go into more detail about this later, but it lets us send information back to the program that calls this script later on.

Save this file as makeNewLayer.py somewhere in your default scripts directory.

Next, we’re going to make the process that we run from Maya…

import maya.cmds as cmds
import subprocess
# replace mayaPath with the path on your system to mayapy.exe
mayaPath = 'c:/program files/autodesk/maya2011/bin/mayapy.exe'
# replace scriptPath with the path to the script you just saved
scriptPath = 'c:/users/henry/desktop/addRenderLayer.py'
def massAddRenderLayer(filenames,layername):
    for filename in filenames:
        maya = subprocess.Popen(mayaPath+' '+scriptPath+' '+filename+' '+layername,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
        out,err = maya.communicate()
        exitcode = maya.returncode
        if str(exitcode) != '0':
            print(err)
            print 'error opening file: %s' % (filename)
        else:
            print 'added new layer %s to %s' % (out,filename)

Okay, a lot going on here. The subprocess.Popen function is calling mayapy.exe with three parameters: first, the script we want to run (makeNewLayer.py) followed by the two arguments that script wants (the filename and the layer name), then the input stream and the output stream which are both set to subprocess.PIPE. I’m not quite knowledgeable enough about the subprocess module to go into specifics, but basically when you run a process there are three “streams”: the input, output and error streams, usually called stdin,stdout and stderr. When you set the input and output streams to subprocess.PIPE when calling Popen(), you can be sure that the process will receive any input stream you give it (which we are not doing here; instead we’re using the sys.argv method shown above) and also communicate back any output it gives.
The next procedure, communicate(), returns two variables: stdout and stderr. We can use this for debugging or monitoring progress. Normally in the Maya script editor, if your script fails for whatever reason, you will get some kind of (possibly) useful information back from an error. However, you won’t have any idea what goes wrong with a subprocess unless you are listening to the error stream.
communicate() does a couple other awesome things. First, and most importantly, it waits for the subprocess to complete before continuing with your script. This is REALLY IMPORTANT, as you definitely don’t want a script like this to continue running until the process is finished. Second, it sets a few variables on the Popen() object, one of which is Popen.returncode. If you have ever slogged through Maya render logs and noticed lines like “Maya exited with status: 210″ or “Maya exited with status: 0,” those numbers are exit codes. Status 0 means the program exited normally; any other exit code means something went wrong. Since I want to be notified if something goes wrong when opening or editing a scene, I check to see if Popen.returncode is anything other than zero, and if it is, I print out the error stream and leave a notice that says something went wrong. If everything worked, I just want to know that it worked, and so I print a string. Since I wrote the output layer to sys.stdout, when I read stdout from the subprocess, I get the name of the newly created layer. I do this since the layer might not come out with exactly the name I wanted if another node in the scene had the same name as the new layer!

I’ve been using the subprocess module along with Maya standalone more and more as I make more complex applications. It’s incredibly useful to be able to process many files at a time… if you get any good ideas, you know where to post them.