Jan 212015

I’m working on a production right now that involves sending absolutely enormous animated meshes (6.5 million polygons on average with a changing point count every frame) out of Houdini and into Maya for rendering. Normally it would be best to just render everything directly out of Houdini, but sometimes you don’t have as many Houdini licenses (or artists) as you’d like so you have to make do with what you have.

If the mesh were smaller, or at least not animated, I’d consider using the Alembic format since V-Ray can render it directly as a VRayProxy, but for a mesh this dense animating over a long sequence, the file sizes would be impossibly huge since Alembics can’t be output as sequences. Trying to load an 0.5 TB Alembic over a small file server between 20 machines trying to render the sequence and you can see why Alembic might not be ideal for this kind of situation.

Hit the jump below to see the solution.
Continue reading »

Mar 252014

I’ve been messing around with using Ptex in VRay for Maya and ran into a particularly weird little problem involving a normal map exported from Mudbox. It’s probably easier to show it than to tell about it (Fig. 1):

bad ptex bottle

Fig. 1. Check out the wax wrapper up top… all chunky and gross.

That wax wrapper looks terrible… not at all like the original sculpt. The mesh is set to render as a subdivision surface (using the custom VRay attributes from the Attribute Editor), but the details are all mangled like it’s not subdividing the surface at all.

Even weirder is what happens when I only render a small region of the image (Fig. 2):

bad ptex with region render

Fig. 2. The details are suddenly cleaner! All I changed was enabling region render.

This was really confusing, and although I’m still not 100% sure what’s going on in VRay to cause this (I’m fairly certain this is a bug), there is at least a solution.

What’s probably happening here is that choosing to render this mesh as a subdivision surface is changing the point count of the object BEFORE any Ptex information is applied to the mesh during rendering. Ptex is very sensitive to your geometry… any change in point count could potentially break things. You’re not allowed to polySmooth objects that will have Ptex textures, for example, or Maya won’t know what points to assign Ptex information to.

The way to get around this is to put the object into a VRay displacement set. Assign displacement and subdivision attributes to the displacement set like you normally would, but don’t attach a displacement map, and make sure the Displace Amount is set to 0. Rendering this way gets you the more correct image (Fig. 3):

correct ptex

Fig. 3. This looks a lot more like the original sculpt from Mudbox.

This seems like buggy behavior to me more than some technical thing I’ve overlooked, but thankfully the workaround is pretty minimally difficult. If you have any better insight as to what exactly is happening here or if there’s a less hacked way to prevent it, I’d love to hear it.

Jan 302012

I was just talking to a colleague about his problems using displacement maps with VRay, and then remembered my confusion when I first tried to work with them. So here’s a post about it!

Normally in Maya/mental ray, when you want to apply a displacement map you just create a displacement material and connect it to the shading group you want displaced. If you want to adjust the amount of displacement, you actually grade the image itself by adjusting the color gain and offset on the file node. It’s simple, it works, whatever.

You can still apply displacement like that in VRay, but there is a better and more flexible way to handle it using VRayDisplacement sets. They’re kind of like VRayObjectProperty sets, but they act as a sort of container for displacement settings instead of generic render settings and object IDs. In order to use these sets, you want to select the objects to displace with a single map, and go to Create > V-Ray > Apply single VRayDisplacement node to selection. A set will be created, visible in the Outliner.

Next up is to assign a displacement texture to the set. This means you don’t have to connect a displacement shader to any shading group; the set will handle that connection.  When you select this set, the Attribute Editor will give you just two options: a checkbox saying “override global displacement,” and a plug for a displacement material. Check the box on, and then connect a texture to the displacement material (not a material, but a texture). I usually run a file texture through a Luminance node first to make the connection easier (file.outColor –> luminance.value), unless I’m using a vector displacement map in which case I’m using color information instead of just luminance or alpha.

So where are all the displacement options? You have to add them. If you don’t change anything, the displacement will use the default values set in the VRay render settings under Settings > Default Displacement and Subdivision. This defaults to an edge length of 4 pixels, a maximum subdivision number of 256 (this is a lot of subdivisions!!), and a displacement amount of 1.0 (which is usually way too high). These are terrible values for most scenes, the displacement will look grossly exaggerated and it will take forever to render.

In order to tweak the settings, you need to add the appropriate attributes to the VRayDisplacement set. With the set selected, open the Attribute Editor and select Attributes > V-Ray > Displacement Control and Attributes > V-Ray > Subdivision and Displacement Quality. Now you have a ton of options to play with, the most important of which are Displacement Amount (color gain), Displacement Shift (color offset), Edge Length, and Max Subdivs. I recommend starting with a displacement amount of 0.1-0.5, and a max subdivs of maybe 16-32 before starting to increase those settings.

I have no idea why these attributes have to be added manually, but hell, it’s still better than mental ray.