It's just not that hard.
I was finishing Sushina's chef's coat yesterday, and so I mildly sculpted a high-res version of the coat in Mudbox because I find that some subtle detail helps to make it feel more like cloth (I tend not to do too much high-res sculpting for the characters in this game, because they are supposed to have more of a clean surreal look, and I have found that putting too much detail into the faces and such breaks that look). But of course I can't just use the high-res mesh, because 3DS MAX will roll over and die in multiple ways (way too slow, and way too impractical to assign the skin weights, completely ridiculous animation update times, etc.) So what I prefer to do is bake a displacement map from the high-res sculpt, and then use that.
But here's the thing: baking never seems to work. I'm constantly having problems with it. God for bid you have even a slight problem with your UVs - Mudbox et al will just lose their shit immediately if that happens. But assuming you do have good UVs, and everything's fine, still Mudbox seems to just have magical problems sometimes and you can't get a bake out of it (I've attached the image you get out of Mudbox if you try to bake the coat as a displacement map).
OK, no big deal, Mudbox didn't work. I'll just export the high-res map to MAX, and let that bake it with Render to Texture, right? Wrong. Even worse result there.
But then I thought, hey wait... I've written this code before! It's in Granny. So I dropped the old Granny plug-in into MAX, and used that to cast. What a surprise! Perfect map (attached).
What is the big deal here? I really don't understand. I mean, I feel like I did a good job on the texture baking tools in Granny, and I definitely think they're better than anything else I've seen. But I was thinking that was true about the quality of the maps produced, and the flexibility. Not about the fact that it works at all. I just don't know how you produce a tool like this that just plain doesn't work. It's not that hard of a problem. Getting the quality good is tricky, getting the speed up is an interesting problem, making it really flexible maybe... but just making it output a map that could vaguely be considered correct? How hard is that??
See? That's what it's supposed to look like! Granny to the rescue!
And for the record, this was actually done directly with GrannyCast with no adjustment parameters. I didn't have to set a search distance or play with anything like in Mudbox. So this is actually the simplest possible cast you can do, and it works great. So I seriously don't know what Mudbox is doing, but whatever it is, it sucks.
Except there doesn't actually seem to be any displacement information in it, just artifacts. What are all those weird splotches everywhere, in the unoccupied regions? Is that it's idea of edge filling? And what's with all the triangular artifacts filling up the actual occupied space? There's no displacement data, as far as I can tell, but there are tons of weird triangular artifacts.
I've seen lots of people post messages like "I can't bake my stuff" and whatever, and there's always this assumption that somehow they've made a bad model or bad UVs. But it's totally not true. Mudbox just has a really buggy baker. And hopefully the fact that I've baked a perfect map from the exact same model data proves that.
