Jump to content

Lorelai's 3ds Max tips and tricks.


Lorelai
 Share

Recommended Posts

VnBpIQL.png

 

 
 
 
Disclaimer: I know there is a newer version of the 3ds Max importer, but for the purposes of this tutorial/guide/whatever, I'm gonna stick with the Figment one (thank you, TheFigment!), version 7.3.7 (new one seems to have similar "issues" as far as Vertex Alpha is concerned, so these tips are still good to know).
 
Also, if vertex colors and alphas work fine for you, then YAY! but you might still want to continue reading, since this will give you better control on WHAT to do with those.
 
 
 
I. Vertex Colors and Vertex Alpha. "But..." -  NO *buts*! It's possible!
 
As we all know, vertex color importing (especially Alpha values) was not really working for 3ds Max users on the 7.3.7 release of the importer/exporter (Figment). And by that I mean that it kinda messed up the vertex colors completely, turning your mesh into a double rainbow (all the way).
 
Also, vertex color Alpha setting was not working either, the exported mesh always having a 1.0 value on the vertex Alpha.
 
But "not working" is saying too much - because it actually works! Rejoice! 
 
You can say that the importer is a bit fickle, and that sometimes on skinned meshes you have to do 1 or 2 imports before figuring out what settings to choose, but it won't take too much of your precious time, and you get nice, shiny vertex colors (and Alpha) as a reward. 
 
So what is the secret? Well...honestly, it's a click of a mouse button away. Yes - SURPRISE!
 
First let's talk about vertex colors. If the importer is going to mess them up, it's going to do so on IMPORT. To check if it did that, select your mesh in the viewport, right click on it and select Object Properties from the menu. In there, toggle on Vertex Channel Display.
 
So how do we do this one button magic to get both Vertex colors and Vertex Alpha to work?
 
Let's take a look at an architecture mesh first. For this example, I will use the whTempleTalos.nif located in: meshes\architecture\Windhelm.
 
Now, when you would import this mesh, and you would like vertex colors to be imported too, you would tick the Vertex Colors box under Geometry, like so:
 
XokIWkb.jpg
 
However, this has the nasty side effect of ruining Vertex Alpha and sometimes (it's fickle, as we will see later) the Vertex Colors too.
The one thing you have to do is click that tick mark again (once) until it looks like so: 
 
0BTnAeu.jpg
 
What this will do is import the mesh with the Vertex Colors (and Vertex Alpha) as separate modifiers on top of the Editable Mesh (if the mesh/TriShape has only white vertex colors, you will get no modifier on said shape). 
 
Let's next select shape WHtempletalos:42 from the scene (hide everything else). 
 
As you can see, it has 2 modifiers above the Editable Mesh. First (from the top) controls Vertex Alpha and the second one has Vertex Color information.
If you need to make any changes, or see the vertex colors/Alpha, use the Paintbox to switch to shaded view. Like in the picture below (displaying the Vertex Alpha on the mesh). 
 
M4kpxLO.jpg
 
Do NOT switch channels on these modifiers at any time because the vertex information will be switched to that channel and it will mess up everything. 
 
Export as usual, and remember to tick Vertex Colors on export. Check your mesh in Nifskope afterwards to make sure everything exported correctly. 
 
Now, on to Skinned Meshes. These are the rebel child and want to cause issues sometimes.
 
If you import any skinned mesh that has both Vertex Colors and Vertex Alpha (like the Khajiit head mesh, some hairs), with the Vertex Colors trick I explained before, you will see that you only get one modifier - one that holds the Vertex Alpha information. If you export like this, you will find that all of your Vertex Colors will be white and you will lose Vertex Alpha information. 
 
Now, if the mesh in question imports for you fine, with the checkmark being white, and it doesn't mess up the VC and VA, then do it that way. But if you need to use the trick to import the VA and VC correctly, here's how to fix the export.
 
Click on the Vertex Paint modifier in the Stack, then from the Modifiers List, add another Vertex Paint modifier. This will magically (yes, it's magic) read the vertex color info and add it in that modifier.
 
Then you can export the usual way.
 
(to be continued...)
 
 
 
Link to comment
Share on other sites

  • 1 month later...

Hello, Prometheus. 

 

First of all, make sure that you import exactly as I described in the tutorial (make sure the checkbox is black) - vertex colors and alpha are lost/messed up on import. Make sure everything imported fine by checking with the method above and also double-checking that the vertex alpha information IS there - select Vertex Paint modifier that has Vertex Alpha information and click on the "Vertex color display - shaded" in the Paintbox to see the actual alpha paint.

 

Second, make sure that you have both modifiers present. If you don't have a modifier for Vertex Alpha and only for that (do NOT have other colors/layers in this modifier to minimize possible issues), then you might lose the info from the mesh.

I don't think the order of the modifiers matter (haven't tested this out), but just in case, keep the Vertex Paint one that stores the Alpha information, on top of the one that stores the actual vertex color.

 

Third (as noted in red above), and this is very important, if at any point you edit the vertex alphas in any way and accidentally (or purposefully) change channels, you will lose ALL paint (alpha) information, as it will get transferred to the channel you switched to.

This is how 3ds Max works, and it's the same for any kind of vertex painting. Here's the quote from their help file itself:

 

 

If you have painted on a layer and then change the channel setting, the painted information will be moved to the new channel. For example if you select Vertex Color and paint, then turn on Vertex Illum, the painted information will be removed from the Vertex Color channel and applied to the Vertex Illum channel instead.

 

Let me know if you still have issues.

Link to comment
Share on other sites

Thanks for the reply , Actually even befoure reading I got what was my issue, and applying the following technique simply seems to have removed any issue of export so far.

 

when working with colors and alpha , the Vertex color must be under Vertex alpha then it exports fine . At least so far .

 

If I edited the underpoly somehow , like chamfering , cutting here and there etc ... the info might be messed up and require a repass of the vertex informations. the best and safer way to do it is to restack both even if the vertex color is untouched .

 

it seems to me that its the alpha the one that poses more issues usually .

 

Anyway so far seems to work and I finally got decent working vertex paint ingame .

Link to comment
Share on other sites

Glad you managed to get it working! :) 

 

So it was as I suspected, order on modifiers does matter. Like I said, I never actually tested it out but I suspected that the one that stores Alpha must be on top. 

 

Now we have confirmation that this is indeed how it should be. 

Link to comment
Share on other sites

  • 1 year later...

Just a small update, after long time I had to retake hands on vertex coloring , and had the same export issues, to solve them I had just left the two stacked one on the other, the colors under the alpha , but I had to change for the first the channel to 1 and the second the channel to 2 , for some reason 3dsmax placed both on channel 3 and the result was a messed alpha ... now with that fix all work fine ... Tought could have been usefull if someonelse had the same issue .

Link to comment
Share on other sites

  • 1 year later...

So reading this is really weird, because general practice for exporting anything from a scene is to collapse all of your modifiers (if it isn't skinned I guess) and with static objects specifically, exporting with modifiers applied seems messy and non intuitive.

 

I'm willing to try it, however, if I'm understanding this guide right. 

 

Edit: Yeeeeeep, so it doesn't really work. I'm not interested in importing meshes that already have vertex colors. I'm creating meshes from scratch which I want to have vertex colors. Whether or not a vertex paint modifier is applied, 0% alpha is always exported as 50.2% alpha. There are only so many times I can do the same thing or read the same instructions before recognizing that the tools have to be broken.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...