NifSkope 2.0 Dev

[Astralify]

Hey all. First of all I want to thank Jon for developing NifSkope, as well as everyone else working on it. You guys are amazing.

Now, the reason for me posting is because kinda need some help with a very specific thing, that has been bugging me the last couple of days. I am sure many of you are familiar with it at this point. I know it's not really related to NifSkope, but I really want to know the reason for it happening, and if it's even possible to be fixed. I am talking about the loss of detail/precision when exporting for fallout 4. I know that Figment's plugin for max as well as Outfit Studio has the problem. You can check out this archive to better understand what I mean.

From what I understand, it appears to be a result from scaling (that I honestly can't figure out why is necessary) and decimal rounding that leads to loss in detail (single precision FP limitation in max/exporters perhaps?). I've tried everything I can think of, for example exporting the mesh scaled 10 times (with the idea to scale it down in NifSkope afterwards), but in NifSkope there is still the same detail/precision loss. Which means that it might be something different. I know that the values in NifSkope under vertex data are the real actual values the game uses (grabbing the mesh with ninjaripper also proves that these are the actual values/scale), so I don't really see why this loss in detail is happening with these exporters. The only reason I can think of is that both Outfit Studio and figment's plugin use the same code, with the same problem, which I think doesn't make much sense, or maybe there is something completely different that nobody knows yet. Now, I know you guys probably have better chance of figuring out why this might be happening, since you are already familiar with the new nif format.

Also, I would like to know if there will be a working "import" function for NifSkope at some point in the future. That can help fix the problem, because right know the only way is to manually copy-paste all vertex positions, in order to fix for example the seams that open on neck and wrists. Of course we can do only the vertices on the edges of these areas, but the jittery-ness of the rest of the mesh still annoying.

Thank you. ;-]

[JuJooGuppy]

Not sure what else you need, but I too experience the invisible objects issue w/ shaders. Switching to lighting only makes them visible as noted. And, as noted, this is on an AMD GPU, specifically an R9 290x. I am running the Crimson driver, the newest non-beta last I checked, iirc. 

Hope its helpful, and glad it isn't just me heh.

I have a couple of quick questions since I am already posting:

A) Whenever I open vanilla extracted meshes that have animations, for example, a door. I can't actually see the animation when I hit play. Where as a custom mesh that has an animation in it, the animations play fine? Is there a way to fix this on my end? 

B) When I open vanilla armor meshes (and most custom ones, for that matter), the mesh itself appears below the 0 mark of the Z axis, with the bones being up above. The only mesh not to do this was a completely custom mesh from what I could tell. Is this the issue you refer to with the skinned objects not appearing in the correct location? 

I think that covers it on technical troubles I have ran into in my learning experience here..

Thanks again for all of your incredible work on this, its beyond appreciated.

[Jon]

@Astralify  I'm not really sure what all I can really address in your post other than two things.  

 

1. NifSkope currently does not actually handle floats correctly and double clicking and exiting the float edit will actually truncate the value even if you do not change the value.  So NifSkope is not the be-all end-all of editing floats either.  I will be addressing this.  This is less of an issue with half floats, which is primarily what FO4 uses.  But yes, when just reading/writing a file there is no precision loss.  Figment, Caliente, and I all worked on the new format together and came up with equivalent half-float treatments for our apps.   **However**, they are turning native floating points from their apps into half-floats upon exporting the meshes, so there is indeed a precision loss.  By definition half-floats are at half precision, so there will always be a precision loss.

 

Unfortunately what you're asking is a problem in Outfit Studio so I can't really help.  I do know that Caliente herself had issues with wrist seams I believe, and may have even had to tweak them in NifSkope after.  My memory of it is a bit fuzzy at this point.  You may want to contact her or ousnius instead, probably on LoversLab, and share your theory with the scale being an issue.

 

2.  I have no desire to maintain the Importer as both the import/export code are a wreck and there are programs that write to NIFs better.  The export was quick to hack in but the import is much different, especially because there are almost 20 unique vertex formats used by FO4.  I'd rather let Figment and others deal with it.  If I ever improve import/export it will be by moving everything to use assimp, a library which will make it much easier to import/export from/to many more formats.   Basically, Import/Export has been "use at your own risk" for years, and most of us would rather strip it out than try to fix it.

 

-------

 

@JuJooGuppy

 

 

A) I'd need an actual example file (or three).  These doors may just be using animations external to the file in this case.  Also, not necessarily all types of transform controllers are supported, and I never actually audited what changed between Skyrim and FO4 as far as how the controller blocks are used.

 

B) This is because basically all skinned human actor stuff is ~120 units too far down.  That is where the actual vertices are.  The nodes you see are not actual bones.  Just references to bones.   To correct this I'd actually have to import the skeleton.nif for that mesh and skin the object correctly.   Basically what you are seeing is the unskinned appearance.

 

--------

 

On the AMD issue front,  I thought maybe GPU ShaderAnalyzer (an AMD program) might shed some light, but it doesn't actually provide any help on potential runtime issues.  There are no compile issues and so the only way I could test myself is if I had AMD hardware...

 

The basic issue is that AMD is extremely strict with shader code, whereas Nvidia is very forgiving and assumes what you are trying to do (and generally gets it right).  So I am likely writing something 'wrong' somewhere that Nvidia gets but AMD does not.  Aside from me buying a second computer that is AMD based, I'll have to rely on other people.

 

First I'd need a test suite, i.e. ~10-15 meshes which have various issues on AMD cards.  Fully invisible, partially invisible, other issues, etc.  I'd need an AMD user to compile the list for me.  Then a few AMD users to test out different batches of shaders with the test suite files.  Any volunteers?  

[Necrocytosis]

Sure, I can do that, when I am at work at least

 

Also, is it possible to have "freeze transforms" on BiTriShapes as well? If you freeze transforms on the NiNodes the BiTriShapes ignore it. It is very handy to sometimes have them zeroed at a place to measure placements etc.

 

Freeze Transforms = transform>Apply . 

[Astralify]

Figment, Caliente, and I all worked on the new format together and came up with equivalent half-float treatments for our apps.   **However**, they are turning native floating points from their apps into half-floats upon exporting the meshes, so there is indeed a precision loss.  By definition half-floats are at half precision, so there will always be a precision loss.

I see. So, is it realistic to expect a um... "fix" sometime in the future for all the tools? Or it requires a massive code rewrite that will take some time?  I will make sure to speak to Figment and Caliente about it as well when I have the time. Although Figment made a statement that he will not develop his plugins further, so that might be a sad thing if he leaves his plugin in this state. I initially spoke to Nightasy, and he was the one that told me about the precision loss is happening with all available tools, so I guess everyone are aware of it. So thank you for clarifying what is causing it. Now my soul can rest for a few days and not worry about it.

 

2. I have no desire to maintain the Importer as both the import/export code are a wreck and there are programs that write to NIFs better. The export was quick to hack in but the import is much different, especially because there are almost 20 unique vertex formats used by FO4. I'd rather let Figment and others deal with it. If I ever improve import/export it will be by moving everything to use assimp, a library which will make it much easier to import/export from/to many more formats. Basically, Import/Export has been "use at your own risk" for years, and most of us would rather strip it out than try to fix it.

Thanks for clearing that up as well. Dealing with these new vertex formats must be a real pain for you guys. I almost never used the import/export anyway, I just thought of it as a last resort for fixing the precision loss problem, because even as you said "NifSkope is not the be-all end-all of editing floats either", the problem is nonexistent compared to the results from the max plugins and Outfit Studio.

[kikaimegami]

Hope you don't mind me blundering in here from the NifTools forums I can help compile that list. I don't think I can replicate the issue the other person was having, but I've definitely noticed other weird stuff with vanilla meshes now that I'm up to date on what the known issues are and that what I'm seeing shouldn't be happening.

 

EDIT: Entirely unrelated quick question, but am I correct in assuming that you can either have vertex colors, or weights, but not both at the same time?

 

EDIT2: Actually, I may have something. Or an interesting note in any case. I'll just put a step by step recount of what I did and what happened, since that's easiest.

 

1. Loaded meshes\Armor\BattingHelmet\FBattingHelmet.nif through Windows Explorer (So it opened a new Nifskope instance. This is while there's a Nifskope instance with no file loaded lurking behind everything else because I have stupid workflow and do most stuff through Windows Explorer.) This is one that doesn't have the issue and loads fine.

 

2. Rather than close that Nifskope instance and open the next file through Explorer like the idiot I usually am (and for future reference, this eventually makes Nifskope crash and burn after you've done it a couple dozen times, so I have to relaunch the dev version >.>), I decided to try loading one of the meshes that usually don't render: meshes\Armor\ArmoredCoat\GlovesF.nif What do you know, it renders fine now.

 

3. Tried a bunch of other ones I knew don't render, and everything is working as intended.

 

4. Close that instance (while still having the lurking instance, which does have a purpose, and that's so I can just have a billion Explorer windows open with different directories displayed since NIF is still associated with the older version of Nifskope) and tried opening the same meshes through Explorer in my normal dumb way. Nope, they don't render.

 

5. Flopped back and forth with the same meshes opening them in different ways and orders. Also tried opening the same meshes with my lurking Nifskope.

 

6. Loading a mesh that renders correctly first makes it able to render anything else you open afterward. My obsessive need to do something a very specific way has been screwing me over. Woo.

 

What it seems then, is that something is weird when Nifskope launches, but it sorts itself if you show it a mesh it likes 

 

EDIT3: Yeah I'll just keep editing this post since I think everyone is asleep and I don't want to spam up the topic with half a dozen new posts.

 

For your viewing pleasure, here's the freakout Nifskope has when I open too many things by spawning new instances and then closing them:

Problem signature:
  Problem Event Name:	APPCRASH
  Application Name:	NifSkope.exe
  Application Version:	0.0.0.0
  Application Timestamp:	0537d5d8
  Fault Module Name:	atioglxx.dll
  Fault Module Version:	6.14.10.13417
  Fault Module Timestamp:	567abfba
  Exception Code:	c0000005
  Exception Offset:	00e4ea5c
  OS Version:	6.1.7601.2.1.0.256.48
  Locale ID:	1033
  Additional Information 1:	0a9e
  Additional Information 2:	0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:	0a9e
  Additional Information 4:	0a9e372d3b4ad19135b953a78882e789

EDIT4: Also, Nifskope takes longer than normal to launch again after this happens.

 

EDIT5: In anticipation of being asked because of the dll listed there, OpenGL is v4.4.

 

EDIT6: I feel like an ass coming in here to report on the shader issue and then bringing this crap up, but I also feel like an ass for not realizing the shader issue wasn't a known issue anymore, nor even related to what I thought the issue was, and so not trying to report on it

 

Plus, considering the error, it may actually be related. I'm no programmer, though.

 

Brought up Task Manager to keep an eye on what was happening there when Nifskope crashes. It seems that if you open files the way I do and then close each new instance of Nifskope once you're done with that file, it doesn't let go of all the memory it took up to load the file, only a portion of it. I had around 2 gigs of memory being used this way when it crashed again. Note that it does shed all its memory used if all instances are closed (or it crashes  ), so it's just when you always have at least once instance running and you're launching and closing additional instances.

 

It also slowly bumps up the memory usage if you just keep loading the same file over and over in the same instance (either by using Reload, or opening the file through the menu), or loading a bunch of different files. If you close that instance, it'll shed a chunk of this, but not all of it. I'm sure you could get it to crash this way too, but it takes much longer to really start hogging memory.

 

I'm just going to make a log of memory usage for a few files, so you can see what I mean. I'm pretty sure files that don't render for me eat up way more memory this way than ones that load fine, but I wont make that statement as fact until I actually check.

 

Or you already knew all this and it's just a thing. In which case, ignore my blathering

[Skibadaa]

Hi.

Im sad to hear that the import function is problematic and wont be updated any time soon, but I fully appreciate the reasons for this. The only feature that I personally have a burning desire for is the "scale vertices" command from earlier versions, as manually editing the X Y Z of individual vertices when reshaping a BStrishape is ultra time consuming on anything but the simplest shapes. So if that were to become a usable feature for F4 meshes my world would be complete.

 

Either way, thankyou for the continuing development of this essential tool! 

[robboten]

Wrote in the "dead" official forum. But please add a wireframe render mode. Would help me a lot to see the vertexes! Thx for developing!

[Kesta]

Is there arguments I can use to load a file from a bsa into nifskope (with the console or via programmation) ?

 

Link to relevant post in xEdit thread

[Jon]

I don't really understand what you're asking.   Starting with 2.0dev3 there is an actual BSA browser inside NifSkope.

[fireundubh]

Kesta wants a command line option, so s/he can run a command like nifskope.exe --targetFile=file1.ext --targetArchive=file2.ext to open an archived file in NifSkope from an external program, namely xEdit.

[Kesta]

I would like to use the windows command line to load a specific nif located inside a .bsa, without having to browse for this .bsa in nifskope before. The objective being to be able to open a specific model referenced by a record in a plugin from TES5Edit. 

 

Think of what the "preview" button from the CK is, you can visualize the model of a form directly, without the CK first popping in a window with "here is the name/path of your file, and here is the content of a .bsa, now browse through it to visualize your actual model". Except that with the xEdit/Nifskope combo, you don't need the CK, and can even edit the meshe.

 

 

Edit : Ninja'd 

 

P.S. : "he"

[Jon]

OK, yeah.  That use case is definitely useful, and probably rather trivial to implement.  I already set up command line param parsing in preparation for such features.

 

I'll look into it soon, but that feature alone probably doesn't warrant a release.  However I have the vertex selection mode that's been sitting for 3 months without a release... so, yeah. 

[MadCat221]

Have the screwy material names in the XML been sorted out and restored to their previous state?  They clash with all the Chunk tools.  That and the loose texture BSA loading problem are why I reverted to Dev 3.

[Jon]

Just use an older nif.xml specific to NifUtilsSuite?  You don't have to point it to the XML in your NifSkope directory.  I didn't like the material change either, but it's technically not our responsibility to maintain compatibility with a non-NifTools program.  Anyway, I can push to have that change reverted (figment also wants it reverted), but changes to nif.xml do not come unilaterally from me.  So you'll be stuck at dev3 for a long time, or you can just plop in an old nif.xml into your NifUtilsSuite dir.

 

And, are you talking about the UV editor issue?  Didn't realize anyone relied so heavily on the UV Editor that they'd revert versions.   Considering its current feature level my suggestion would basically always be to use 3ds Max (or Blender) for editing UVs.

[zilav]

I would like to use the windows command line to load a specific nif located inside a .bsa, without having to browse for this .bsa in nifskope before. The objective being to be able to open a specific model referenced by a record in a plugin from TES5Edit.

Press Ctrl+F3 in xEdit, find your model and double click on it.

 

Jon, will you check errors in alpha4 regarding some Skyrim meshes? I PM'ed you one test example some time ago, and heard that a lot of people have the same problem too.

[Jon]

@MadCat

 

Follow up about the NifUtilsSuite thing..  ttl269 informed me you can just find 

<MatScanName>SKY_HAV_</MatScanName>

in NifUtilsSuite.xml and change it to

<MatScanName>MAT_</MatScanName>

NifUtilsSuite has basically no chance of ever being updated again, so it's a very solid suggestion. 

 

---

 

@zilav

 

Yes I know that there were XML regressions.  However I didn't know the stuff you were sending wasn't particle related like the other regression I know about, with NiParticleSystems and NiPSysData.  I narrowed your mesh issue down to https://github.com/niftools/nifxml/commit/bbd0d5e2511e4c6f8cc31fd2e84905ff6233a9e4  which 2.0dev4 was the first to use this change.  The mesh you gave me is not vanilla, and I notice with the old XML the BS Num UV Sets is 4095 instead of 4097.  Is this NIF valid and does it load in game?  Also, why is it 4095 instead of 4097?

 

----

 

Edit: 

 

@zilav and @Cat 

 

Does this nif.xml resolve your mesh issues?  For zilav, the weird Num UV Sets and for Cat, the particles?

[zilav]

@zilav

 

Yes I know that there were XML regressions.  However I didn't know the stuff you were sending wasn't particle related like the other regression I know about, with NiParticleSystems and NiPSysData.  I narrowed your mesh issue down to https://github.com/niftools/nifxml/commit/bbd0d5e2511e4c6f8cc31fd2e84905ff6233a9e4  which 2.0dev4 was the first to use this change.  The mesh you gave me is not vanilla, and I notice with the old XML the BS Num UV Sets is 4095 instead of 4097.  Is this NIF valid and does it load in game?  Also, why is it 4095 instead of 4097?

It is a custom made mesh, not mine though. However it does work flawlessly in game, CK and alpha3.

[Jon]

Yes, but there is still the question of if it's supposed to be 4095 or if that is some kind of mistake.  Obviously the nif.xml shouldn't catastrophically fail because it's set wrong but it's the first instance I've ever seen of 4095.  If it has some actual meaning as compared to 4097 for the game then we also need to know what the difference is.

 

Anyway, I reverted the Num UV Sets change in the nif.xml I linked above.  If you could contact whoever made the mesh and ask "Why 4095?" I would appreciate it.

[zilav]

Anyway, I reverted the Num UV Sets change in the nif.xml I linked above.  If you could contact whoever made the mesh and ask "Why 4095?" I would appreciate it.

I'm sure it was a mistake, not some on purpose setting. I had to fix a lot of other settings myself, but that one slipped through. At least now I know where to look at in the future

Thanks!

[MadCat221]

Just use an older nif.xml specific to NifUtilsSuite?  You don't have to point it to the XML in your NifSkope directory.  I didn't like the material change either, but it's technically not our responsibility to maintain compatibility with a non-NifTools program.  Anyway, I can push to have that change reverted (figment also wants it reverted), but changes to nif.xml do not come unilaterally from me.  So you'll be stuck at dev3 for a long time, or you can just plop in an old nif.xml into your NifUtilsSuite dir.

 

And, are you talking about the UV editor issue?  Didn't realize anyone relied so heavily on the UV Editor that they'd revert versions.   Considering its current feature level my suggestion would basically always be to use 3ds Max (or Blender) for editing UVs.

 

 

I use it just to look at layout and see what parts are where on the texture if I'm working on the texture in Photoshop without having to fire up Max too or get all involved in exporting a UVW (which is a rigamaroll for NIFSkope).  The bug precludes that.

[zilav]

Does this nif.xml resolve your mesh issues?  For zilav, the weird Num UV Sets and for Cat, the particles?

It works now in alpha4 so far, thanks again.

[Hana]

I use it just to look at layout and see what parts are where on the texture if I'm working on the texture in Photoshop without having to fire up Max too or get all involved in exporting a UVW (which is a rigamaroll for NIFSkope).  The bug precludes that.

Ditto. Also for minor UV editing for bug reports like stretching or moving UV's that overlap other parts of the texture in error. One of the most handy features we had.

[vurt]

What am i missing if the Lightning function (lightbulb icon) does not work on a Mesh? I have one mesh where this will have an effect, but most of my tree meshes are as black as usual when i use it..

 

I was thinking there's something i have missed in BSLightingShaderProperty so i copied the whole thing from the working to the non-working, but no change.

 

I'm looking to render my tree meshes with decent lightning inside of Nifskope (to make LOD's of them). The trees are terribly dark, black almost, so it would be great to have this functioning.. 

 

Works: http://i.imgur.com/z9Grzp6.jpg

 

Does not work: http://i.imgur.com/9XnGMhV.jpg

 

Meshes:  https://www.dropbox.com/s/u4xslts25z55xp2/trees.rar?dl=0

 

If anyone could help it would be very appreciated.

[Jon]

Are you sure you didn't reverse the labeling on the NIFs?  working.nif is super dark and the lighting is completely wrong on it, and nonworking.nif looks fine.   I don't have textures though so I can't say for sure.

 

nonworking.nif:  https://i.imgur.com/rzFRG7X.png

working.nif: https://i.imgur.com/0mY02iw.png

 

Both with frontal lighting.  The first one looks perfectly correct considering I don't have textures.

 

I can make the second one look better if I redo the normals: 

 

frontal:  https://i.imgur.com/oWhOJYD.png

top-down:  https://i.imgur.com/GXN5NP1.png

 

Anyway, if you can, please send me the textures necessary for those two files so I can really see what's going on.    Also if you're going to make billboards you probably want to go into orthographic mode.   It's the 5th button after the Undo/Redo on the toolbar.

 

Edit:  If I do a Copy/Paste Over using the BSLightingShaderProperty from the one that looks fine to me, the one that looks super dark suddenly looks much brighter but the normals are still clearly wrong, and this is definitely getting rendered correctly by NifSkope.   So if the lighting looks fine in game, your normal maps must be reversing the lighting or something.  

 

Edit2:  The brightness change when pasting over the other BSLSP is because working.nif doesn't have SLSF2_Vertex_Colors on the BSLSP despite the NiTrShapeData having vertex colors.