Jump to content

NifSkope 2.0 Dev


Jon
 Share

Recommended Posts

Whooooops, that would be my fault :X  wasn't paying attention, lol. Poll answered.

 

No biggy.  I'm not sure anyone after you answered the poll either because the votes had been unchanged since before your post.   :)

 

Anyway, I restructured the post so that the poll link wasn't buried in the middle.

Link to comment
Share on other sites

660Ti, after running GPU caps viewer it is actually 4.5

 

Yeah, that's why I mentioned that knowing driver version is important for that database.  I think Nvidia has retroactively added OpenGL 4.5 support for their entire line of cards going from 400 series to 900 series.   However, sometimes people refuse to upgrade their drivers past a certain point for various reasons like game compatibility/performance, so they might be stuck with lower OpenGL versions.

 

I'm definitely not planning on a hard requirement of anything above 4.0 though...  It's possible to just query for specific extensions, if say I want to use something from 4.4 I can do so as long as I check that your computer has it.

 

So, a vote for 4.3 is functionally equivalent to 4.4+ at the moment..   I'm really only concerned with anyone who is at less than 3.3.  If it's enough people I'm going to have to consider fallbacks in the new renderer, or just fall back to the old renderer.

Link to comment
Share on other sites

  • 1 month later...

It's nothing too amazing, but I decided I'd tease a little functionality I've been working on.  I've started utilizing NifSkope's BSA reading capabilities (for textures) to load NIF files directly from BSAs.  So, for those who don't want to extract their entire BSAs, they won't have to.

 

Right now the UI is just a preliminary "at least it works" setup. :)   I plan on it looking more like a file browser.  Especially because opening large BSAs just doesn't work in a flat list.

 

BiGhrTK.png

 

 

There are a number of issues that doing this brings up though, where the program currently assumes that the open file actually exists on the filesystem.   Mostly boring, unimportant things, but a few things I could probably get community input on:

 

Primarily, do I include these files in the "Recent Files" list?   Or do I only add it to Recent Files once you save a copy to the hard drive?   Do I split the two up and have a Recent Files and a "Recent Archive Files"?

 

Or, do I include the actual BSAs you've opened in the Recent Files area?  Clicking on these will open the file browser for that BSA, and maybe the UI for the file browser will have a "Recent Files" for the NIFs you've opened from inside that particular BSA.

 

------

 

In this same vein,  I plan on having some kind of info panel that will tell you all the resources currently loaded for that file and where they are loaded from.  For example textures A, B, and C are loaded from `Skyrim - Textures.bsa` while textures D, E, and F are loaded from `HighResTexturePack01.bsa`.   That way you will know for sure what textures the currently open file is using, and be able to correct the priority of your BSAs or texture paths in the Settings if necessary. 

 

It would also give the option for you to extract/copy all of these files to one centralized location, so that if the file and all its resources are strewn about multiple BSAs or hard drives you can quickly collect all the files in one place.   I could potentially allow for the creation of a zip file with all the needed assets too, but that runs into cross-platform issues that I will need to research in order to implement.

Link to comment
Share on other sites

Ahh, you're back!

 

I think the BSA should be listed in the "recent files" since the NIF and textures being loaded are part of it unless extracted.

 

On BSAs... will it automatically parse all BSAs in the data folder or will NIFSkope need a command to load specific ones to find textures (or other referenced files)?

 

A bug and a few feature requests...

The bug: If nodes are set to be invisible, selecting one will make it visible and highlighted yellow.

 

If I set nodes to visible, selected nodes do not turn yellow anymore.

 

And feature requests:

 

When looking at a vertex color array, is it possible add a feature so you can click on a specific vertex in the model view pane and have it selected in the array in the block details pane?  Vertices are made visible when in a vertex color array, and specific vertices selected in the block detail array are highlighted in the view pane.  However, the list for them can be huge, and finding a specific vertex you want to change is practically impossible unless it's a very low vertex count. Masically, my request is to do this in reverse: select in the view pane instead of the array list.

 

Also, are you able to set up "Update Tangent Space" to detect if two overlapping vertices have the same normal vector?  If so, could you make that function also give them the same tangent and bitangent vectors?  They do cause normal map rendering issues, though nowhere near the degree that bad normal vectors do...

Link to comment
Share on other sites

Or, do I include the actual BSAs you've opened in the Recent Files area?  Clicking on these will open the file browser for that BSA, and maybe the UI for the file browser will have a "Recent Files" for the NIFs you've opened from inside that particular BSA.

I like this particular option, mostly because I'm not sure of the other option and how you would present a "recent files" and a "recent archive files" in the same place. This option however will tell me exactly where I'm going, but I would definitely want the "recent files" presented once I'm in the bsa.

 

As for the textures, excellent idea. I'd like the panel to be hide-able though, like everything else. :)

Link to comment
Share on other sites

If you're back on it, I have a bug I found...

 

 

When looking at a vertex color array, is it possible add a feature so you can click on a specific vertex in the model view pane and have it selected in the array in the block details pane?  Vertices are made visible when in a vertex color array, and specific vertices selected in the block detail array are highlighted in the view pane; basically, my request is to do this in reverse.

 

I'll take a look into the bug.  And BTW,  I may seem "back on it", but I never actually left it.  :)  I was mostly experimenting with my new renderer ideas and also trying to decouple the existing renderer from other parts of the code so that the two can be interchangeable.   I haven't been as successful or quick with all that as I'd like though,  so I decided to put it on hold for awhile while I work on implementing some much easier ideas.   That way you all at least get some goodies. 

 

 

As for the feature request, I've often mentioned my desire for vertex selection among other typical modeling features, like transform gizmos for the current selection.  The issue is these are hard to implement.

 

The issue right now is that the selection mechanism uses a "color buffer" where the model basically gets rendered into flat colors (Example), and these colors are an encoded ID.  This ID is decoded when clicking on a color (which happens invisibly to the user), and then you look up what block this ID maps to.   Currently the only thing you can select with your mouse is visible meshes, so this mapping is easy because these objects have an index associated with them (e.g.  "8 NiTriShape" in Block List).

 

In NifSkope, individual vertices are tracked in the model the same way the entire top level blocks are, by something called a QModelIndex (unimportant to elaborate).  The thing is vertices don't also have a simple integer index like the main blocks do, so that makes encoding the vertices to not collide (same color) with each other or other blocks pretty much impossible.  There are 256^3 possible color combinations, which is plenty,  but I have to assure that "Vertex 5 in Block 4" has a different ID from "Vertex 4 in Block 5".  

 

So then I have to come up with a few things to alleviate this.  "Selection Modes" where the color buffer changes to either Object mode or Vertex mode.  Then I need to come up with a way to actually list all the vertices in a model and map to their QModelIndex in a way which each vertex can be assigned a unique color for the color picker.  I could also go an entirely different route,  via raycasting for selection.  So, I know how I could do it,  it's just whether I can understand all the ancient rendering code enough to messily hack it in.

 

The thing I can't really emphasize enough is how painful it is mucking about in the old renderer code.  To the point where each time I want to implement a new feature I question if it's worth it to try to hack it into such old code,  or if I just reroll a new renderer for all these new features.   Which is why I've been looking into just coming up with a new renderer so that I can tackle several features on my wishlist,  including proper modeling tools.

 

I could still attempt to do both.  Hack the feature in the best way I can into the old renderer--which has been my method for all the other features I've introduced since 1.1.x--and then just do it the "right way" when I write the new renderer.

Link to comment
Share on other sites

I like this particular option, mostly because I'm not sure of the other option and how you would present a "recent files" and a "recent archive files" in the same place. This option however will tell me exactly where I'm going, but I would definitely want the "recent files" presented once I'm in the bsa.

 

As for the textures, excellent idea. I'd like the panel to be hide-able though, like everything else. :)

 

I dunno, something like this mockup:  https://i.imgur.com/MOVHpRx.png...  Or, the alternative I think we both prefer would be more like this: https://i.imgur.com/8BiHgDG.png (BSAs would be listed in the empty area)

 

Then the browser window for the BSA could house the list of recently opened files from it.  The only downside really is that it's a lot more information to track, which I can certainly handle.   Basically for each BSA, I could track maybe the 5-10 most recently opened files from it.  And maybe only show 5-10 most recent BSAs which were opened.   So that's anywhere from 25-100 filepaths I have to keep track of.  Assuming that once a BSA falls off the "recent archives" area, I wipe all the information for it.   I could theoretically keep track of the 5-10 most recent files from every BSA you ever browse, but this information would bloat over time.

Link to comment
Share on other sites

Yeah, definitely #2 pic was what I had in mind.

The problem with #1 would be knowing what bsa the recent file was from. There's a billion files named cuirass_0.nif in all the bsa's, for example. :X

Link to comment
Share on other sites

Yeah, definitely #2 pic was what I had in mind.

The problem with #1 would be knowing what bsa the recent file was from. There's a billion files named cuirass_0.nif in all the bsa's, for example. :X

 

True... but the full filepath of the file is shown in the status bar when hovering on the Recent Files list.   Example  (Bottom left)

 

I could have adapted that to show the path of the BSA for archive files.   But I still do prefer the second method where the Recent Files is just part of the BSA browser.

Link to comment
Share on other sites

Seems I missed some of your edits from your last post as I was replying to an older version of your post

 

On BSAs... will it automatically parse all BSAs in the data folder or will NIFSkope need a command to load specific ones to find textures (or other referenced files)?

...

Also, are you able to set up "Update Tangent Space" to detect if two overlapping vertices have the same normal vector?  If so, could you make that function also give them the same tangent and bitangent vectors?  They do cause normal map rendering issues, though nowhere near the degree that bad normal vectors do...

 

Not sure I understand the BSA question.  The only thing I'm adding right now is treating BSAs like a regular folder so that you can open NIFs from them.   The texture loading remains unchanged and works in the following order:

 

1.  Loose files at the root of the open NIF.  For example  `C:\Users\Jon\Desktop\blahblah.nif` is open and so the program looks first at e.g. `C:\Users\Jon\Desktop\textures\...`

2.  Loose files, in the order listed in Settings > Resources > Paths

3.  BSAs in the order listed in Settings > Resources > Archives

 

For any particular texture, searching ceases the moment a match is found.  So if you want particular paths or BSAs to take priority, they should be higher on the list.

 

This will apply to NIFs loaded from a BSA as well.   Note that if you want to speed up your program/NIF load times, you should remove any BSA from Settings > Resources > Archives that doesn't actually contain DDS files.   The BSA list there will *not* be co-opted for the BSA browser for opening NIFs, because firstly that list is for assets referenced in NIFs and not the NIFs themselves, and secondly that'd mean you'd need to add new BSAs there before being able to browse them.

 

Basically, just like there is  "File > Open..." there will be "Browse Archive...".  You select a BSA and it loads a list of all the NIFs in it.  Then you pick what file to open.  I'm also considering just making this a dockable tab like  "Block List" and "Block Details"  because it's something you could just keep open while browsing through files.  Sort of like UE4's Content Browser.

 

Like I also mentioned, I want to have an Info pane for the loaded NIF which tells you all the currently loaded textures and which path/BSA it is loaded from.   If your question was pertaining to this, the answer is that NifSkope is already looking through the BSAs listed in your Settings,  this pane is just going to clarify for each texture where it's actually been loaded from.

 

---

 

Re: the tangent space code,  I'd need a good sampling of meshes that fit your example use case.  I'd need to be fairly sure that in all cases that two overlapping verts with near-identical normals are meant to also have the same tangent/bitangent.   It's possible that there is a selective bias here, where say we find 0.1% of NIFs show this issue, but what we don't notice is all the overlapping normal/verts where differing tangents/bitangents is actually intentional and doesn't look wrong.

 

Even then, it's something I'd have to research more in depth.  For instance I think that the tangent and bitangent must be aligned to the UV directions in the texture space.  The overlapping vertices you mention are definitely on different triangle faces, and as such may not even have the same texture or UV alignment.  

Link to comment
Share on other sites

Do you have to tell it to load a BSA to get textures, or will it be more automated?

 

That's a more succinct rephrasing... should have started with it.

 

As for the tanspace problem, I know a prime candidate: the dwemer centurion bust.  In USKP, I corrected a rather unsightly concave pair of large triangles on the back sides, but the mirror UVW seam down the center has a noticeably different rendering in the normal map compared to the original with the bad concavity, despite the normal vectors being the same.  Tangent and bitangents are

 

Any mesh with unified normals that went through a modder's import/export will probably have a split tangent/bitangent along the split UVW seams that NIF exporting seems to do.  Ever since... I don't remember which version, I don't trust the 3DS Max exporter to export the tangents and bitangents properly, so I export with just normals and use Update Tangent Space in NIFSkope to generate the rest of the tanspace (after setting BS Num UV Sets to 4097... anyone know the labeling for that bit flag?)

 

Packing the original and USKP'd versions up...

 

EDIT: https://dl.dropboxusercontent.com/u/36069651/DweCenturionBust.7z

Link to comment
Share on other sites

I'll take a look at those files.
 

Do you have to tell it to load a BSA to get textures, or will it be more automated?

 
OK, so do you mean that you don't think that the Auto-Detect Archives button in Settings > Resources > Archives is good enough?   It can't really get more automated than that.  You at least have to tell NifSkope about your game installs.   I mean, I guess I could switch more to a "tell NifSkope where this game is located" setup, and it'd just scan *all* the BSAs before loading a NIF, but this would actually remove options for the end user, like telling NifSkope what priority to give BSAs by reordering them in the Archives list.  It would also increase loading times for NIFs to do that.
 
I guess there are a few improvements I can think of for this mechanism:
 
1.  I could have the Auto-Detect work when opening NifSkope, telling you that there are new BSAs that it hasn't registered.  But I would have to modify the way I store the list of BSAs.  Right now, ideally you need to Auto-Detect, and then manually remove all the BSAs that don't contain textures.   For example this is my Unoptimized list after using Auto-Detect and my Optimized one.  Anyway, the second you remove a BSA from the list, then NifSkope would detect it as new the next time you start up,  so I would have to actually keep all the BSAs in a list but mark them with something like "include" or "skip".  
 
2.  Actually recurse through all BSAs during the detection process and discard BSAs that don't have any texture files.  Or have an "Optimize" button which removes them after the fact.  Or a "Skip BSAs without textures" checkbox under the Auto-Detect button. 
 
3.  Determine based on NIF version which BSAs can be skipped because they belong to another game.  This would speed up loading of NIFs depending on where the BSAs for the game are in the list.  Automating something like this would be really problematic though.  I'd rather the user set up the version-game-BSA associations themselves. 
 
 
Also, I suppose that when it comes to the new BSA-loaded NIFs,  I could modify the program to look in that particular BSA without you having to have already specified it in the Archives list.   But this wouldn't help for vanilla NIFs, since the meshes are in their own BSA.   However it would help with loading NIFs from mod BSAs, where the entirety of all the assets for the mod are in one BSA.   Issue is that the moment you save that NIF to your filesystem that it will no longer look in that BSA.  So, yeah..  you'd still need to add the BSA to the Archives list.
 
If I still haven't touched upon the issue, I suppose you'll have to provide me some use cases, because I'm still not really sure I get what you're asking for. 
 
-----
 
Just a small update on my progress...   I have the BSAs actually loading into a tree now (Example).   Though I have to implement a custom sorter to get the folders to sort above the files.   I was also expecting loading enormous files to be slow based on my experience with BSAOpt,  but it's actually freakishly fast for me.   It takes <0.5s for the 1.4GB Skyrim meshes BSA.  It takes 10x that for me with BSAOpt.
 
Ideally I will have searching/filtering too.   It's actually pretty trivial to do with the way I've stored the tree of files.
 
---
 
I do have one question though... Do Morrowind BSAs follow the same folder schemes as later Bethesda games?  
 
That is, are all NIFs under "/meshes"?  IIRC I think they are under a different path in Morrowind, which is an issue because I'm truncating the BSA tree to just the "meshes" folder so that you get something like this:
 
rWxAbgb.png
 
So that you're only seeing the part of the BSA where the NIFs are.  I still do parse the entire BSA and build the entire tree first, so I suppose that I can just say "If there is no 'meshes' folder, show the entire tree".

Link to comment
Share on other sites

So I have an issue when it comes to filtering, basically do you want:

 

1.  The filter to search the filename (or folder name) only

2.  The filter to search the entire filepath

3.  Both (some kind of checkbox to switch the behavior)

 

Here's two sets of examples demonstrating why each way can be bad/good:

 

"ebony"

Filename only (Bad)

Full path (Good)

 

So the "bad" one, nothing under the "ebony" and "ebonymail" folders actually contains the string "ebony" so they won't show.  Obviously this is not ideal.  Many armors are only identifiable by their parent folder so only that folder would show, meaning searching for them would be pointless.

 

"dwemer"

Filename only (Good)

Full path (Bad)

 

The "bad" one here is just that you might get a lot more results than you are expecting if you had a particular file with a particular string in mind when searching.   You'd then have to wade through all the stuff that says just "dwe" (prefix) or "dwarven".   But at the same time, these results are useful because they're showing you a lot of stuff related to Dwemer.

 

So, right now my opinion is that I think that #2 should be the default behavior, but I am open to figuring out a "Filename Only" checkbox or something.  Also, it supports either full RegEx searching or a more simplified wildcard style, like the searching in the CK.   I think I am opting for the latter since not too many people are RegEx gurus. :)

 

Link to comment
Share on other sites

d13frYx.png

 

I've incorporated the widget into a dockable tab since I felt it was less intrusive.  I have it defaulting to the same tabbed area as Block List / Header, but you can of course move it to wherever you'd like.  You can also keep it undocked as a floating window.  

 

To save on vertical space I've also decided against having an "Open" button at the bottom like prior versions.  You can simply double-click on a file to open it. 

 

As far as the tabbed behavior, the Block List tab will always rise to the top when clicking on the mesh in the viewport.  It's always done that, so nothing changes with having the Archive Browser tabbed together with the Block List.  When you do open a BSA from the menu, the Archive Browser will similarly take focus.

 

The area next to the BSA filename (e.g. "Skyrim - Meshes.bsa") will house a Recent Files dropdown for that specific BSA.  I will likely also put a Recent Archives dropdown in that same area instead of polluting the main Recent Files menu with a list of BSAs (like this previous mockup).   I think keeping it all together like this is best instead of having stuff relevant to the Archive Browser in multiple areas of the program.

 

----

 

Also, a side note about the dockable widgets...   Having the text "Archive Browser" directly above "Skyrim - Meshes.bsa" in that example really bugs me (visual clutter).   The dockable window title is redundant when it is docked and tabbed (the title shows at the bottom in the tab section).  Unfortunately I will need to create a custom title bar widget for the dockable windows to get this behavior.   Luckily Qt lets you do this easily, I just need to replicate the float/close buttons on the right of the dock title bar too.  

 

Also, I'm compiling with Qt 5.5 at the moment, and in 5.6 they finally let you reorder the dockable tabs by dragging the tabs.  Currently reordering the dockable windows is a nightmare.    If you accidentally undock any of the dockable tabs,  the easiest way to get it back is to just **double click the top of the window** and it will go back to where it was.   If you drag it back to where it was, it'll change the order.  So, once 5.6 is out I will start compiling with it to get the improved behavior.

Link to comment
Share on other sites

Jon apologies if this was mentioned before .. Just passing through and going out soon so no time to go through the topic ..

 

Got a few errors with loading shaders - Presuming these are to do with oblivion due to the naming, and the fact that I do not have oblivion installed

 

ob_normalmap_vcol_e.prog:
 
The fragment shader uses varying D, but previous shader does not write to it.
The fragment shader uses varying A, but previous shader does not write to it.
Out of resource error.
 
ob_parallax+glowmap.prog:
 
The fragment shader uses varying HalfVector, but previous shader does not write to it.
Out of resource error.
 
ob_parallax.prog:
 
The fragment shader uses varying HalfVector, but previous shader does not write to it.
Out of resource error.
 
ob_parallax_vcol_ad.prog:
 
The fragment shader uses varying HalfVector, but previous shader does not write to it.
Out of resource error.

 
V2 pre-alpha 2
 
Edit : Ah wait, now I have more time .. Just found its already a known issue for machines sporting Intel Integrated Graphics, and additional switchable Dedicated Graphics ATI / NVidia cards

 

Shaders:  Yes, more meaningful messages is already planned.  Like I mention above, anyone getting errors will just have to rename or delete their shaders folder for now.  Switchable graphics is not something I know how to detect so they'll just need to make sure they're using the dedicated graphics before starting NifSkope.

 

 
Changed to High for your exe and its no longer a problem

bGmp973.jpg

 
MNelson is very helpful with this stuff, and I believe he already has some kind of detection for multiple Graphics cards in Skyrim Performance Monitor

MQkDJCX.jpg

Link to comment
Share on other sites

NifSkope 2.0.dev3 (2015-11-02)
 
Summary
 
- New Archive Browser (Image) for opening NIFs directly from BSAs.
- Improved BSA auto-detection. The "Auto Detect Archives" feature in Settings > Resources > Archives now only includes BSAs which actually have textures.

- You can use your Back/Forward buttons on your mouse to navigate through previously selected blocks.  UI support may come later.

- Some unnecessary/useless columns hidden by default e.g. in "Block Details".  These come from nif.xml metadata and are not actually editable data in the file which can be confusing for new users.

 

Upgrade Notes

 

It is recommended you clear your Archives list and redo the auto-detection. This helps cut down on program and NIF load times.

 

Compiled with Qt 5.5 which removed a dependency on ICU libs, so the download/install is now much smaller.  Your UI will reset, e.g. window sizes and column widths.  All other settings will remain intact.

 

---

 

Let me know if you run into any issues with the new build or the new features.

Link to comment
Share on other sites

It seems if you zoom in enough in Orthogonal view mode, a negative zoom wedgie occurs.  Could a limiter be put on how far in you can zoom to prevent this?  It causes facings to invert and some other glitching in the rendering.

Link to comment
Share on other sites

There already are zoom min/max limits.  I just may need separate values for orthogonal mode.  I do see some weirdness with environment maps and specular.  I don't see any actual *faces* invert, just some lighting effects. 

 

Can you tell me what files you notice these glitches on?  i'm not finding any meshes where any glitches are particularly noticeable.

Link to comment
Share on other sites

Jon, do we know what Shader Flag 2 - anisotropic lighting is/does?

 

I've been going through all the Hairs and not seeing any consistancy with it's use. For example; Human Females do not use it, but Human Males do. On the other hand, Female Child uses it. Or, a hair.nif will use it but the related hairline nif doesn't.

 

Along the same line, Shader Flag 2 - assume shadowmask. Sometimes it's used, sometimes not, sometimes with anisotropic lighting, sometimes without.

 

I'm hoping this will make sense if I understand it. Thanks.

Link to comment
Share on other sites

I think I (at least partially) figured out what Assume_Shadowmask does when I was troubleshooting a problem with Saerileth's Gemling Queen Jewelry mod (which derives and expands upon the "true circlet gem" effect I first implemented)

 


Hey again, Saerileth...

 

Remember the problem that the circlet gems had with rendering while lit by an interior shadowcaster light, which inexplicably didn't have the problem when lit by the sun?  I think I found the solution.  It's not an inherent bug in the render engine but a shader flag we missed.

 

I noticed that the eye envmap shine seemed to work when lit by an interior shadowcaster light.  Piqued, I picked apart the shader flags and whatnot and compared it to the Copper Ruby Circlet (since that's what my current character wears).  After trying the Eye Envmap setup (which was a complete bust), I noticed two shader flag settings that the eyes had but not the circlets: SLSF1_Cast_Shadows, and SLSF2_Assume_Shadowmask.  After process of elimination, I determined that SLSF2_Assume_Shadowmask in the Shader Flags 2 set was what enabled the eyes to have enviromap reflect effects when lit by a shadowcaster light.

 

Essentially... I think it's involved in a setup where there's an envirocube reflection setup *and* alpha transparency.  Eyes have alpha transparency too, since the eyelashes are part of the eye NiTriShape and the whole eye NiTriShape has the alpha property.  The "outward" gem NiTriShape was essentially invisible when illuminated by an interior shadowcaster light (but not a non-shadowcaster light, or by exterior cell ambient lighting from weather systems... go fig :shrug: ).

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...