[WIPz] TES5LODGen

[zilav]

TES5LODGen by Ehamloptiran

Objects LOD generator tool for Skyrim.

Works in conjunction with LODGen.pas TES5Edit script which exports information about LOD objects in selected worldspace to a text file used by a command line LODGen.exe utility to build the actual objects LOD meshes (*.BTO files).

Differences compared to CK objects LOD generator:

- doesn't build texture atlas, generated meshes will be using original textures from LOD files

- lack of atlas allows using of tileable textures

- possible to use original full meshes as LOD models

- possible to use _far.nif LOD models located next to original mesh like in Oblivion

- can read meshes in Oblivion nif format

- flexible set of rules to define what models to include into LOD and how

- automatic tangents creation for meshes without them

- optional removal of tangents and vertex colors from LOD meshes

Start lodgen by applying "LODGen.pas" script in TES5Edit to anything after the background loader has finished.

World: worldspace to build LOD for, shows a list of worldspaces from loaded plugins that have lodsettings file

Save Preset: save current settings

Load Preset: load settings

Default: reset all settings to default values

LODGen export: name of text file to export LOD information

LODGen path: path where BTO files will be placed, script assigns proper path when changing worldspace

Mesh lodgen rules: a list of ordered rules that define what objects to export for lod generator. Script processes all references of STAT records in selected worldspace, gets full model name and checks it against rules starting from the top one. Any case insensitive occurence of value in a "mesh mask" field inside full mesh name activates that rule. Then "VWD only" field is checked against STAT record flags. If it doesn't match, then the next rule is checked. When the rule matches, fields LOD4, LOD8 and LOD16 define what mesh will be used in near, average and far LOD, and sctipt goes on to the next reference. The possible values are:

Empty - no mesh

Full model - a full mesh will be used

Static LOD4 - level 0 lod mesh as set in CK

Static LOD8 - level 1 lod mesh as set in CK

Static LOD16 - level 2 lod mesh as set in CK

_FAR model - a file having the same name but with _far.nif at the end will be used

If no rules match, that reference is not exported.

In the screenshot above there are 3 rules (those are for demonstration only, not real ones):

1) first one exludes rock01.nif mesh from the lod

2) second uses full mesh for LOD4 and none for other levels for every mesh that contains "meshes\rock\"

3) third default "\" rule matches any mesh since every mesh name has at least a single slash

The result could be read as "use default static lod models for VWD meshes, but all meshes in meshes\rocks\ will be using the full model even non VWD ones, except rock01.nif that is not included in LOD at all".

LODGen options:

- "don't fix tangents" check for skyrim meshes since they already have good tangents

- "don't generate tangents" remove tangents from generated BTO files, might actually look better on some badly decimated meshes

- "don't generate vertex colors" the same as above but for vertex colors

- "ignore material" looks like CK places all meshes that have assigned directional material into objSnow segment, check this setting if you are using lodgen for a custom worldspace without snow

- "remove unseen faces" removes LOD geometry that is under landscape LOD, requires landscape LOD *.btr files to be present. "Unpack landscape LOD" does it for the selected worldspace

- "keep underwater LOD" do not remove geometry that is under water lod

- "generate LOD levels only" select to generate only specific lod level

- "generate for blocks" generate lod only for selected exterior blocks

- "global LOD object scale" scales all lod by this modifier, for example 1.01 to make it slightly bigger. Might help when using full models in LOD to avoid Z-flickering on a transition border between full meshes and LOD in game

- "warn me to unpack..." script will ask to unpack all required lod meshes into Data folder for LODGen utility since it won't read meshes from BSA archives

Download the latest TES5Edit with LODGen

http://www.afkmods.com/index.php?/topic/3750-wipz-tes5edit/

[zilav]

WARNING!!!

Due to a lack of texture atlas generated LOD files are bigger than the vanilla LOD and requires more than average PC. No problem for a smaller worldspaces, but Tamriel worldspace in Skyrim is very packed with LOD, so generated LOD won't work for everyone resulting in infinite loading screens and/or crashes.

The only way to optimize it is to create RAEVWD mod for Skyrim that will not only add new lod objects, but change all existing vanilla lod models to use the same atlas texture. A manual made atlas approach instead of automatic atlas generation that CK does. And that would be a lot of work to reUV lod.

[Hana]

Awesome, just awesome.

A few questions;

- Only Objects, not trees?

- There is a step in the current Object LOD process to 'fix' quite a few meshes first to update translations.

With Nifscope

    There are nearly 800 LOD meshes but only 84 of them need attention. Here is a list of the files.The list is missing,Dwemer/dwefacadecolumnxlgfree01_lod_1,Dwemer/dwefacadecolumnxlgfree01_lod Media:LodMeshesPendingTranslations.txt
    Open each lod mesh and right click the NiTriShape, select Transform, Apply. Repeat with all shapes and save
    Make sure all of the LOD meshes are free of pending translations.

Is this still a required step?

- Re: the big texture warning - confused. There's quite a few low quality LOD textures that already exist, and a process to convert it into a texture atlas. Is it possible to continue using that process in combination with this process?

- Re: Unpacking the LOD meshes: It's not clear here, but have you taken into account not all the LOD meshes are in the LOD folder? There's a bunch of lod meshes in the Architecture folders mixed in with the regular meshes, and possibly hidden in other folders.

- Just a minor issue for me. The use of the term _far nif is going to confuse everyone. That's an Oblivion term. Skyrim calls them _lod nifs.

All in all, I'm looking forward to breaking this! I mean, using this.

[zilav]

- Only Objects, not trees?

Trees use their own LOD system, and CK plus tes5edit script to patch BTT files work nicely together, there aren't any limitations to overcome that would require an own utility.

 

- There is a step in the current Object LOD process to 'fix' quite a few meshes first to update translations.

Is this still a required step?

Should not be. But it is released for testing, so go an check

 

- Re: the big texture warning - confused. There's quite a few low quality LOD textures that already exist, and a process to convert it into a texture atlas. Is it possible to continue using that process in combination with this process?

CK combines all used lod textures from objects into a single atlas texture per worldspace which allows it to merge segments by material and HD LOD flag only. Our LODGen doesn't build atlas, the original textures from LOD models will be used by generated lod. Segments are merged by texture, material and HD LOD flag. It is way less optimized than using atlas, but more versatile: you can replace lod textures for example without rebuilding the whole lod, or add a new LOD models with new textures and rebuild lod only for particular block, or use tileable textures on LOD models, use unlimited number of textures unlike 64 limit in CK per worldspace, etc.

But that comes at the cost of performace. So if a CK like performance required, existing vanilla lod models must be updated manually to use the single atlas texture. We are not going to build atlases automatically, too much effort.

 

- Re: Unpacking the LOD meshes: It's not clear here, but have you taken into account not all the LOD meshes are in the LOD folder? There's a bunch of lod meshes in the Architecture folders mixed in with the regular meshes, and possibly hidden in other folders.

TES5Edit doesn't care what and where, it tracks the list of exported models that are missing in Data folder and allows to unpack them. This could lead to a mess when using mod managers, that's why it is better to unpack and install all needed meshes beforehand manually from mod manager.

 

- Just a minor issue for me. The use of the term _far nif is going to confuse everyone. That's an Oblivion term. Skyrim calls them _lod nifs.

It is Oblivion term, and LODGen allows to use the same system in Skyrim. You place _far.nif meshes next to original ones and just build the lod, no need to set up lod levels on statics in CK. Just like good old times of TES4LODGen. And as a bonus it actually can read meshes in Oblivion format without any conversion. I generated lod for Skyblivion just by unpacking RAEVWD meshes into Skyrim's Data folder and running LODGen, simple as that.

[MadCat221]

I really am leery about extracting BSA data into my Data folder.  Could an external BSA extract archive be designated?  I already have it all extracted into a folder completely separate from my Data folder.  Arthmoor and Hana have done the same, as I'm sure many other modders who work with assets have done.

[Arthmoor]

I really do strongly believe that reading from the BSAs needs to happen, or the program should be forced to read the extracted files from some other location outside of the Data folder. Once something like this gets out and people start unpacking their BSAs it's going to cause major havok with so many mods it won't even be funny.

 

You'd probably be surprised at how many modders who do anything regularly with meshes and textures have unpacked copies sitting around outside the main data folders.

[zilav]

I really do strongly believe that reading from the BSAs needs to happen, or the program should be forced to read the extracted files from some other location outside of the Data folder. Once something like this gets out and people start unpacking their BSAs it's going to cause major havok with so many mods it won't even be funny.

 

Reading from BSAs doesn't depend on me. LODGen already can be forced to read from other locations. When starting LODGen, script shows command line in messages tab

Saving objects LOD data to D:\Projects\TES5Edit\Edit Scripts\LODGen Tamriel.txt
LOD references: 14115, unique LOD objects: 681
Executing LODGen...
D:\Projects\TES5Edit\Edit Scripts\lodgen.exe "D:\Projects\TES5Edit\Edit Scripts\LODGen Tamriel.txt" "d:\games\Skyrim\Data" --dontFixTangents --removeUnseenFaces --skyblivionTexPath

The first parameter is export file, the second is path to the Data folder. Copy paste this line, replace the path with anything you need and run lodgen manually. You'll need to have BTR files there too though. I can add an option in UI to specify a custom Data path.

I don't think that probem of unpacking files is that big like you say. LODGen doesn't need textures, doesn't need full meshes (unless used in lod), only lod meshes that are used in LOD4, LOD8 and LOD16 are required, and they usually reside in the separate meshes\lod folder for Skyrim and don't conflict with anything.

[zilav]

Uploaded LODGen script with Data path setting.

"Unpack landscape LOD" and "Warn me to unpack..." both unpack files there, and specified data path will be used by LODGen.

Get the latest version from tes5edit thread.

[Sheson]

Hi I spent some quality time with this awesome tool and have this feedback:

 

The vanilla game only has vertex color in the lod level 4. level 8 and 16 have no vertex colors. It would be nice if this would be a setting as well, so only one run is needed.

 

At lod levels 8 and 16 the SLSF2_HD_LOD_Objects shader flag on mountains causes some serious full snow cover. It seems this should be set to SLSF2_LOD_Object to replicate the look of the vanilla game.

 

You may already know this  Thx for another great tool.

 

[zilav]

I'd like to replicate how CK does it, not how it looks the best in this particular case. The reason it obvious - what looks good in Skyrim can look worse in custom worldspace with new meshes. And this tool was built for custom worldspaces in the first place.

If you can confirm that CK doesn't use HD_LOD flag in lod8 and lod16 the same as not using vertex colors there, then this behaviour can be replicated. But programmer responsible for mesh building is missing now, so not sure when any fixes to lodgen might happen.

[Sheson]

I double checked. I created a bunch of 4/8/16 level tiles with CK for Tamriel and Solstheim in various places.

 

The CK HD-LOD texture flag only seems to affect lod4. If it is set then lod4 has vertex colors (only if source nif has it of course) and the HD_LOD_Objects flag for these objects.

 

lod6 and lod16 seems to always ignores the CK HD-LOD texture flag, so no vertex colors and just LOD_Objects flag.

I also checked a couple more vanilla worlds BTO files with nifskope, same thing.

 

I would make it an option to LODGen

[zilav]

Thanks, please keep any information coming. The more you discover, the better LODGen will become in the end hopefully

[Sheson]

It seems LODGen ignores rotation in nif files.

 

 

for example:

 

 

meshes\lod\solitude\spatiowall02_lod.nif

meshes\lod\solitude\spatiowall03_lod.nif

meshes\lod\solitude\spatiowallsteps01_lod.nif etc

 

Some walls of the Thalmor Embassy have the wrong orientation.

 

 

compare Tamriel.4.-20.24.bto with vanilla

[zilav]

I think CK also ignores some of them considering you need to apply translation and rotation in nifscope on vanilla lod meshes before building lod. So try to apply rotation here too.

[Sheson]

Maybe it is the other way around and it should ignore rotation, I don't know

 

Check the image. I used the same untouched vanilla lod nif files for CK and LODGEN.

 

Otherwise I have tremendous success creating LODs with new objects nifs for rocks and and nordic buildings that are missing from vanilla.

[Arthmoor]

Heh, that rotation thing looks like a bug Elminster had with lodgen for Oblivion once. The result of forgetting to convert degrees into radians or something like that.

[zilav]

Maybe it is the other way around and it should ignore rotation, I don't know

Me too, that's why just rotate it in nifscope until works

As I already mentioned, the author of lodgen is not available now, so nothing I can do about it.

[Prometheus]

WARNING!!!Due to a lack of texture atlas generated LOD files are bigger than the vanilla LOD and requires more than average PC. No problem for a smaller worldspaces, but Tamriel worldspace in Skyrim is very packed with LOD, so generated LOD won't work for everyone resulting in infinite loading screens and/or crashes.The only way to optimize it is to create RAEVWD mod for Skyrim that will not only add new lod objects, but change all existing vanilla lod models to use the same atlas texture. A manual made atlas approach instead of automatic atlas generation that CK does. And that would be a lot of work to reUV lod.

hi

Firstof all. Thanks for auch a great tool. .

Second I have some wuestions on lod making and in particular ur last sentence.

I wantto make lods for my large distant object but those are like mountains, so far when I gen them in ck it always getthe mountainslab texture , the gray one of the mountains , how can I do to have my mountains not only have the skyrim tonss but be able to use also custom mountains with different textures? Do I need to create a new custom texture for them but how to make them be redirected so that ck recognize it ?

I did some ice mountains retexturing the normal skyrim ones , but using them together results only in gray mountains lods and not a mix.

Also whats the difference between the mountain lods and the normal lods ?

Also I see that there are 4 level of lods normally for obhects , in my usual lod creation I redid the mesh and used parts of the split large texture for lods picking parts of it to be used to texture parts of the lods in a way that somtiling and alpha is not allowed...

Not a problem but a constrain.

Though I always limited me to reuse the vanilla textures , can I create new large texture clunchs to be used endependantly by lods made by me as well as for mountains too?

[alt3rn1ty]

Prometheus I do not know if this will be a help for you ( maybe partial help ), because I have not studied skyrim LODs much

 

But on the textures : I notice most mountain replacers out there usually just use the two mountainslab textures, and associated normal maps

 

 

There are more - Have a look at Gamwich's Mountain or Molehill

 

He has gone to the trouble of also doing matching LOD textures

 

The file path is [C]Windows > Programs Files or Program Files (x86) > Steam > SteamApps > Common > Skyrim > Data > Textures > Landscape > Mountains

 

Uninstall: Deactivate with NMM, or just manually delete (or overwrite) the mountainslab01_n.dds, mountainslab01.dds, mountainslab02_n.dds, and mountainslab02.dds files.

 

For the mountain LOD files, the file path is The file path is [C]Windows > Programs Files or Program Files (x86) > Steam > SteamApps > Common > Skyrim > Data > Textures > Lod

 

Uninstall: Deactivate with NMM, or just manually delete (or overwrite) the mountainpeak01lod_n.dds, and mountainpeak01lod.dds, mountainslablod_n.dds, mountainslablod.dds, mountaintrimalphalod_n.dds, mountaintrimalphalod.dds, mtncliff01lod.dds, mtncliff02lod.dds, mtncliff03lod.dds, mtncliff04lod.dds, mtncliffsm01lod.dds, mtnpeak02lod.dds, mtnridge01lod.dds, mtnridge02lod.dds, mtnridge03lod.dds, and mtnrtrimslabod.dds files.

 

Enjoy!

Gamwich

[Prometheus]

Thats just replacing the two mountainslab textures used by lods to mach his retexture .

Simple as it is not the issue I refer. To.

I wanted to be able to use more mountain slab textures so that I can have in the same worldspace mountains of ice and rock at the same time perhps by creating a custom mountainslab3 , mountainslab4 that the ck could pick when generating lods .."

[zilav]

I wanted to be able to use more mountain slab textures so that I can have in the same worldspace mountains of ice and rock at the same time perhps by creating a custom mountainslab3 , mountainslab4 that the ck could pick when generating lods .."

First of all, that mountain slab texture you see everywhere is called HD LOD texture. Any worldspace can have a single HD LOD texture only and it is defined in worldspace properties. Then any static object in this worldspace that have HD LOD flag set will use it instead of texture referenced in LOD mesh. Beth used this trick to make mountais look cool because Skyrim is surronded with them, but unfortunately it is usually not enough for custom worldspaces that don't have the same rocks eveywhere like in your case.

So if using CK LOD generator, uncheck HD LOD flag on your ice rocks so they will use textures in LOD meshes and not mountain slab. Or redirect worldspace properties to your own HD LOD texture if you don't need that slab and want to replace it.

[Prometheus]

First of all, that mountain slab texture you see everywhere is called HD LOD texture. Any worldspace can have a single HD LOD texture only and it is defined in worldspace properties. Then any static object in this worldspace that have HD LOD flag set will use it instead of texture referenced in LOD mesh. Beth used this trick to make mountais look cool because Skyrim is surronded with them, but unfortunately it is usually not enough for custom worldspaces that don't have the same rocks eveywhere like in your case.

So if using CK LOD generator, uncheck HD LOD flag on your ice rocks so they will use textures in LOD meshes and not mountain slab. Or redirect worldspace properties to your own HD LOD texture if you don't need that slab and want to replace it.

as I undesrstand the hd lod texture mountainslab 1 and 2 are the only two standard options for distant mountains , so if I want to have my own hd lod dustant tecturetogether with the mountains ? How to do ? Bc the standard lod textures have a size limit.

[zilav]

There can be only one HD LOD texture per worldspace, you either use yours and set it in worldspace properties, or use vanilla mountain slab but your LOD textures will be only 256x256. This is how CK lodgen works.

However TES5LODGen doesn't have any limitations and can use any number of textures of any resolution for lod objects, but it is still in beta and has some visual glitches.

[Sheson]

LODGen.pas should skip everything that has an enable parent, because it is either disabled at start or maybe later in the game.

This is only a minor problem with standard LOD, but becomes increasingly apparent when creating LOD for imperial tents for example.

 

I propose to add this line somewhere around line #162

 

if Assigned(ElementByPath(e,'XESP\Reference')) then
  Exit;

[zilav]

Thanks, better to use ElementExists() here though.