Jump to content

[WIP] FO4LODGen now included in xEdit


Sheson

Recommended Posts

FO4LODGen by Ehamloptiran, Zilav and Sheson
Static objects LOD generator tool for Fallout 4

FO4LODGen is now included in the latest xEdit.exe builds posted to 

However, the current LOD development happens at over at xLODGen - beta with terrain LOD.

To generate static object LOD, either rename xEdit.exe to FO4LODGen.exe or start as FO4Edit.exe (xEdit.exe -fo4) and right click a plugin that adds a worldspace and select Other, Generate LOD.

Use the -o:"c:\Output" command line parameter to select another output folder instead of the games Data folder. This will make it easier to create BA2 archives, to avoid blurry textures because of loose texture files.

For better results, download FO4LODGen Resources from https://mega.nz/#!NQYg3YoR!IJI0-K2Id94LnScyb2RAW5hcBl0LEVSeXdBb71HlKpQ and install FO4LODGen.esp and FO4LODGen - Main.ba2

The FO4LODGen Resources archive includes a doc folder with a small readme.txt and example screenshots that explains what each plugin and the updated meshes/materials do.

Hover over the options of the FO4LODGen window to see hints what each setting does. If a texture atlas is not created it will use the vanilla texture atlas (Commonwealth and SanctuaryHillsWorld are supported atm) if possible and single LOD textures if not.

________________________________________________________________
 
 
post-2652-0-51723500-1454160531_thumb.jpg post-2652-0-34652500-1454160521_thumb.jpg
 
_________________________________________________________________
 
New features for FO4
no 2D tree LOD - trees are done as object LOD
supports all 4 LOD levels  4, 8, 16 and 32
Material swap
LOD for references with an enable parent can be enabled/disabled
MultiRefLOD - replaces LOD for a group of reference by a single LOD mesh in (higher) LOD levels
LOD support for SCOL static collection
 
_________________________________________________________________
 
Static LOD Levels STAT
Static records always had 4 LOD levels but only the first 3 levels were used. Now level 32 bto files actually exist. Export and LODGen.exe have been updated accordingly
 
Material Swap MSWP
A reference or base record can link to a MSWP record which is a list of base and swap material *.BGSM file names. Textures definitions and additional shader flags in the material files take precedence of any data/material file defined in a mesh. Material swap linked from a reference is used over material swap linked from static. The material file swap data is exported and used by LODGen.exe.
If a material swap file is defined its data takes precedence to determine the LOD textures, flags and values.
 
Enable Parents XESP
If a reference has an XESP enable parent marker and its flag bit 8 is set (name currently unknown #8), the form id of the enable parent is used in the bto to enable/disable its object LOD.
LODGen.pas can also exports all enable parent form ids regardless of this flag #8, so all object LOD reacts properly to their parents state.
Further testing how load order changes affect this is required.
 
Linked References with MultiRefLOD KYWD
References linking to a reference marker with the keyword "MultiRefLOD" KYWD:00195411 are groups of objects that have some of their LOD levels replaced by the LOD level data of the base record of the marker.
Typically the LOD level 4 is based on the object LOD of each individual reference of the group, while higher LOD levels are replaced by a single LOD object representing the entire group.
 
Static Collections SCOL
Static collections are groups of arranged statics. Statics that have LOD defined work just the same as if the static was used by a reference. LODGen.pas exports the additional positional data, while LODGen.exe handles the correct transformation and rotation in 3D space.
 
_________________________________________________________________
 
 
Vanilla object LOD
Vanilla static object LOD uses only a very limited set of available options like shader flags. In fact there is only two types of LOD, non-transparent LOD and transparent LOD.
LOD only supports binary transparency, the threshold is defined as 128
FO4 doesn't seem to support snow/ash LOD shaders like TESV
FO4 vanilla LOD does not use vertex colors like TESV

_________________________________________________________________
 
 
LODGen.exe object LOD
The game engine supports some shader flags or values that are not used in vanilla LOD, allowing for additional features. The flags/values are either already set in the LOD source NIF/BGSM or require updated NIF/BGSM. Data in BGSM takes precedence over data in nifs. So setting a flag or value in a single BGSM changes the behaivvior of all NIFs using that BGSM. The use of these additional features is optional.
 
Read older nif formats from other games while exporting BTO in the newer format
Use DoubleSided flag - improved LOD for trees and other transparent objects.
Use Backlight Power - improved LOD for trees
Use Alpha Threshold - set alpha threshold to value of source LOD - useful if LOD textures has true alpha channel
Keep entire source LOD shader 1:1 - Passthru. Obviously this can cause game to CTD if using unsupported setting

 

Link to comment
Share on other sites

  • 2 weeks later...

I don't really have time to look right now, but when generating the BTO files are you generating them as a BSSubIndexTriShape segmented into a grid?  It's generally 4x4 but it's slightly more complicated than that.  Smaller BTOs that aren't square or a full cell in size usually seem to be less than 4x4, and sometimes there is only 1 segment regardless of size.  The only ones that seem consistently broken into a grid are the '4' levels...  8, 16, and 32 are mostly 1x1 but I have found a few '8' that are split into a grid.

 

It looks something like this

 

rxlR6Ji.gif

Link to comment
Share on other sites

Wow, looks very promising. Hope for full support for Skyrim :-)

Uhm TES5LODGen and then DynDOLOD...

 

I don't really have time to look right now, but when generating the BTO files are you generating them as a BSSubIndexTriShape segmented into a grid?  It's generally 4x4 but it's slightly more complicated than that.  Smaller BTOs that aren't square or a full cell in size usually seem to be less than 4x4, and sometimes there is only 1 segment regardless of size.  The only ones that seem consistently broken into a grid are the '4' levels...  8, 16, and 32 are mostly 1x1 but I have found a few '8' that are split into a grid.

 

It looks something like this

 

 

Yes this already produces correct and fully working LOD with all the finer details like quad segments etc. These segments already exist in older games as BSSegmentedTriShape , so it knows which triangles to unload when the cell is attached.

In fact it already produces improved LOD over the the vanilla game just by using doubleside and backlighting flags. In addition it can make use of unused LOD meshes shipping with the game and use full meshes for LOD - see optional esp.

Link to comment
Share on other sites

  • 7 months later...

I am trying to generate LOD4 using the full models with dev snapshot 0.07.  I keep running into errors with missing bgsm files.  Is there any way to work around this or to easily generate the missing files?  I don't yet understand how the materials files play into everything.

 

Would it be possible to add an option to continue with the next BTO when LODGen is unable to generate a particular one?  That way the majority of the batch would hopefully be created successfully, and the problem ones could then be tackled individually.

 

I am also frequently running into memory errors.  The memory usage of LODGen when these occur is between 1.7 and 1.8 GB.  This may be related to the missing bgsms somehow though.

 

Does LODGen generate the LODs in any particular order?  It doesn't seem to, but maybe I am just not seeing the pattern.

 

At this point, FO4LODGen is only dealing with the meshes and not the textures, correct?  So even if I generate higher detail meshes, they'll still be skinned with the bland vanilla LOD textures?

 

I really look forward to DynDOLOD for Fallout 4, but don't mind playing around with LODGen directly until it is released.

 

If there is a better forum to ask these questions, please let me know.  Thanks.

Link to comment
Share on other sites

I was able to work around the memory problems I was having by creating a batch file that ran LODGen individually for each LOD4 quad.  There were still three quads that failed due to missing bgsm files.  There were also 17 quads that reduced to 0 triangles.  I'm assuming those are quads that are water with no objects above the surface.

 

For the missing bgsm files, one turned out to be misspelled.  Making a copy of the file with the correct name allowed that quad to generate.  The other two quads with missing bgsm files had data paths to several files incorrect.  I had to make copies of several files from data\materials to data\data\materials.  Are the pointers to the materials files in the nifs, or in the esm?  I'd like to report these to Bethesda and to the unofficial patch folks, but want to make sure I'm specific in where the problems lie.

 

It will be interesting to see what these look like in-game.

 

For this iteration, I didn't check to Use Backlight Power or either of the alpha options.  Do those mainly just affect trees?  Should I check all three for best detail, or will some of them potentially cause problems?  Does it even matter when using full models?  I plan to run another iteration with those three checked and see what the difference is in-game.

 

Thanks in advance for any replies to this or my previous post.

Link to comment
Share on other sites

I haven't touched FO in a couple month, so I am answering from memory, so please forgive meif I may actually get something wrong :)

 

Yes this is for "vanilla" LOD generation only. None of the fancy things from DynDOLOD. No promise that it will ever happen, but it will need more SKSE functions to become available and a Steam sale for entire FO4+DLC...

 

There soon will be a x64 version of LODGen.exe (you should be able to copy it from DynDOLOD release and just use it - if using newer version from DynDOLOD does not work, let me know). Maybe it will help with the memory errors.

 

Looks like you already found the the lod level and --x and --y settings to create specific quad file.

 

This currently does nothing with textures. It will either use the texture atlas or the single texture if it isn't in the atlas.

 

The order it generates the quads per lod level is roughly defined by the order of objects and their x, y location in the export file I think. Since it is multithreaded though, there is no real repeatable order of things.

 

The bgsm files are referenced from both through esm or nif. In the nif  the bgsm file path/file is in the name of the shader if I remember correctly. bgsm from the esm should take precedence if I remember correctly.

 

I haven't tested with the latest updates of FO4 since earlier this year. Is the missing bgsm happening with file from the DLCs or the vanilla game? It sure was fine back in Feb/March.

 

Backlight power flag is set only on the included bgsm for trees, yes. But you could always set on whatever you like yourself.

 

The benefit of using alpha threshold is also only really seen on trees as most other LOD models for buildings do not have alpha or do not have such thin details as the branches.

 

You just have to test yourself what works best for you.

Link to comment
Share on other sites

Thank you Sheson for your reply.

 

The missing bgsm files turned out to be causing the memory errors.  Once I fixed those, LODGen ran through without any memory errors.  I fixed them by making copies of the files to where they were referenced to be at (one was a misnamed file, the others had an extra data/ in their path), not by updating the nifs or esm though.  I'm not familiar enough with the CK or FO4Edit to fix them properly.  I'll list the problem files when I'm in front of my gaming PC again.  I was running LODGen against the base game, latest patch (1.7.15, I think), no DLC, and no mods.  It's possible the bgsm issues may have been found and fixed in the unofficial patch though.  I didn't try with that enabled.

 

Using full models for LOD4 made almost no visible difference in game that I could see, despite the files being much larger.  I tried from a few different vantage points where I could see long distances.  I expected there to be a small, but noticeable change, but no such luck.  This was with ultra settings, but I also tried shorter LOD distances.

 

I'm guessing that a higher resolution texture atlas would probably make more of a difference.  Manually creating that and pointing things to the correct locations in it are beyond my current skill set though.  Can anyone point me to information on how the texture atlases works?  Like, if you increase the texture size, do any pointers (in the meshes, I assume) have to be updated?  Or are the pointers relative to the size of the texture?  In other words, do the pointers point to exact coordinates in the texture atlas, or do they automatically scale if the size of the texture is changed?  I'm guessing that at least some of them automatically scale since there are higher resolution skins for the vanilla body that don't require updated meshes.

 

I wish there was some way to tell exactly where the LOD levels change in game, like to have the engine draw a red line across the ground or something.  Hmmmm... Is there any way to generate BTOs with completely flat geometry or something like that?  Those could then be used to see where the boundaries of a particular LOD level were in game.  I could probably somewhat figure out the boundaries by fiddling with the LOD distance settings in the INI I guess.

 

I couldn't see any difference in game in the Commonwealth worldspace from enabling the backlight power option.  The alpha threshold option definitely made a difference though - it made all of the trees look like upside-down feather dusters. :-)  Also, using LOD4 generated from full models and using the backlight power option and both alpha options causes the game to CTD when loading a save.  I haven't determined which of the options is the culprit yet.

Link to comment
Share on other sites

Thank you Sheson for your reply.

 

The missing bgsm files turned out to be causing the memory errors.  Once I fixed those, LODGen ran through without any memory errors.  I fixed them by making copies of the files to where they were referenced to be at (one was a misnamed file, the others had an extra data/ in their path), not by updating the nifs or esm though.  I'm not familiar enough with the CK or FO4Edit to fix them properly.  I'll list the problem files when I'm in front of my gaming PC again.  I was running LODGen against the base game, latest patch (1.7.15, I think), no DLC, and no mods.  It's possible the bgsm issues may have been found and fixed in the unofficial patch though.  I didn't try with that enabled.

 

Using full models for LOD4 made almost no visible difference in game that I could see, despite the files being much larger.  I tried from a few different vantage points where I could see long distances.  I expected there to be a small, but noticeable change, but no such luck.  This was with ultra settings, but I also tried shorter LOD distances.

 

I'm guessing that a higher resolution texture atlas would probably make more of a difference.  Manually creating that and pointing things to the correct locations in it are beyond my current skill set though.  Can anyone point me to information on how the texture atlases works?  Like, if you increase the texture size, do any pointers (in the meshes, I assume) have to be updated?  Or are the pointers relative to the size of the texture?  In other words, do the pointers point to exact coordinates in the texture atlas, or do they automatically scale if the size of the texture is changed?  I'm guessing that at least some of them automatically scale since there are higher resolution skins for the vanilla body that don't require updated meshes.

 

I wish there was some way to tell exactly where the LOD levels change in game, like to have the engine draw a red line across the ground or something.  Hmmmm... Is there any way to generate BTOs with completely flat geometry or something like that?  Those could then be used to see where the boundaries of a particular LOD level were in game.  I could probably somewhat figure out the boundaries by fiddling with the LOD distance settings in the INI I guess.

 

I couldn't see any difference in game in the Commonwealth worldspace from enabling the backlight power option.  The alpha threshold option definitely made a difference though - it made all of the trees look like upside-down feather dusters. :-)  Also, using LOD4 generated from full models and using the backlight power option and both alpha options causes the game to CTD when loading a save.  I haven't determined which of the options is the culprit yet.

 

The UV coordinates for the texture atlas are relative (between 0 and 1). As long as locations and relative dimensions are same the same it will work.

 

I suggest to use the terrain LOD for doing flat terrain surface. If you know nifskope in-and-out it is rather easy. Reduce number of vertices etc to 4, then have 2 triangles. The 4 corners for each quad are x,y 0,0 and 4096,4096 then use scale of 4/8/16/32 for the levels. z should be higher than default water height for the world. Use 4 different color textures for each LOD level. Then create a batch to copy from the 4 source files to the true btr files names. I think I used a dir output of the terrain LOD folder to create the list of destination filenames and inserted the copy command in front of etc. I did all this once for Tamriel. Since FO4 uses a different nif format those files can not be used.

I did it for Tamriel once. Since FO4 uses a different nif format those btr won't work and would have to be redone.

 

The backlight option is very obvious with the pre-war trees. Check the screenshot in first post.

 

If you use lots of full models for LOD you can quickly run out of memory or into other limits I suppose. All depends on what models you do this with and how often they are used in the world. It would be always better to at least check the full model in nifksope and remove everything (like animations) but the static model/shader and use as a "LOD" model.

Link to comment
Share on other sites

  • 2 months later...

Sorry if I'm being dumb, but I don't seem to be able to get this to work. I have it renamed it opens as "FO4LODGen 3.2" not xEdit, good, but it just does the Background Loaders, finishes and does nothing else. Picture attached.

Never had any problems with FNV or Skyrim, but I'm having trouble with this one. Any idea what I'm doing wrong? Thanks!

lodgen.png

Link to comment
Share on other sites

Please post the full log, especially the part where it states what BA2's are being loaded. Looks like it couldn't locate any suitable worldspaces by their lodsetting files which are stored in archives.

Link to comment
Share on other sites

9 hours ago, zilav said:

Please post the full log, especially the part where it states what BA2's are being loaded. Looks like it couldn't locate any suitable worldspaces by their lodsetting files which are stored in archives.

This? https://pastebin.com/bNsqc7dY -- Sorry didn't know that was needed. I should also state I'm running it through MO2.

I shouldn't need mods to add LODs though, should I? This can be used to even enhance to base game, yeah? I have about 50 esps (not counting Sim Settlements AddOn packs.) But it's possible I have none that add things to the world.

Link to comment
Share on other sites

  • 2 weeks later...

I managed to generate a Commopnwealth LOD for Fallout 4 for my mod which places a few trees in the world, but each time i try to load my save the game crashes to desktop. Any idea what im doing wrong?

The LOD meshes are looking fine in NifSkope, but the game seems not to like them ...

Link to comment
Share on other sites

We can help with LOD options but Zilav and myself don't use MO2 and normally it's simply not possible to support it since we don't use it. I don't know if Sheson does. For now it's Beta so share, but you may also want to see what people do for MO2 on the STEP forums.  You can't really ask us is that okay do I need to configure this or add this because we have no idea what setup options to suggest for MO2.

Link to comment
Share on other sites

@drmabuse1981

Looks great, I love it when things fix themselves. 

In case anyone runs into game crashes with generated LOD in the future, see if temporarily hiding the generated *.BTO files changes anything and if that is the case see if you can narrow it down to a single/fhandul BTO (area of the mod changes) and then provide them for troubleshooting.

 

@SimonPDX

I don't use MO2 with FO4. Instead of renaming to FO4LODGen, start normal as FO4Edit.exe and right clicking on master plugin, Other, Generate LOD should work I suppose.

Link to comment
Share on other sites

Actually i have a crash when fast-traveling to diamond city, disabling the BTO files seems to fix it, but i will hopefully get that fixed too.

 

 

Link to comment
Share on other sites

  • 2 weeks later...

This may be an odd question, but how do I undo what FO4LODGEN has done?  Removing the files in Data\Textures\Terrain\Commonwealth\Objects causes all distant textures to be purple.  I'm not sure why it doesn't just default back to the files in the vanilla .ba2 archives, but it isn't.  As far as I know, the base game should have LOD textures located in it's textures.ba2 archives.

 

Am I missing something here?

 

 

EDIT:  I just extracted the Fallout4 - Textures.ba2 files one by one and the Commonwealth.Objects.DDS files are still there in Textures4.ba2.  So why is it acting like there are no textures?

 

EDIT2:  I figured it out.  I had to remove the meshes as well from \Data\Textures\Terrain\Commonwealth.  Is there anything else I need to remove, or were the textures and meshes it?

Link to comment
Share on other sites

@Herdo Best approach is to use the Output path, and set it to somewhere else on your HD.

I dont know if you are familiar with command line, but if you open a Command Prompt in windows (google search it), and type in an executables name, you can add any valid switches / parameters an exe will accept. So ..

> "D:\path\to\Lodgen.exe" -o:D:\NewFolderForFiles

Or you can right click LodGen.exe Icon, choose Properties, and set the Output path in the Target box

Here's a bit of help on the above, go to the following guide for DynDOLOD .. 

.. Then scroll down to the section :

5. Output Paths and Wrye Bash App Icons

 

Then once you have the Output path set, all files generated will go to the folder designated as output.

Pack the resulting meshes and textures folders into a zip.

Use whatever mod manager to install and uninstall them as required.

= A lot less messy undoing things :) 

Link to comment
Share on other sites

  • 1 month later...

Hi, I am using version 3da194b and I can't generate meshes at all (Texture generating is fine). I got the following errors and LODGENCMD.exe stopped working:

System.IO.EndOfStreamException: Unable to read beyond the end of the stream
   at System.IO.BinaryReader.FillBuffer(Int32 numBytes)
   at System.IO.BinaryReader.ReadUInt16()
   at LODGenerator.Common.BSAArchive..ctor(String archivepath, LogFile theLog, Boolean verbose)
   at LODGenerator.Common.BSAArchive.Load(List`1 files, LogFile theLog, Boolean verbose)
   at LODGeneratorCMD.Program.Main(String[] args)
[Commonwealth] Objects LOD generation error: LODGen process error, exit code E0434352
[SanctuaryHillsWorld] Objects LOD generation error: LODGen process error, exit code E0434352
[DLC03FarHarbor] Objects LOD generation error: LODGen process error, exit code E0434352
[NukaWorld] Objects LOD generation error: LODGen process error, exit code E0434352

The LODGen_log.txt is empty and these are copied from the program interface.

i didn't change the settings of the tool (object lod, etc).

I have run the exe as admin and have set the permission of game files to everyone. I have also checked my texture mods and all the ones that have nif files only change nifs from dlcs, but I get the errors when generating all nif lod files, main game and dlcs.

thanks in advance for any help.


   

Link to comment
Share on other sites

@Objir please post to pastebin or zip and upload to a file service both the FO4Edit_log.txt and export file Edit Scripts\LODGen.txt.

Best to delete both files first and then generate LOD for a single worldspace, doesn't matter which as long as the error happens.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...