Jump to content

[RELz] LOOT - Load Order Optimisation Tool


WrinklyNinja

Recommended Posts

LOOT

A plugin load order optimiser for TES IV: Oblivion, TES V: Skyrim, TES V: Skyrim Special Edition, Fallout 3, Fallout: New Vegas and Fallout 4.

Introduction

Setting the right load order for your mods is a crucial step to enjoying a stable modded game. The Load Order Optimisation Tool (LOOT) can help with that, by providing automated load order sorting that's simple to use and fully customisable.

While sorting, LOOT checks for load order errors such as incompatibilities and missing requirements, and notifies you of any issues that it detects. It also provides thousands of plugin-specific messages such as usage notes and bug warnings, to help keep your game healthy.

Downloads

LOOT can be downloaded from its website. Unstable snapshot builds may also be available on JFrog.

See the documentation for more information.

LOOT's previous official thread was on the Bethesda.net forums here.

Link to comment
Share on other sites

Welcome Wrinkly. LOOT has gone through a few new updates since I last posted on the topic .. But I still have nothing to moan about recently. I now have it managing Steam FO4, Steam SSE and GOG Oblivion without any issues.

00.5.png

Link to comment
Share on other sites

Here's a "State of LOOT" update because I haven't posted much about it in a while:

Since LOOT's last release (v0.11.0, in May), I've fixed a few bugs in the LOOT API:

  • A crash-on-sort due to corrupt plugins
  • Incompatibility with junction links
  • Masterlist updating failing when working with divergent histories (which would affect people upgrading from very old masterlist formats, and developers using their own forks of the masterlists)

I've also made a few updates to LOOT itself:

  • An obligatory CEF update (we're now on Chromium 60)
  • Fixed an unhelpful "Cannot read property of 'text' of undefined" error message that could get displayed when something else went wrong
  • Fixed updating the masterlist causing the messages in the editor to have no content
  • Updated the UI to use Polymer v2 (and they've just announced work on v3 with major changes again, so in hindsight choosing Polymer was totally not worth it).
  • Fixed installing LOOT beside Skyrim and SkyrimSE simultaneously (rather than elsewhere and using the Registry to find them)

Before I make another release, I want to also add support for the new ESL plugin file format which has been introduced for Fallout 4 and will be added to Skyrim SE at some point in the near future. However, I don't own FO4, so I'm reliant on others feeding me relevant information on the format and related changes to the load order system.

If you know something, the tracking issue is #816 on GitHub, so posting there would be helpful (or here is also fine). Example plugins that I can test against would also be useful (I keep some in a repository here). Also, if anyone's feeling particularly generous, they could gift me Fallout 4 on Steam - that's how I was able to add and maintain support for the other Fallout games (which I've never played past opening the launcher :P).

Other than that, I've been enjoying playing with Rust (the language, not the game), and rewriting LOOT's supporting libespm and libloadorder libraries in it (because apparently I have too much free time or something), so the next version of LOOT may ship with their rewrites if they're ready. It might seem a little pointless other than as a learning exercise (which is mostly the point) but the crash-on-sort bug I mentioned above would be impossible in Rust. I'm not planning on doing a wholesale rewrite of LOOT in Rust though, because the LOOT API has a large surface area that would be very painful to reimplement in a way that's callable from C++ (because for interop Rust provides C compatibility, but C APIs are horrible). Also because I need to have better things to do with my time...

To throw in some numbers, the Nexus page gets ~ 23,000 downloads/month, and downloads from GitHub are ~ 54,000 unique downloads/month (IDK if Nexus download stats are for unique downloads). This is weirdly stable, there's not a lot of variation over time, but I guess there's a large user base that updates irregularly. Surely people can't still be buying Skyrim?

Anyway, I think that's just about everything I can think of. As always, thanks to everyone who contributes and/or provides feedback, and especially Freso, pStyl3 and Sharlikran for various things but particularly taking care of the Nexus page and answering questions there.

2 hours ago, alt3rn1ty said:

Welcome Wrinkly. LOOT has gone through a few new updates since I last posted on the topic .. But I still have nothing to moan about recently. I now have it managing Steam FO4, Steam SSE and GOG Oblivion without any issues.

 

  Reveal hidden contents

 

00.5.png

 

 

Good to hear, and thanks for the welcome. I've got to get used to using a proper forum again...

Link to comment
Share on other sites

Welcome to AFK Mods WrinklyNinja! :horker:

 

16 hours ago, WrinklyNinja said:

Also because I need to have better things to do with my time...

If you have and want to make use of it then perhaps VM at TAL would appreciate some insights about his rewrite of BOSS.  Sorry, no link since TAL is offline at the moment.

Link to comment
Share on other sites

Oh, gee. Finally able to post to the official LOOT thread again! For some reason, the current Bethesda forums' Akismet setup insists that anything I post is spam. :facepalm: Not that I have much to add at this point. I'm looking forward to the ESL support, but I'm currently away from home for 2 weeks, so won't have access to my own FO4 install (or anything else running on Windows), so I can't help much with that. I was planning on ESL'ing a couple of my plugins once the update hit… but the update came around the day before I was travelling, so I was busy packing. C'est la vie.

Link to comment
Share on other sites

7 hours ago, Leonardo said:

Welcome to AFK Mods WrinklyNinja! :horker:

 

If you have and want to make use of it then perhaps VM at TAL would appreciate some insights about his rewrite of BOSS.  Sorry, no link since TAL is offline at the moment.

I've already got the thread bookmarked and I get email summaries of posts there. I've only posted a few times though, and haven't been following VM's development otherwise.

Link to comment
Share on other sites

:) For the next ~44 hours I can test anything you want wrinklyninja, then I'm without a real internet connection once again for some time.

Regarding the Creation Club, apparently everyone who has Fallout installed will automatically download any and all .ba2 archives from all CC-mods, regardless if they where purchased or not. Gopher also talkes about it (see 0:54). Cartogriffi also answers some legal concerns here.

Also, to quote hlp:

As far as I can see, ESL use 2 bytes for load order and 6 for formID, so they don't need to be renumbered. The runtime load order index was increased from 1 byte to two which changed the internal format of TESFile (ModInfo). Also CC content has been added to the list of file automatically loaded when the game start so probably they will load before any mod.

What I wonder is if all ESL share the same most significant byte for load order (which would limit to 256 ESL at once) of if they can use more than one. Anybody tried to load 300 ESL yet ?

Link to comment
Share on other sites

I'm now in possession of five different .esl's. Has anyone ideas for tests regarding load order?

ccbgsfo4016-prey.esl

ccbgsfo4018-gaussrifleprototype.esl

ccbgsfo4038-horsearmor.esl

ccfsvfo4001-modularmilitarybackpack.esl

ccfsvfo4002-midcenturymodern.esl

Link to comment
Share on other sites

7 hours ago, pStyl3 said:

:) For the next ~44 hours I can test anything you want wrinklyninja, then I'm without a real internet connection once again for some time.

Regarding the Creation Club, apparently everyone who has Fallout installed will automatically download any and all .ba2 archives from all CC-mods, regardless if they where purchased or not. Gopher also talkes about it (see 0:54). Cartogriffi also answers some legal concerns here.

Also, to quote hlp:

 

 

Nebulous112 has very generously gifted me a copy of Fallout 4, so I'll be able to investigate things myself now (or once it's downloaded, it's huge!), don't worry about it. Thanks for the info from hlp, that makes things... interesting.

Link to comment
Share on other sites

Ha, glad you made that post, reference the PM and friend invite .. I didn't mention the reason was I was that close to gifting the FO4 Season Pass :)

Link to comment
Share on other sites

4 minutes ago, alt3rn1ty said:

Ha, glad you made that post, reference the PM and friend invite .. I didn't mention the reason was I was that close to gifting the FO4 Season Pass :)

Ha, glad to save you from doing so. I don't have the DLC, but AFAIK they're not necessary for anything I need to do anyway.

Link to comment
Share on other sites

So I had to do a clean install of Windows to fix a different issue with the latest Insider build not installing, and now I've got a weird problem with LOOT.

Specifically I've got the "won't sort" issue for Skyrim SE. Fallout 4 and Oblivion are working fine.

To give an example of what I mean, the DynDOLOD plugin and my Bashed patch are hanging out in the middle of my load order.

I've done the following steps in order:

  1. Removed "%USERNAME%\app data\local\Skyriim Special Edition\"
  2. Removed "%USERNAME%\app data\local\LOOT\"
  3. Set NTFS permissions on D:\games\* to give "Authenticate users, administrators, and my account (which is an administrator but not the administrator), and confirmed that there were no permissions issues on my SSE "Data" directory by using the old NT CMD.exe "copy con" trick to create a new file under my user context (if it shits blood you have a perms problem).
  4. Rebooted.
  5. Logged back in, loaded Steam.
  6. Ran the Skyrim Special Edition Launcher. Confirmed video settings. Launched game to save load. Checked mod list and load order to make sure plugins.txt is written out.
  7. Confirmed a fresh plugins.txt is sitting in my "%USERNAME%\app data\local\Skyrim Special Edition\" folder. I am owner and have full control permissions on it.
  8. Loaded Wrye Bash. All of my plugins were unchecked. BAIN tab shows installs in situ as I left them before OS reload bash .dat files were backed up).
  9. Downloaded a fresh copy of LOOT 0.11 and installed it to "D:\game related\ElderScrollsStuff\LOOT\" (from what I'm reading this makes LOOT dig into the registry to find the games).
  10. Right clicked header on Mods tab in WB, enabled all plugins.
  11. Launched LOOT as Administrator.
  12. LOOT recognizes the plugins and shows all 145 of them, albeit not sorted. All CRCs are matching(I have two monitors; WB is in one, LOOT the other).
  13. Clicked "Sort Plugins. Masterlist revision  b0c79dd with date  2017-08-17 was downloaded.
  14. Applied changes when LOOT completed its sort.
  15. Reviewed load order. DynDOLOD is somewhere in the middle of my load order and my Bashed Patch has a few plugins sorted under it

I've googled about 10 pages in with about as many variations of "LOOT won't sort SSE" and tried several extant work-arounds with no success.

I don't really think this is a LOOT issue necessarily, (LOOT is super-reliable so I think it is a SSE issue TBH) but I'm curious if anyone has had this issue before and what a work-around might be. Thanks!

EDIT: The only thing I didn't do yet was enable debug logging, which I'll do in the AM. I also need to look through tie GitHub issues. Gotta sleep now. :)

 

Link to comment
Share on other sites

Yep I have a debug log now. I've done a bit more testing and isolated it to SSE.

This archive contains a debug log for Skyrim LE (successful), SSE (shows the proper load order getting clobbered by the automatic sort), and then the last one after I:

  1. Double-checked ownership of all of my NTFS stuff, made sure there were no stray SIDs, GUIDS, etc mucking up the works.
  2. Removed the entire SSE profile in My Games.
  3. Removed the SSE folder in App Data
  4. Loaded SSE launcher, let it detect my video settings etc.
  5. Launched SSE, no mods enabled.  No plugins.txt written on exit.
  6. Relaunched SSE, enabled mods one-by-one manually in their mod picker. Plugins.txt written on exit.
  7. Launched LOOT, sorted. even with a completely disorganized load order it sorted the plugins wildly incorrectly, however it did produce the exact same sort result that it did with the pre-sorted list, so it seems to be programmatically doing it.

The only other bit of info I can add is I routinely clean up mods and bash tag religiously. This would change the CRC for some of them, however before the OS reload I never had any issues with getting an acceptable sort result.

This can be replicated with both the Github version and the loot_0.11.0-32-g31af55b_dev.7z  build from Bintray.

https://drive.google.com/open?id=0B8gJeDB67uPfTzNMNWFSeXR2ZG8

Edit: Also nothing is traversing a junction point and I'm not using MO (not enough nopes exist).

Edit 2: I've opened an issue on GitHub in the main LOOT repo since this does not appear to be a metadata issue.

Link to comment
Share on other sites

Just a shot in the dark: are there any mods in the list that LOOT isn't likely to handle? (I sometimes run some of my own which have to be sorted separately).

 Or even reducing the mod list just to test if it will work with a basic mod load? 

Link to comment
Share on other sites

41 minutes ago, lmstearn said:

Just a shot in the dark: are there any mods in the list that LOOT isn't likely to handle? (I sometimes run some of my own which have to be sorted separately).

 Or even reducing the mod list just to test if it will work with a basic mod load? 

That's a really good idea and I'm planning on doing that shortly. I've got enough disk space that I'm going to back everything up and do a sort of "scorched earth" rebuild from scratch to see if I can replicate it.

I do have some odd things in there like Wyrmstooth (one of the first things I ported to SSE for myself) that wouldn't be in the metadata.

Link to comment
Share on other sites

Update on this: I reinstalled completely from scratch and installed a few mods, which got LOOT working again. Now I'm restoring from a Wrye Bash backup to get my mods back in place.  I'll report back when I test with my full SSE load back in place.

EDIT: Negative. After getting my hopes up I restored my Wrye Bash stuff and reinstalled everything it started sorting funky again.

EDIT2: ok now I'm getting somewhere. Despite losing a 40 hour save, I've done a scorched earth rebuild and through 4 hours of manually rebuilding my load order, running LOOT each time I think I've narrowed it down to a single mod that I had installed in the last session before the OS reload. I'm not going to name the mod yet because I don't have definitive proof that it is necessarily the root cause, but it could be a trigger for the behavior.

Link to comment
Share on other sites

I'm having trouble figuring out which log corresponds to what in the archive you uploaded. You've got three API debug logs, one Skyrim, one sse_first_run_not_accepted, and one unlabelled - which of them correspond to the plugins.txt and LOOTDebugLog/CEFDebugLog and loadorder.bak files that you've also included?

I can't tell which "shows the proper load order getting clobbered by the automatic sort" either...

Looking at the logs, I'm guessing that the unlabelled one is the one that has a load order that's most broken, as DynDOLOD.esp is appearing furthest up the load order in it, is that right? The "not accepted" load order looks roughly correct, aside from the DynDOLOD.esp loading about 20 positions too early, is that right? The Skyrim SE masterlist is missing a global_priority: 115 value that the Skyrim masterlist has for DynDOLOD.esp, try adding that as user metadata for Skyrim SE and see if that helps. By "not accepted", does that just mean you didn't apply the load order in the UI, or did something go wrong?

Link to comment
Share on other sites

I haven't investigated everything yet @Beermotor but what Wrinkly is saying looks to be the case. I feel there is a posibility that you have been banging your head on the wall, uninstalling everything, making coffee, eating doughnuts, setting permissions, going to the gym to blow off steam, taking time with the family to take your mind off the frustration that you can't figure out why LOOT is sorting things the way it is, watching Guardians of the Galaxy for the third time, when in reality nothing is wrong and there is no bug.

DynDOLOD.esp isn't in the masterlist with any metadata. When Skyrim SE was released the LOOT team did not decide to just blindly copy and paste the entire oldrim masterlist.  There is no way to know which mods have been converted and which haven't. I looked just now at the new mods section under Newest Today and the mod at the top of the list was 12,000. I don't think I would have time to check that many mods and see which need to be added from the old Skyrim masterlist and ported into the new masterlist. About the only ones I would port though are ones that have after or priority metadata since unlike BOSS, LOOT doesn't need the plugin in the masterlist to sort it.

In the defense of the decision, I have seen many mods where the author added SE somewhere in the name like SMIM-SE [insert variation here].esp which would have made the pre-existing name invalid anyway. If you have seen (I vaguely remember seeding things) a bunch of hullabaloo on the Nexus about LOOT not sorting Skyrim SE correctly, that's probably because nothing is in the masterlist telling LOOT which mods need special sorting.

I added DynDOLOD just now.

I am happy to look at other things if you want to post your oldrim load order (From Wrye Bash) and your Skyrim SE load order (Also using Wrye Bash) and we can compare them. Please remove the [] with xml in it, as those are sorta silly to add. Keep the Spoiler tags though that's important.

Link to comment
Share on other sites

1 hour ago, WrinklyNinja said:

I'm having trouble figuring out which log corresponds to what in the archive you uploaded. You've got three API debug logs, one Skyrim, one sse_first_run_not_accepted, and one unlabelled - which of them correspond to the plugins.txt and LOOTDebugLog/CEFDebugLog and loadorder.bak files that you've also included?

I can't tell which "shows the proper load order getting clobbered by the automatic sort" either...

Looking at the logs, I'm guessing that the unlabelled one is the one that has a load order that's most broken, as DynDOLOD.esp is appearing furthest up the load order in it, is that right? The "not accepted" load order looks roughly correct, aside from the DynDOLOD.esp loading about 20 positions too early, is that right? The Skyrim SE masterlist is missing a global_priority: 115 value that the Skyrim masterlist has for DynDOLOD.esp, try adding that as user metadata for Skyrim SE and see if that helps. By "not accepted", does that just mean you didn't apply the load order in the UI, or did something go wrong?

I apologize, I didn't do a very good job of naming them. I try to be too helpful and forget I'm being cryptic in the process.. The "not accepted" one was here I hit cancel and didn't accept the sort, and the Skyrim one was from a successful Skyrim LE sort.  I had originally intended to put a readme.txt in there but I was half-asleep at 4:00AM. :/ 

Long story short, yes the regular unlabeled one was the most broken with DynDOLOD.esp loading up 20 positions or so too early.

Fortunately I just finished a DynDOLOD rebuild a moment ago and was about to sort it.  Setting the global_priority: 115 in the user metadata nails it to the bottom! Thank you for the heads up on that. :)

So, what vexed me and initiated this report was I never ever had this result or had to adjust where DynDOLOD or DMM were ordered before I reloaded my operating system. LOOT is and has always been super-reliable for me. Perhaps it has something to do with the fact that I've been using the same LOOT install (git data) since last October?

I also wanted to give a heads-up because the Windows build that started all of this is basically a RC for the Fall Creator's Update.  We were afraid it would break something in Wrye Bash and absolutely broke MO, so I felt I would be remiss if I didn't report any kind of unusual behavior after installing it.

Edit: I've closed the issue on GitHub since this sorted it and Sharlikran has updated the SSE masterlist.

Link to comment
Share on other sites

14 minutes ago, Sharlikran said:

I haven't investigated everything yet @Beermotor but what Wrinkly is saying looks to be the case. I feel there is a posibility that you have been banging your head on the wall, uninstalling everything, making coffee, eating doughnuts, going to the gym to blow off steam, taking time with the family to take your mind off the frustration that you can't figure out why LOOT is sorting things the way it is, watching Guardians of the Galaxy for the third time, when in reality nothing is wrong and there is no bug.

DynDOLOD.esp isn't in the masterlist with any metadata. When Skyrim SE was released the LOOT team did not decide to just blindly copy and paste the entire oldrim masterlist.  There is no way to know which mods have been converted and which haven't. I looked just now at the new mods section under Newest Today and the mod at the top of the list was 12,000. I don't think I would have time to check that many mods and see which need to be added from the old Skyrim masterlist and ported into the new masterlist. About the only ones I would port though are ones that have after or priority metadata since unlike BOSS, LOOT doesn't need the plugin in the masterlist to sort it.

In the defense of the decision, I have seem many mods where the author added SE somewhere in the name like SMIM-SE [insert variation here].esp which would have made the pre-existing name invalid anyway. If you have seen (I vaguely remember seeding things) a bunch of hullabaloo on the Nexus about LOOT not sorting Skyrim SE correctly, that's probably because nothing is in the masterlist telling LOOT which mods need special sorting.

I added DynDOLOD just now.

Nah I didn't get too bent out of shape about it.  I had a nice sorted plugins.txt from a Wrye Bash backup that I was using, but my main concern was that the Windows 10 Fall CU had done something to affect LOOT in some way.

So speaking of the masterlist, that makes perfect sense because of not only the porting status, but the CRC of all of these plugins would change after being ported.  I actually have a ton of mods that I've also collected ITM/UDM info on over the past year or so that I've been wanting to submit but hesitated because I didn't feel comfortable editing the metadata directly.  Is there another way to submit these reports (out of xEdit) to the team?

Also thank you for adding DynDOLOD. :)

 

Link to comment
Share on other sites

16 hours ago, Beermotor said:

So speaking of the masterlist, that makes perfect sense because of not only the porting status, but the CRC of all of these plugins would change after being ported.  I actually have a ton of mods that I've also collected ITM/UDM info on over the past year or so that I've been wanting to submit but hesitated because I didn't feel comfortable editing the metadata directly.  Is there another way to submit these reports (out of xEdit) to the team?

I figured you hadn't made that big of a deal, but I was being silly, and having fun with that anyway.

You are going to be so excited when you see this you are going to be bursting with excitement!

WrinklyNinja is the reason why I started working with the modding community. Being given the chance to be part of BOSS got me started. I was just editing a text file at the time but as time went on I was motivated to do more. I realized that placing the mods in the BOSS masterlist was hard because some mods changed more then just weapons and armor or spells. Wrinkly told me without TES5Edit it would be difficult to place them into a better group. Then he said nobody had been in touch with Elminster. That got me thinking because I didn't have a clue as to what TES5Edit was or who Elminster was. I didn't know where to start, but that wasn't going to stop me. Even at first Arthmoor didn't believe what he was reading. Picture it, some guy comes out of nowhere with only a few posts because I had been posting in the BOSS Forum, someone nobody knows at all, and says they are going to make TES5Edit. They are like what, yeah sure I'll believe it when I see it.

I found Elminster, and long story short... here we are.

Link to comment
Share on other sites

8 hours ago, Beermotor said:

So speaking of the masterlist, that makes perfect sense because of not only the porting status, but the CRC of all of these plugins would change after being ported.  I actually have a ton of mods that I've also collected ITM/UDM info on over the past year or so that I've been wanting to submit but hesitated because I didn't feel comfortable editing the metadata directly.  Is there another way to submit these reports (out of xEdit) to the team?

By the way after cleaning plugin in xEdit you can right click on it again and view cleaning statistics in BOSS/LOOT format.

Link to comment
Share on other sites

@Sharlikran  That's pretty cool and my impression of you has always been whenever I saw you were working on something, I was reassured it would be quality stuff. You do good work. 

@zilav thank you, that's how I gathered up all of the reports I had. That's a fantastic feature. :) 

Like mentioned earlier I have quite a bit of data I can submit, I just need to get comfortable with submitting it first. I just forked the SSE Masterlist and I'll dig through the documentation a bit before I make and push any changes. I need to get up-to-speed and I'll probably dip my toes in the water with a few simple entries before I start adding things in earnest.  

Do you guys have any pointers or advice before I get started?  My background with YAML relates to custom compiler directives for Apache builds. 

Link to comment
Share on other sites

3 hours ago, Beermotor said:

Like mentioned earlier I have quite a bit of data I can submit, I just need to get comfortable with submitting it first. I just forked the SSE Masterlist and I'll dig through the documentation a bit before I make and push any changes. I need to get up-to-speed and I'll probably dip my toes in the water with a few simple entries before I start adding things in earnest.  

Do you guys have any pointers or advice before I get started?  My background with YAML relates to custom compiler directives for Apache builds. 

Watch your indentation. Set up LOOT so it doesn't auto update the masterslit that way you can edit the masterlist in the users folder and run loot to test it before making the Git commit. If you commit a mistake to the live server, don't revert it or rebase the branch because LOOT won't be able to sync properly for some reason. (Unless that changed) This won't apply if you are committing to your fork because users pull from the LOOT server not your fork.  Don't worry about making mistakes at first you will get a notification from Travis (Automated system) if there is a syntax error.  Try not to rely on Travis, but we all make mistakes, it happens. I don't know if Travis notifies you when you have the repo forked though. EDIT: Some doesn't apply, I was wondering.

At first maybe stick to entering xEdit info. I'd kinda prefer that if you are going to add priority or after metadata that you ask about it. The reason is if you found 10 or 20 mods, one of them is bound to have an author state his mod need to load after another mod, or load last. Depending on how well he knows his stuff that may not really be true.  It was the first thing that I learned when I was editing the BOSS masterlist. These days when people say that, I say show me the conflict in xEdit first and then we will talk.

An example would be General of the Army and Minuteman Overhaul for Fallout 4. There is a chance someone will come along and say one has to load after the other for some reason.  They don't have a common master so you could add after metadata. However, those mods are by the same author and I maintain one of them, and I know they are incompatible.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...