Jump to content

Wrye Bash - All Games


Utumno

Recommended Posts

Alright I still need an expert's opinion on bsas backslashes - are both forward and backslashes allowed ? Can a file have both ? @Arthmoor can you ping any of the bsa tools devs ?

@CrankyCat  - well turns out there were a couple bugs in ba2 handling - will pack a new version ASAP - fixing those, the patch completes successfully. However remains true that Bash only parses forward slashes ( \ ) - that's not an easy fix (to do it properly that is) and I'd like some expert opinion as I said

@alt3rn1ty - I hadn't missed that post but I needed the files - and their filenames - are they always Fallout4/Skyrim.ccc ? I think I should hardcode the esls anyway as an extra level of security against moving them around

Btw I have removed the exclusion groups in case you have the sticky up

Stay tuned - release soon - I still need the new bsa slashes issue and the saves display of esl masters sorted.

 

EDIT: important - what's the load order of those files in ccc ? Is it as it appears in the ccc files ? Links to docs, expert opinion, C++ hackers or/and oracles appreciated

 

Link to comment
Share on other sites

1 hour ago, Utumno said:

Alright I still need an expert's opinion on bsas backslashes - are both forward and backslashes allowed ? Can a file have both ? @Arthmoor can you ping any of the bsa tools devs ?

@CrankyCat  - well turns out there were a couple bugs in ba2 handling - will pack a new version ASAP - fixing those, the patch completes successfully. However remains true that Bash only parses forward slashes ( \ ) - that's not an easy fix (to do it properly that is) and I'd like some expert opinion as I said

@alt3rn1ty - I hadn't missed that post but I needed the files - and their filenames - are they always Fallout4/Skyrim.ccc ? I think I should hardcode the esls anyway as an extra level of security against moving them around

Btw I have removed the exclusion groups in case you have the sticky up

Stay tuned - release soon - I still need the new bsa slashes issue and the saves display of esl masters sorted.

 

EDIT: important - what's the load order of those files in ccc ? Is it as it appears in the ccc files ? Links to docs, expert opinion, C++ hackers or/and oracles appreciated

 

I would see if we could ping @Jon to see if he knows anything (I just did, didn't I? :P ) or if anyone has his contact information, ianpatt from the Script Extender team might be able to field the question. Worst-case scenario I have my Twitter account set up now and can reach out some Bethesda devs.

Regarding the CCC files, as far as I know the files will always be named "Fallout4.ccc" and "Skyrim.ccc".  The contents of the files and the order in which the CC plugins are listed in the files will change as CC content is released.   I read this somewhere so I'll see if I can get some written documentation.

Link to comment
Share on other sites

As Beermotor says, thats the way I understand it too with the .ccc files, before the latest official update the creation club plugin names were being included in the games exe, which would necessitate an update to the exe every time a new batch of creation club content came along forever more (causing the SKSE team a major headache for one long term, forcing them to always update in response).

Now that the new official update is released, and we have these .ccc files, those logically will be the file that gets periodically updated instead of the games exe. So its content will be evolving as time goes on (which may mean if you hard code things in Wrye Bash, and an update comes along to the .ccc files list, then Wrye Bash will need an update in response - Possibly a bad idea ?).

I believe the Load Order of those is as per the order they are in in the .ccc files (there is nothing else to determine their load order I dont think in a vanilla game setup), same as when they were hard coded in the exe. But really they have just been ordered alphabetically, maybe it does not matter, and all of them have no conflicting records, and alphabetical ordering is optimal for loading.

Although there is this https://www.creationkit.com/fallout4/index.php?title=Data_File

"

  • .esl Files released by Bethesda Game Studios (Creation Club releases) are flagged as master files and always load immediately after the official Bethesda masters, in lexicographical order. These .esl files cannot be deactivated.

"

lexicographical I think is similar to alphabetical, which agrees with how we see them ordered in the .ccc files. I have not seen any other official documentation for any of the above though, its just logical conclusions on my part from what we have seen of official responses to previous CC updates causing the community concerns, this update seems like Bethesdas response to the outcry's.

 

Edit: The Exclusion group notes in the sticky posts was removed before I went on holiday. Beta 2 notes in the sticky now mention that the only thing left todo = Grok ESL ordering. I am also still linking the Utumno WIP Nightly builds, in case anyone out there wants to benefit from the recent Form and Records version 44 fix now included (So with the release being so close, and a fix available for anyone wishing to try it, I still have not put up any notes as mentioned before I went on holiday with regards to form version 44 potentially causing people problems just from using an older bashed patch .. Ravenmind pointed out the spoiler you missed regarding that unanswered question, where also in my following post I was indeed referring to Wrye Bash being your Baby, and not yourself :X).

Very much looking forward to the release, so many issues have been resolved, and taking onboard new games has been amazing.

Link to comment
Share on other sites

Thanks @zilav - can a file have both? this would be a pain

18 hours ago, alt3rn1ty said:

But really they have just been ordered alphabetically,

Well - not in Fallout4.ccc. And given the wiki quote:

18 hours ago, alt3rn1ty said:

lexicographical I think is similar to alphabetical, which agrees with how we see them ordered in the .ccc files

it is, including numbers (oops not quite https://www.quora.com/Whats-the-difference-between-lexicographical-and-alphabetical-order )- but since in the files the order is not lexicographical (at least in fo4) either the wiki is wrong and the correct order is given in the .ccc or the other way round. I need this settled to code the parser for those files.

Thanks for editing the nexus descriptions!

18 hours ago, alt3rn1ty said:

Ravenmind pointed out the spoiler you missed regarding that unanswered question, where also in my following post I was indeed referring to Wrye Bash being your Baby, and not yourself :X).

I did miss that yes - but from now on you have green light from me for any edits you deem necessary, you do know what's going on both in the dev front and the community. Now back to pampering my baby :D (well actually back to RL, will be quite busy for a while)

Link to comment
Share on other sites

3 hours ago, Utumno said:

Well - not in Fallout4.ccc. And given the wiki quote:

it is, including numbers (oops not quite https://www.quora.com/Whats-the-difference-between-lexicographical-and-alphabetical-order )- but since in the files the order is not lexicographical (at least in fo4) either the wiki is wrong and the correct order is given in the .ccc or the other way round. I need this settled to code the parser for those files.

And another opinion from HLP ( AKA Hugues92 ) which backs up what I believe that they will load in the order they are in in the .ccc file ..

There is nothing else could be ordering them

Pretty sure @WrinklyNinja is also going to be assuming the same for LOOT, that the .ccc files are the order in which those plugins will load.

Link to comment
Share on other sites

My daughter had another episode yesterday so I may not comment further.

Bethesda does not release source code or documentation everything is done by observation. Even decoding is done by making a plugin, changing one field in the plugin and then observing the change. All programming and decoding for xEdit is done by observing official tools.

If the official Bethesda tools create BA2 (or whatever BSA files) with either type of slash then account for it. I myself pack both BA2/BSA files using Archive.exe/Archive2.exe from a project folder and not the Data folder.  The only other way I know to make a complete Bethesda Archive is with a Json file from the CK.  You could have cases where the CK added both types of slashes.  You are not going to get Bethesda to change that.  Don't restrict functionality when mod authors are using official tools in acceptable ways and at the behest of Bethesda.  Fighting Bethesda is a fight you won't win.

If you don't have documentation from Bethesda on how they sort ESL files, then make a choice.  When the community provides feedback on what works and what doesn't then update it to work according to how Bethesda is doing things.  ESL files load in FE in memory when you play the game.  We have observed that the files appear to be merged at runtime. They should not be patched since they are handled differently then normal ESP/ESM files at runtime.  Because Bethesda has chosen to handle ESL files that way we have observed then Bethesda doesn't seem to care about the sort order of ESL files. For those reasons Bethesda adds the list of ESL files arbitrarily to a block of file names in the save game.  They are in the order Bethesda has chosen (that appears in the CCC files) which has been observed to follow ESM/ESP files. Sorting ESL files among the other ESM/ESM files goes against what Bethesda is doing. Sorting them last after ESP/ESM files including the Bashed Patch and dynamically created plugins that LOOT/BOSS sorts after the Bashed Patch makes the most sense because it follows what Bethesda is doing.

Currently from my observations of the saves tab in regards to user created and official ESL files is that the current sorting method causes the save game have orange check boxes, period. For my own personal use I would read the Save Game header as Bethesda saved it.  I would sort the official ESL files after the Bash Patch according to the .ccc files and then then user created files alphabetically since that what I see Bethesda doing. I would think the logic for the check boxes would then just fall into place and the boxes would be the color noted in the documentation again.

Both the contents and the file name of the .ccc files were in my post. There is no official documentation anywhere about them.  What people are observing is that everyone has the exact same file names and the same contents. The consensus is that Bethesda will update those files as new Creation Club content is released. Hard coding that list into a 3rd party tool will mean the need to update the tool every time that happens. Bethesda is not going to tell us how frequently they will update those files so most tool authors will be reading the external files because that makes the most sense. 

As far as I know both LOOT and xEdit will be reading the .ccc files. It is also rumored Vortex will do the same.  I can't confirm that myself because I have not used Vortex personally.  Users have been asked not to comment on Vortex for the time being, but I still see conversations about it.

Link to comment
Share on other sites

6 hours ago, Utumno said:

I did miss that yes - but from now on you have green light from me for any edits you deem necessary, you do know what's going on both in the dev front and the community. Now back to pampering my baby :D (well actually back to RL, will be quite busy for a while)

I added a little note just on the Skyrim SE sticky post (not necessary on the others) : The sticky note already includes a link to dropbox, and the following will probably pull in a few more development testers than usual :) ..

 

Skyrim Special Edition users of Wrye Bash 307.beta1

Recommend you use the Utumno-WIP version linked above which has now solved all issues with plugin Headers and Records which were not Version 44. The current consensus is that running SSE with any plugins that are not v44 slowly starts to cause problems with your game, so oldrim plugins not properly converted, AND that includes any Bashed Patch created with previous versions of Wrye Bash. When you have the development version of Wrye Bash installed, go to the mods tab, right click the Bashed Patch and rebuild it. Rebuilding it will correct any Header or Record versions to be correct for this game in the Bashed Patch - Other plugins with incorrect versioning for this game are your lookout. Apologies if this has already caused any issues, the problem could not be anticipated by Wrye Bash development, and it took time to correctly understand and solve it. This issue only applies to Skyrim Special Edition Wrye Bash.

Link to comment
Share on other sites

@Utumno I added a bunch of mirrors to all four Wrye Bash Nexus pages, linking them all plus the two dev topics (here and bethnet) plus the GitHub repository. Nexus pages have a dedicated Mirrors Tab which we haven't taken advantage of since spawning more game support.

 

Also had a few issues popped up on Skyrim SE Nexus comments.

The first from Kkemno, I thought his issue was either missing master or missing game registry settings, but apparently he has no missing masters, but the game behaves as if he has (loads so far to the menu, but then crashes when loading in all resources and mods). I thought this one was similar to Issue #396, but Kkemno has tried the latest dev nightly build and has the same problem - Problem = Game crashes trying to load after building and using a Bashed Patch, but was fine with an older nightly build. I cant come up with anything that could be a cause apart from maybe corrupt other mods being merged, but no Load Order or BugDump has been provided. I cant repro with my setup, but thats not surprising I'm not too adventurous with mods.

Edit 2 : Disregard Kkemno's problem, I think he has just previously used an older installer and has not yet used the sticky post routine for clearing those installer problems .. Manually installing Standalone he has solved his problem.

The second just been posted, from Venutious, is something I have never used. Its also on the Skyrim SE comments but is actually about an Oblivion setup ..

Quote

I can't create dummy masters using version 307 beta 1.  It generates a RenameOp warning "Unable to rename temporary file from "filename xxxx", Unknown details" for a list of files and then crashes out with an IOError 2 as it can't find the esm file in the Oblivion Data directory

I've got round it by running Wrye Bash 304.4.

 

Edit : Sharlikran has started responding to the two of those issues, so he may seduce more helpful info from the two of them with a different approach.

Link to comment
Share on other sites

Gonna throw this one out there as a feeler for now in a future version. How feasible would it be to interface Wrye Bash with the tracking list for a user on Nexus so that it can see when mods get updated and mark them as such somehow in the UI?

Link to comment
Share on other sites

22 minutes ago, Arthmoor said:

Gonna throw this one out there as a feeler for now in a future version. How feasible would it be to interface Wrye Bash with the tracking list for a user on Nexus so that it can see when mods get updated and mark them as such somehow in the UI?

You mean to re-introduce the feature that existed in WB292 (IIRC, right click on an archive and select "Open at" in BAIN) I think.

Link to comment
Share on other sites

On 16/11/2017 at 8:11 PM, Leonardo said:

You mean to re-introduce the feature that existed in WB292 (IIRC, right click on an archive and select "Open at" in BAIN) I think.

Leo Wrye Bash still has that feature ..

pGyIRnJ.png

 

@Arthmoor +1 for the feature request, I have nexus notify me of mods being tracked, and occasionally use the feature Leo mentioned to Open at Nexus as a short cut to getting there, but prefer not to have auto updating ..

@Utumno .. What Arthmoor suggests sounds good, maybe putting a * next to a mod which has proven lower in its version check on loading the Installers Tab?, or maybe not an auto version check, but a manually triggered command. Right click the Installers Tab Header Bar and click "Check all mod versions", then we dont spam Nexus with many Wrye Bash version checking automatically, and bring Nexus to its knees like NMM once did when they first introduced auto updating to their mod manager.

The version check would have to cope with the odd hiccup of different authors ideas of a version number - For example V5.1 becoming FINAL. But those are few and far between, most authors do have a logical incrementing version.

Edit : I had spoilering troubles trying to spoiler the above image without spoilering the rest of the post. Gave up.

Edit 2 : Clarifying who this is for the benefit of :).

Link to comment
Share on other sites

I love that idea Arthmoor! I like to change the filenames to my own format & currently have to append "-######" to the end of my packages to get the Open At feature to go to the correct Nexus location. Not really a big deal, as it also lets me easily know which file# to type into the addr bar if I'm looking for it.  I like the idea of a manual trigger to check for updates. In order to catch any updates that are listed under the update category instead of main files, such as with Better Cities etc., maybe there could be a check to see if a different or new file exists under Main Files, Updates, and Optional Files. Not sure how this could be done if based just on the filename. My OCD won't let me leave an archive named "Main-00000-1-0.zip" Maybe it could check the last updated date/time fields in the Files tab and compare that to the created or modified date of the package. Then showing a possible update or new file exists if the date is later.

Well, this would be a nice feature, but it looks like Utumno's got a lot of other things to do first. Wish he had some help or I knew enough to assist.

Link to comment
Share on other sites

I wasn't asking for it to be immediately implemented, just wondered how feasible it would be to monitor a user's tracking feed on Nexus and inform them that updates are pending. That old "Open At" feature isn't really what I had in mind, but if it could be made to do this that would be fine.

And no, I am definitely not proposing any sort of auto-update. Nor am I even proposing that Wrye Bash be hooked up to the Nexus API to make that possible.

Link to comment
Share on other sites

Just appended a new line to the nexus sticky posts ( trying to keep those short but I keep finding things to note :(

"WIN 10 - WINDOWS PROTECTED YOUR PC - DONT RUN : If you get this message when trying to run the WIP Installer, click MORE INFO instead of dont run, and you will get the button to run it."

When you get this ( especially after running a new dropbox installer from the nightly builds ) its not immediately obvious how to run it and the message puts you off trying to because the way its worded makes you cast aspersions on the software as if Microsoft had good reason to protect you from something, the "more info" is just a slightly different shade of blue in the text of the dialogue message (a bit like some of the privacy options they like to try and hide a bit in this way), clicking it opens up the option to run the exe anyway. Even I take a step back from immediately running it when I see that dialogue, no matter how many times I have told it to go away before it still makes me pause for thought, so someone new to using the WIP may well be put off even trying it.

 

Edit : Ah @Arthmoor sorry my post regarding your suggestion : The only part I meant to be at you was the +1 for the idea. The rest of my post was just ruminating how to go about it mostly for Utumno to cherry pick ideas/considerations if he links the conversation in the future from GitHub.

Link to comment
Share on other sites

I like the asterisk idea. I'm not a compulsive updater due to the horrors I've seen (e.g. the MA who thought it would be a good idea to compact the FormIDs of his plugin) so I don't want my mod manager pestering me to update something that doesn't need updating. A subtle visual indication is good.

Also, to play devil's advocate here: there are tons of bugs with the Nexus API; much like the FOMOD spec it does no sanity checking, so it can be easily defeated and give incorrect results if a mod author incorrectly names or tags their files.  I used NMM once a long time ago just to see how this worked and was annoyed with how it told me that an optional file was perpetually out of date  due to the MA not updating an XML file in the zip.

 

Link to comment
Share on other sites

@Utumno - From the comments on Skyrim SE Nexus Wrye Bash page :

"having Insignificant Object Remover.esp in the merge bash patch its causing infinite start loading screen, i disabled it and its working fine now. Im using the Utumno-WIP version 307.201711041935"

I can confirm there are a couple of issues with this mod, but for me instead of Infinite Load Screen .. The game CTDs

Reproduce - Grab Insignificant Object Remover

Its packed for FOMod and as usual will not work without a bit of rearranging, in the Core folder, then in the Full folder, extract the plugin and install that.

Then rebuild the patch, Wrye Bash will offer to deselect it for merging, let it.

When done, run the game - It CTDs on loading for me.

I also found that after merging, only 3 of the possible 6 records were actually merged, so it was not fully merged as Wrye Bash is indicating.

The three records that were merged, had the Density value correctly carried over, but its a bit of a puzzle as to why the merge is making the game CTD .. The only other thing different being the Record Type has had its version forced from 39 to 44

Maybe this record type needs to remain as was ? I wonder if @Arthmoor has any insight on the problem.

Open spoiler for screenshots :

 

T4IvxoL.png

Gv6FR0K.png

Wrye Bash starting
Using Wrye Bash Version 307.201711041935 (Standalone)
OS info: Windows-10-10.0.15063
Python version: 2.7.12
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: None; output encoding: None; locale: ('en_GB', 'cp1252')
filesystem encoding: mbcs
Using scandir 1.5
bash.pyo  316 _main: Searching for game to manage:
bush.pyo   76 _supportedGames: Detected the following supported games via Windows Registry:
bush.pyo   78 _supportedGames:  Oblivion: e:\oblivion
bush.pyo   78 _supportedGames:  Skyrim Special Edition: D:\Steam\steamapps\common\Skyrim Special Edition
bush.pyo   78 _supportedGames:  Fallout4: D:\Steam\steamapps\common\Fallout 4
bush.pyo  136 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.pyo  142 _detectGames: Set game mode to Skyrim Special Edition found in parent directory of Mopy:  D:\Steam\steamapps\common\Skyrim Special Edition
bush.pyo  156 __setGame:  Using Skyrim Special Edition game: D:\Steam\steamapps\common\Skyrim Special Edition
testing UAC
mods_metadata.pyo  227 __init__: Using LOOT API version: 0.10.1

 

Edit : Weird thing, if I reselect the mods plugin (tick instead of plus), then again rebuild the bashed patch, I can randomly get all 6 records to be merged and the form version for the records is preserved as 39 ... Game still CTDs though even after the original (now fully merged plugin) is deselected again. Bit of a head scratcher this one I think.

Link to comment
Share on other sites

EDIT:  I answered my own question.  After some testing, it appears that the old Valda's fork for Fallout 3’s Wrye Flash's BSA Redirection is broken.  It makes the required ini changes but does not install a dummy bsa file into data directory.  After turning off BSA Redirection in Wrye Flash and installing Archive Invalidation manually, all problems are solved.

 

Wrye Flash for Fallout 3 is kinda old, does anyone know if Wrye Flash's BSA Redirection work properly, or should I be using another method of BSA Redirection instead?

The reason I am asking is I am having a minor archive invalidation/BSA Redirection problem with Wrye Flash.  The only tool I have ever used for archive invalidation/BSA Redirection in all games is Wrye Bash/Wrye Flash, and its never failed me before.

But I was trying to install a body replacer last night and it worked fine for the raider and ghoul races, but not for the normal races.  I am still getting vanilla upperbody textures for the normal races, which looks hideous on a Type 3 or 5 body.  I have checked and double checked (both through Wrye Flash and by checking the data folder directly) to make sure the files were installed into the right folder (textures/characters/female) and that the right textures were installed.  But the game is using the vanilla textures from the Fallout BSA file.

I also checked to make sure the BSA Redirection flag was set in Wrye Flash.

The only way I was able to get the right textures to appear in game was by creating a new folder with a different path name and putting the textures in there and changing the file path the three race records (Caucasian, Asian, African and Hispanic) to point to the new folder instead of textures/Characters/female.  If I do that then I can get the new textures to show up in game.  But I would like to get the BSA Redirection figured out so I don't need such inelegant workarounds.

Any suggestions?

Link to comment
Share on other sites

6 hours ago, alt3rn1ty said:

@Utumno - From the comments on Skyrim SE Nexus Wrye Bash page :

"having Insignificant Object Remover.esp in the merge bash patch its causing infinite start loading screen, i disabled it and its working fine now. Im using the Utumno-WIP version 307.201711041935"

I can confirm there are a couple of issues with this mod, but for me instead of Infinite Load Screen .. The game CTDs

Reproduce - Grab Insignificant Object Remover

Its packed for FOMod and as usual will not work without a bit of rearranging, in the Core folder, then in the Full folder, extract the plugin and install that.

  Reveal hidden contents

T4IvxoL.png

Gv6FR0K.png

Wrye Bash starting
Using Wrye Bash Version 307.201711041935 (Standalone)
OS info: Windows-10-10.0.15063
Python version: 2.7.12
wxPython version: 2.8.12.1 (msw-unicode)
input encoding: None; output encoding: None; locale: ('en_GB', 'cp1252')
filesystem encoding: mbcs
Using scandir 1.5
bash.pyo  316 _main: Searching for game to manage:
bush.pyo   76 _supportedGames: Detected the following supported games via Windows Registry:
bush.pyo   78 _supportedGames:  Oblivion: e:\oblivion
bush.pyo   78 _supportedGames:  Skyrim Special Edition: D:\Steam\steamapps\common\Skyrim Special Edition
bush.pyo   78 _supportedGames:  Fallout4: D:\Steam\steamapps\common\Fallout 4
bush.pyo  136 _detectGames: Detecting games via the -o argument, bash.ini and relative path:
bush.pyo  142 _detectGames: Set game mode to Skyrim Special Edition found in parent directory of Mopy:  D:\Steam\steamapps\common\Skyrim Special Edition
bush.pyo  156 __setGame:  Using Skyrim Special Edition game: D:\Steam\steamapps\common\Skyrim Special Edition
testing UAC
mods_metadata.pyo  227 __init__: Using LOOT API version: 0.10.1

 

So in the interest of making a BCF for this mod, I downloaded, installed, and took a look at it in SSEEdit.

It looks like the MA created the plugin by making a new v44 header plugin, then copied records out of Skyrim.esm into the ESP. As a result, all of the records are form version 39.

Spoiler

IOR-bad-plugin.JPG.3619441033cbae8aa08dde32ea198eff.JPG

If I were a gambling man I'd wager that's the reason his Bashed Patch isn't letting his game load.  This is exactly the kind of plugin @Arthmoor was talking about that messes with people's games.

I would love to create a BCF with fixed versions of the plugins as a community service but ethically I can't. :cry:

EDIT: Simply opening the plugin in the CK and resaving it fixes the form version issue. I don't know why this is so hard for people.

Link to comment
Share on other sites

I offered him a simple restructure so that sub-packages of the plugins are selectable in Bash, hopefully he will accept the idea, but if not thats an MAs prerogative. Fingers crossed also that a simple pass through the new CK will resolve the plugins problems being merged in a Bashed Patch. @Beermotor Ah you already tried and proved passing through the new CK works in this case :)

Link to comment
Share on other sites

No idea why someone would intentionally do that except to fool detection scripts that look for this stuff. That crosses the line from laziness to an actual attempt to sabotage someone's game though and I'd report such things to Nexus because it violates their ToS to have SSE mods that are not properly ported to be compatible with SSE.

Saving in the CK would have taken this person much less time than doing what they did in xEdit, unless they pirated their game and can't get the CK for SSE.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...