Wrye Bash - All Games

[godescalcus]

2 minutes ago, alt3rn1ty said:

Yeah from the nightly build when its updated just grab the source files and overwrite mopy folder with the new (I think, its been a long while since I had Python installed .. Pretty sure thats the case though), keep a copy of your last known good mopy zipped as a backup to roll back to if you experience problems which make using it more of a problem than you can manage, then just overwrite with the older mopy files. On a rare occasion I have experienced issues where after doing a roll back, the newer saved settings files are then no longer compatible with the older setup, and so you experience random bizarre problems after the roll back, but then you just delete all settings files like bashsettings.dat, and installers.dat, and then the next time Wrye Bash loads up, you have to set your settings again and overwrite install all your BAINs, and do a Full Refresh on the Installers Tab .. and that pretty much puts you back in a good place again.

Thank you for being so thorough, even the python.exe -m pip install --upgrade scandir bit which I had read about before but would have forgotten completely  I'm eager to try this and even help if I can. Loving to see the present interest in wb!

[Beermotor]

18 minutes ago, godescalcus said:

My experience is basic - I used to install the standalone executable loose files, then I replaced it with the python source. Right now I run wb through Wrye Bash Launcher.pyw. I wanted to ask you if it's possible/adviseable to just replace my mopy folder with the latest nightly build or bleeding edge version or whatever, keeping my personal preference files (folder locations and such). My knowledge of wb goes no further than trying to fix issues with plugins in xEdit when an error occurs, before blaming wb  But I'm not afraid of some learning and I do have a backup of my last stable skyrim.

I was going to post basically the same thing @alt3rn1tysaid but he beat me to it.

the new version is light-years ahead of the one on the Nexus and once you get Scandir added the speed increase will melt your face off.

There are numerous bug fixes (BCF files are back on the menu) and this one now has the ability to manage BodySlide packages which is something I know a lot of people had been asking for.

[godescalcus]

1 minute ago, Beermotor said:

I was going to post basically the same thing @alt3rn1tysaid but he beat me to it.

the new version is light-years ahead of the one on the Nexus and once you get Scandir added the speed increase will melt your face off.

There are numerous bug fixes, this one now has the ability to manage BodySlide packages which is something I know a lot of people had been asking for.

Will try to avoid spoiling this greatly informative topic so I'll just add that you've answered another of my questions. Likely you just crunched this whole topic in one sentence, that the new version is light years ahead. I'll probably keep my backup and just start modding from scratch to see how it goes. I wonder about FNIS integration, like BS it makes changes to installed files, I'd love to see wb managing those. With my current build I also have to fetch esp's changed by xEdit and either replace the ones in the installers or archive them as hot fixes. I probably sound obsessive... But I like to let wb run my data folder as thoroughly as possible.

[Beermotor]

1 minute ago, godescalcus said:

Will try to avoid spoiling this greatly informative topic so I'll just add that you've answered another of my questions. Likely you just crunched this whole topic in one sentence, that the new version is light years ahead. I'll probably keep my backup and just start modding from scratch to see how it goes. I wonder about FNIS integration, like BS it makes changes to installed files, I'd love to see wb managing those. With my current build I also have to fetch esp's changed by xEdit and either replace the ones in the installers or archive them as hot fixes. I probably sound obsessive... But I like to let wb run my data folder as thoroughly as possible.

No worries, this is the general-purpose catch-all Wrye Bash thread so you're welcome to come any time and ask questions, report bugs, etc.

This version should pick right up where you were, so you won't have to start modding from scratch unless of course you want to be obsessively safe. You can literally drop this in on top of your existing 307 Beta 1 install and go from there.

[Sharlikran]

On 9/8/2017 at 9:04 AM, Utumno said:

Not trivial - to begin with how can I differentiate the SSE saves from Skyrim ones ? pairs of  (version, formVersion) ? What are the valid ones for those games ? Is all this documented someplace (even in source code) ?

I have been researching classes.  One thing that currently happens is that to know whether or not you can edit save games you have to enable save game editing. That ability is in __init__.py as shown here.

That code is duplicated in all games in their init.py, so first move that to saves.py under the class.

class SaveFileHeader(object):
    canReadBasic = True   # All the basic stuff needed for the Saves Tab
    canEditMasters = True # Adjusting save file masters
    canEditMore = False   # No advanced editing, needs clarification
    save_ext = u'.ess'    # Save file extension
    cosave_ext = None     # Save file extension

    def __is_not_sse(self):
        if SaveFileHeader.version != 12: return true
    
    def __bool__(self)
        if __is_not_sse: return True
        
    def __can_edit_save(self)
        if __is_not_sse: return True

     
Then for Oblivion everything is fully functional. For All other games, canEditMore would be false by default. Since you inherit the default values, there is no need to add that again unless it needs to change as you know. For all of the elder scrolls games the save game extension is .ess, for Fallout change that to .fos, and then the same for the cosaves. Technically everything could go there, the Magic Name, and anything else related to the save games.  cosave_ext could even be a list [U'.obse', u'.pluggy'] and so on. 

Globally the ability to change the masters could still use canEditMasters. I don't know the code but that's probably the ability to check the option while on the save tab to enable or disable editing the masters of the save game. Keep that the way it is and utilize it only for the GUI. However, use dunder bool to change the behavior of SaveFileHeader so that you check per file based on the version number.  If dunder bool is overkill ro seems silly, then just use a normal method.  As I specified Fallout 4 does have a version 12 for a specific version of the save game, so that isn't perfect, but that's the gist of my idea anyway.

 

[Leonardo]

Today, I read the OP and I know most people should do that, but often people forget to read the OP or read the readme for a mod.

Anyway, when I read the "reporting a bug" section I got an idea that could help any user to generate a bugdumb via a simple command in WB.

What I mean is that a DOS command should work as intended when running a BAT file, unless there is something, I am not aware of, that prevents a WB user to do that via a "Generate a bugdump" command in WB.

[Sharlikran]

If you have the python install you use the Wrye Bash Debug.bat. If you use the EXE the documentation mentions either -d or --debug, because they are long and short versions of those parameters. You can also enable debug from settings, which is the gear icon.

P.S. I posted something on fliggerty, when you have time.

[lmstearn]

4 hours ago, Leonardo said:

Today, I read the OP and I know most people should do that, but often people forget to read the OP or read the readme for a mod.

Anyway, when I read the "reporting a bug" section I got an idea that could help any user to generate a bugdumb via a simple command in WB.

What I mean is that a DOS command should work as intended when running a BAT file, unless there is something, I am not aware of, that prevents a WB user to do that via a "Generate a bugdump" command in WB.

That's kind of a good idea really.

You know, MS have always "welcomed" users to post their crash dumps to some kind of Room 101 repo for years.
With an increasing user base why can't WB?

Consider a filter bot to remove sensitive data, and to remove duplicate dumps from different users, but keep a tally of the dump category. Comes with the usual disclaimer, feedback box, and the usual blah about how much the team is dedicated to your issue and the lightning speed of response times.

And for a certain set of OS related segs, provide an option to send a copy to MS. They will love it too!  8|

[Beermotor]

23 hours ago, lmstearn said:

Consider a filter bot to remove sensitive data, and to remove duplicate dumps from different users, but keep a tally of the dump category. Comes with the usual disclaimer, feedback box, and the usual blah about how much the team is dedicated to your issue and the lightning speed of response times.

And for a certain set of OS related segs, provide an option to send a copy to MS. They will love it too!  8|

I think making it easier for the users of executables to create debug logs would be a fantastic idea.  If we had a button or option "relaunch in debug mode" available it might be even snazzier, but that may be a v308 feature.

In the interim we could also just include a batch file with the standalone executable that does the same thing that the debug batch file in Python version does.  

It may be as simple as this:

@ECHO OFF && PUSHD "%~dp0" && SETLOCAL EnableDelayedExpansion
:: Wrye Bash Debug.bat
::

SET OUTFILE=BashBugDump_startup.log
cls
:exebash
ECHO Launching Wrye Bash 307 in debug mode
ECHO Launching Wrye Bash 307 in debug mode >>%OUTFILE%

"Wrye Bash.exe" -d %1 %2 %3 %4 %5 %6 %7 %8 %9 >>%OUTFILE% 

:end
ENDLOCAL && EXIT /B

^I'm sitting at work on a Mac so I can't really test that at the moment, and I've not done any batch file stuff since Clinton was in office.

I'm sure someone will come along and correct me if I'm incorrectly assessing the ease of adding that to the installer builder.

[lmstearn]

More recent versions of Windows now encourage the use of Powershell, but the original cmd shell will hang around for years. Powershell can upload the files to a repo apparently, but haven't dabbled too much. Will it run on a Mac?

[Beermotor]

36 minutes ago, lmstearn said:

More recent versions of Windows now encourage the use of Powershell, but the original cmd shell will hang around for years. Powershell can upload the files to a repo apparently, but haven't dabbled too much. Will it run on a Mac?

Yes Powershell will run on a Mac but it won't run DOS/Lan Manager/NT-style batch files.  Wrye Bash will run native on a Mac too, but since I don't have any games to manage I can't tell if it will get past that part.

[Leonardo]

@Beermotor  The only thing you need to add to that bat file is the DOS command "CLS" (CLearScreen) since the DOS screen often has various messages from Windows when using the "C:\" prompt bat file in Windows and it's easier to read a bugdump without seeing Windows messages on the screen in DOS.

[Leonardo]

15 hours ago, Sharlikran said:

If you have the python install you use the Wrye Bash Debug.bat. If you use the EXE the documentation mentions either -d or --debug, because they are long and short versions of those parameters. You can also enable debug from settings, which is the gear icon.

P.S. I posted something on fliggerty, when you have time.

I knew there was something useful I could use in WB, thanks Sharlikran.

I replied to your post there.

[Sharlikran]

Spoiler

cellRecAttrs = {
    u'C.Climate': ('climate',),
    u'C.Light': ('ambientRed','ambientGreen','ambientBlue','unused1',
                'directionalRed','directionalGreen','directionalBlue','unused2',
                'fogRed','fogGreen','fogBlue','unused3',
                'fogNear','fogFar','directionalXY','directionalZ',
                'directionalFade','fogClip'),
    u'C.Music': ('music',),
    u'C.Name': ('full',),
    u'C.Owner': ('ownership',),
    u'C.RecordFlags': ('flags1',),
    u'C.Water': ('water','waterHeight',),
}

bash_tags = ['C.Climate', 'C.Music']

attrs = list()
for tag, attributes in cellRecAttrs.items():
    print tag
    if tag in bash_tags:
        for attr in attributes:
            print attr
            attrs.append(attr)
print attrs

 

 

[Beermotor]

10 hours ago, Leonardo said:

@Beermotor  The only thing you need to add to that bat file is the DOS command "CLS" (CLearScreen) since the DOS screen often has various messages from Windows when using the "C:\" prompt bat file in Windows and it's easier to read a bugdump without seeing Windows messages on the screen in DOS.

Yes that's a great idea. the '@echo off' should take care of some of the spam but the cls should take care of anything in between the start of the bat and when the app starts. Thanks!

3 hours ago, Sharlikran said:

  Reveal hidden contents

 

cellRecAttrs = {
    u'C.Climate': ('climate',),
    u'C.Light': ('ambientRed','ambientGreen','ambientBlue','unused1',
                'directionalRed','directionalGreen','directionalBlue','unused2',
                'fogRed','fogGreen','fogBlue','unused3',
                'fogNear','fogFar','directionalXY','directionalZ',
                'directionalFade','fogClip'),
    u'C.Music': ('music',),
    u'C.Name': ('full',),
    u'C.Owner': ('ownership',),
    u'C.RecordFlags': ('flags1',),
    u'C.Water': ('water','waterHeight',),
}

bash_tags = ['C.Climate', 'C.Music']

attrs = list()
for tag, attributes in cellRecAttrs.items():
    print tag
    if tag in bash_tags:
        for attr in attributes:
            print attr
            attrs.append(attr)
print attrs

 

 

This looks deliciously like it is related to the patcher.  I'm following this with great interest.   I downloaded your FO3/FONV branch a few days ago to check it out, and if you need any help testing or investigating anything please let me know. 

[Utumno]

On 9/8/2017 at 7:04 PM, Utumno said:

Thanks - how does it specify SSE compressed save ? What values can it take ?

---

Not trivial - to begin with how can I differentiate the SSE saves from Skyrim ones ? pairs of  (version, formVersion) ? What are the valid ones for those games ? Is all this documented someplace (even in source code) ?

@Sharlikran - what I need to know is (if you know) what are the valid (version, formVersion) pairs for each game. No need to link me to code and explain to me what's where, I very well know

Also _compressType (SSE) - what values does it take ?

If you don't know just let me know

Thanks

[Beermotor]

An Alpha of SKSE64 for SSE just dropped. Obviously this isn't for public consumption yet, but it's a good sign. 

 

Edit: It looks like it is similar to the SKSE as  far as the structure goes. The names appear to be the same with "64" tagged on.

skse64_1_4_2.dll
skse64_loader.exe
skse64_steam_loader.dll

 

[Sharlikran]

4 hours ago, Utumno said:

@Sharlikran - what I need to know is (if you know) what are the valid (version, formVersion) pairs for each game. No need to link me to code and explain to me what's where, I very well know

Also _compressType (SSE) - what values does it take ?

If you don't know just let me know

Thanks

I thought the only value for _compressType was 2, but when I went to link that in the xEdit code I see that Hlp has it as an Enum in xEdit. 0 = None, 1 = Zlib, 2 = lz4.

For the version and formVersion of save games, there is no documentation on that. This is like any other kind of decoding. Bethesda doesn't tell us what the values are so we have to find them by observing.  To get the values I added to the doc strings, I made an if statement to print that into the bash log. Then as I found values, I put in an if statement if != to a known value, print out the values. As you can see that's never going to be accurate and you wouldn't want to add that to a list and check against it because someone is bound to have values you aren't looking for.

For plugins you have a Form Version and the Plugin Version like in the Wrye source code. Zilav told me the values aren't important to xEdit necessarily. In cases where a special condition is needed there is already something to handle that case. No documentation on Form Versions and Plugin Version anywhere that I am aware of.  The UESP Wiki just has the record format in Wiki form.

[Sharlikran]

1 hour ago, Beermotor said:

Edit: It looks like it is similar to the SKSE as  far as the structure goes. The names appear to be the same with "64" tagged on.

skse64_1_4_2.dll
skse64_loader.exe
skse64_steam_loader.dll

 

I went to http://skse.silverlock.org/ to play it safe.

[Sharlikran]

3 hours ago, Beermotor said:

This looks deliciously like it is related to the patcher.  I'm following this with great interest.   I downloaded your FO3/FONV branch a few days ago to check it out, and if you need any help testing or investigating anything please let me know. 

No I just watched a video on Dictionaries, that's already in the Wrye Source code for the Cell patcher. It would be useful for the Race patcher though.

[alt3rn1ty]

On 27/08/2017 at 11:10 AM, Beermotor said:

I'm on the Insider's Ring running the Creator's Update that breaks MO/MO2: version 1703  Build 16251.1002.

I'm on Win 10 x64 Creators update (not the insiders, just normal public release) which just updated today to Ver 1703 Build 15063.608, and did a restart boot install of new software.

I think the version the public is going to be subjected to is quite a way off (though understandable with the amount of software that makes up the Windows OS, it takes a lot of team development to get to the public stage) .. Do you have a rough guesstimate how long it could be before that version hits the streets ? (I understand its probably a hard question to answer with any accuracy but knowing your background you may have a feel for it).

[Beermotor]

2 hours ago, alt3rn1ty said:

I'm on Win 10 x64 Creators update (not the insiders, just normal public release) which just updated today to Ver 1703 Build 15063.608, and did a restart boot install of new software.

I think the version the public is going to be subjected to is quite a way off (though understandable with the amount of software that makes up the Windows OS, it takes a lot of team development to get to the public stage) .. Do you have a rough guesstimate how long it could be before that version hits the streets ? (I understand its probably a hard question to answer with any accuracy but knowing your background you may have a feel for it).

The official announced release date for the Fall Creator's Update is scheduled for 17 October 2017. They put out a blog post earlier this month to announce it.   I don't have any information on the version or build numbers for the final public release of this however since they're still pushing out last minute fixes and compatibility shims for things. 

[caligatio]

Hey, sorry to break your discussion but I have a problem you guys might be able to help with.

I want to make a mod for FO4 including new npcs and such but it is supposed to run on the PS4 too, so I'm a bit restricted with the character creation. I cant use LooksMenu for example and Face Ripper doesnt work anymore and will most likely never get an update. Creating a good looking char in the ck is simply impossible so I remembered that older versions of Wrye Bash had a function to extract the pc data from a savegame and save it as .npc file, right?

Is it possible to get that function back?

If not (or there is simply no time for that), do you know how a npc file should look like for FO4? I looked through some old npc files from skyrim and I might be able to create a bat file that uses the json files created by looksmenu to generate a npc file.

Any help would be really nice.

[Sharlikran]

One thing to understand about tools is that they don't normally remove functions. For example, currently if you use Wrye Bash to look at a Skyrim SE save file you can see the names. What normally happens is the ability to see it or edit that is disabled if nobody knows the format for the information.  In the beginning we could not read the part of the save file that held the file names of the mods used to make the save game until someone figured out that it was compressed data.  Then someone added a small block of code that emulated LZ4 compression to display the names, but you can't change them because that would require compressing the data and rewriting the file.

Without someone to decode the save games, find the location, and write the Python code, then that data can't be exported. That's why it's not there. This project was maintained by several people when Oblivion was the current game. Now only 1 person maintains it. People like myself try to help by adding records for other games from xEdit and I recently decoded the file names for saves in Fallout 4 and did some basic cosave decoding. However, what you want would take many more people working on the project.

It's just not possible. Your option is only the CK at this point. Not even xEdit can export the Face Gen Data.

[caligatio]

Hmm... I see, that sux. I would gladly help out, but my programing skills are very limited especially with phyton

Well I guess I'll have to try my best with the json files then. Just need to figure the differences between the Skyrim and Fallout 4 npc-files.