Jump to content

Wrye Bash - All Games


Utumno

Recommended Posts

4 hours ago, Sharlikran said:

I don't understand the screen shot. It shows txt and xml files.  Those files are currently ignored.  I'm also concerned because in the past Wrye installed just files that come from Bethesda, like in the BSA files.  Not every single file type because it belongs to something that is already non standard, and was never allowed to install in the past.  If we start adding all manner of files like txt and xml files as valid files then Wrye will start installing all sorts of additional and unneeded files that aren't related to BodySlide at all.

So to clarify that screenshot:  you don't see the .bsd, .osd, and other files that are used by BodySlide because I shrunk the window up so I could get all of the subpackages into the screenshot. :)

As of 20170802 Bash looks for "Tools/Bodyslide" for FO4 and "CalienteTools/Bodyslide" for Skyrim/Skyrim SE now and will track anything within those subpackages that matches the files extensions used by BS/OS (the list of these file extensions and types was provided by Ousnius) as long as they aren't executables.  I originally lobbied for the EXE ban to be overridden for BodySlide, but the historic policy of Wrye Bash has been to skip executables for security and liability reasons.  I couldn't think of a counterpoint to this argument other than user convenience. 

Regarding the BodySlide exe files being removed during a Clean Data, there is a method to sidestep this from happening that I'm going to roll into the documentation eventually.

 

Link to comment
Share on other sites

1 hour ago, Utumno said:

Re: bodyslide (issue 379) - any luck  @Beermotor in pushing the wiki ? There we should have all needed info/details.

Negative. I'm having OSX ssh-agent issues on both of my Macs due to long histories of them being used at work with our private git repos.  I have not tried Git in Windows Services for Linux yet however so that's my next step.

If I can't get that to work then we may be able to try looking at the Fork and Squash method I found here.

 

Link to comment
Share on other sites

1 minute ago, Beermotor said:

Negative. I'm having OSX ssh-agent issues on both of my Macs due to long histories of them being used at work with our private git repos.  I have not tried Git in Windows Services for Linux yet however so that's my next step.

If I can't get that to work then we may be able to try looking at the Fork and Squash method I found here.

 

I have invited you to the repo - IIRC you must accept that to get write rights - which will permit you to push to wrye-Bash therefore its wiki - what a mess

Link to comment
Share on other sites

12 minutes ago, Utumno said:

I have invited you to the repo - IIRC you must accept that to get write rights - which will permit you to push to wrye-Bash therefore its wiki - what a mess

Give me a second, I need to go on a wild GitHub repo-invite email invite hunt. :)

Link to comment
Share on other sites

7 minutes ago, Utumno said:

I found ithe email.  I told you I suck at GitHub.

Background: fortunately one of our devs was on Slack and told me the same thing you did. He also said "don't you ever check your email?". :)

I've got another question that is specific to this repo but I don't want to spam the thread with it, so I'll PM you about it.

Link to comment
Share on other sites

1 hour ago, Beermotor said:

Negative. I'm having OSX ssh-agent issues on both of my Macs due to long histories of them being used at work with our private git repos.  I have not tried Git in Windows Services for Linux yet however so that's my next step.

If you want to *practice* Git on Windows then send a request to AndalayBay about having access to GitLab (use a different a name than GitHub, but GitLab is basically the same as GitHub) at TAL or read this about hosting your project at TAL. :)

 

Now back on topic, sorry Utumno for being slightly off-topic.

Link to comment
Share on other sites

Thanks for explaining that Beermotor as I didn't understand what was missing from the screenshot.  Thanks Alt3rn1ty, I mentioned the .xml and .txt files because of what I saw in the screen shot. So like you mentioned you don't want redundant files, that was the point I was making also.

  • Like 2
Link to comment
Share on other sites

Utumno, in response to your request on GitHub.

You have taken code that has been pieced together over the years by many authors and have rewritten it in a more elegant way, which is quite a feet. Now you want me to write an elegant Wiki page or document so you can rewrite the code you haven't touched yet.

When you see the Wrye Bash code your head explodes because of all the things you don't like about how the code is written. Kind of like some of the points in this video, Stop Writing Classes.  Aside from the bulk of the code that you have rewritten, the code you see in Wrye is what you have to work with regardless of how elegant it is or isn't written.  If you need it to be more elegant all I can suggest is to look at it and figure out what it's doing and then make it more to your liking. Which I have mentioned before.

I appreciate the vote of confidence that I could write a Wiki for later use with Wrye Bash. I am many things but an expert at writing documentation isn't one of them.  When I was in College my teachers wanted more supporting data in my Speech and English Literature classes.  I was never good at doing that.  Bethesda doesn't give out the supporting data anyway.  Both Bethesda's and the community's information is scattered. It is sometimes slightly outdated or inconsistent but in a very minor way.  It usually just needs a minor change.  However, the source code for xEdit is accurate and up to date.

The UESP Wiki says on the Oblivion page for the TES4 Mod File Format, "While these pages still provide the best public documentation of the Oblivion file formats, all mod makers should also be using Tes4View, which currently provides the most complete and correct understanding of the mod file format in a readily accessible manner."

I recommend looking at how the information is presented in xEdit.  Download the various versions of xEdit (version 3.2 currently) and install them so you can visually see what information is offered. When you first open xEdit (TES5Edit, TES4Edit, FO4Edit) you will have to go to Options.  Right click on the left side, the part where the plugin names and the records appear. Once you Right click go to Other at the bottom, and then Options and you will be on the General Tab.  You will see several boxes that say to Hide information. That hides many things that might be useful to you but makes the record less readable, so uncheck those things. I suggest this because if you ask something I don't know how to explain, or my attempt to explain isn't working, then I'm just going to make a screen shot. So to avoid all that, just look at what you see in xEdit and then ask about what you are seeing.

When I first started working on TES5Edit Zilav had not started helping yet. He started a month or so later.  So Elminster asked me to only define the records. I seemed capable of doing that but when I got into the supporting code I had no idea what it did. He explained he would take care of all that when he compiled it.  Once Zilav came along that's what he did. So I have never had to learn all the stuff you want to know.  Over the years I have learned many things but only what I needed or what I wanted to know in order to do something I wanted to do.

I don't think that will be enough for you. Linking xEdit code will work in some cases when it's a short block of code because I can explain it and ask Zilav or Hlp for clarification as needed.  Some of the code is simply too complex and uses other code that Wrye Bash doesn't have.  Also I don't know where everything is.  I can explain what I know but I think that's about all I can offer honestly.

As far as Wiki pages go there is already the UESP Wiki. I don't know if you have ever gone there but it probably has the information you want. I just don't know if it's in the way you want it. I really don't want to rewrite what is already there to present it in a way that makes sense to you. I am happy to make minor updates though if something is wrong, but only if I am sure of the changes.

 - Oblivion TES4 Header information
 - Skyrim TES5 Header information
 - Fallout 3 Header/Record information - WrinklyNinja
 - Fallout NV Header/Record information - WrinklyNinja
 - Fallout 3 Group Record information - WrinklyNinja
 - Fallout NV Group Record information - WrinklyNinja
 
One of the things about those Wiki pages is the format of the 24 bytes doesn't change much. In my opinion you really only need to compare Oblivion, with one of the other games. About the only thing that changes is the Flags because they are record specific.  The TES4 header is defined in the following Data which is a group of subrecords.  A GRUP or Top Group also has 24 bytes and indicates the beginning of what will follow. Behtesda's internal tool sometimes introduces duplicate Top Groups which are combined in xEdit. Each record starts with 24 bytes, and then the subrecords follow.  After the 24 bytes is the size of the data that will follow from what I understand.  From what I know Oblivion is 20 bytes, and that's about the only difference.  Zilav only mentioned the other day when I asked that Oblivion doesn't have a Form Version because it was the first use of the new format.

That's what I'll provide for now, because I just don't think you can realistically know what you need to know until you start looking at plugins, the record headers, and see what xEdit shows in the GUI.  How much have you used the various creation kits from Bethesda? How much have you used xEdit in the past?  How much do you understand about plugins?  If I explain what I know, how much will you understand, even if it's presented in an Plugins For Idiots kind of way?

Link to comment
Share on other sites

6 hours ago, Utumno said:

Will see what broke in bodyslide - @Daidalos just force push utumno-scandir rebased on utumno-wip so I can see what's in there

Done.

I'll now go and see what I can do about showing startup errors as discussed in #373.

Link to comment
Share on other sites

New files in Fallout 4 \ Data \

The following just downloaded to update the game ( see this article ), so the new files among these need adding to the FO4 Wrye Bash official content so they are exempt from "Clean Data" :


ccBGSFO4001-PipBoy(Black) - Main.ba2
ccBGSFO4001-PipBoy(Black) - Textures.ba2
ccBGSFO4003-PipBoy(Camo01) - Main.ba2
ccBGSFO4003-PipBoy(Camo01) - Textures.ba2
ccBGSFO4004-PipBoy(Camo02) - Main.ba2
ccBGSFO4004-PipBoy(Camo02) - Textures.ba2
ccBGSFO4006-PipBoy(Chrome) - Main.ba2
ccBGSFO4006-PipBoy(Chrome) - Textures.ba2
ccBGSFO4016-Prey - Main.ba2
ccBGSFO4016-Prey - Textures.ba2
ccBGSFO4018-GaussRiflePrototype - Main.ba2
ccBGSFO4018-GaussRiflePrototype - Textures.ba2
ccBGSFO4019-ChineseStealthArmor - Main.ba2
ccBGSFO4019-ChineseStealthArmor - Textures.ba2
ccBGSFO4020-PowerArmorSkin(Black) - Main.ba2
ccBGSFO4020-PowerArmorSkin(Black) - Textures.ba2
ccBGSFO4038-HorseArmor - Main.ba2
ccBGSFO4038-HorseArmor - Textures.ba2
ccBGSFO4044-HellfirePowerArmor - Main.ba2
ccBGSFO4044-HellfirePowerArmor - Textures.ba2
ccFRSFO4001-HandmadeShotgun - Main.ba2
ccFRSFO4001-HandmadeShotgun - Textures.ba2
ccFSVFO4001-ModularMilitaryBackpack - Main.ba2
ccFSVFO4001-ModularMilitaryBackpack - Textures.ba2
ccFSVFO4002-MidCenturyModern - Main.ba2
ccFSVFO4002-MidCenturyModern - Textures.ba2
Fallout4 - Startup.ba2
Fallout4 - Interface.ba2
Fallout4 - Materials.ba2
Fallout4 - Meshes.ba2
Fallout4 - Misc.ba2
Fallout4.esm

I do not have the Creation Kit, so there may be other official files need adding I am not aware of. I included the last six files on that list because I am not entirely sure every one of them was there prior to this update, Fallout4 - Startup.ba2 looks new, but may not be. There was also a new Fallout4.exe but thats no concern for Wrye Bash, it doesn't live in Data.

Spelling is correct in that code box, I copied all new files to a separate folder then redirected a list to a .txt file, which was then pasted into the code box.

 

I think we can probably expect a similar bunch of files for Skyrim SE too next month sometime

Link to comment
Share on other sites

55 minutes ago, alt3rn1ty said:

Fallout4 - Startup.ba2 Fallout4 - Interface.ba2 Fallout4 - Materials.ba2 Fallout4 - Meshes.ba2 Fallout4 - Misc.ba2

Those are present in my mainline Fallout 4 installation.

Link to comment
Share on other sites

Yeah, it's only the CC files that are actually new. Though one questions why those would be sent down with a beta patch for people who have not paid for any of it.

Link to comment
Share on other sites

2 hours ago, Arthmoor said:

Yeah, it's only the CC files that are actually new. Though one questions why those would be sent down with a beta patch for people who have not paid for any of it.

Edit: never mind I saw the other thread in the FO4 Unofficial Patch thread. This appears to be a standard beta load or something.

This raises the question: where are the plugins?   Has anyone launched the game with a purchased CC mod to see if the plugin magically appears in the Data directory after the game launches?

The reason I ask is I'm curious if they're going to implement some kind of DRM with CC to prevent people from snagging mods and sharing them with their friends.

Link to comment
Share on other sites

I suspect Fallout 4 saves will need to be looked into again:

abQFGuq.png

I can supply a couple of saves with the beta installed if you want them.

EDIT: Actually I may as well just attach them now. The "Exit Save" had no ESL files attached, the other one does.

Saves.7z

Link to comment
Share on other sites

@alt3rn1ty I just finished checking and I am about to go in game and make some new save games, but I agree with Arthmoor. There doesn't seem to be any new files just the Creation Club ones.

Link to comment
Share on other sites

I had posted on GitHub some questions, comments, and just general rambling.  I was looking for a template Jon had provided not realizing the mistake I was making.  Then I explained here that the Skyrim SE Save Game code was imported into Fallout 4 when they are different. Which is why there is an error.  I wanted to help resolve the issue.  However, I work on things to help the community not to make any one individual happy or discontent. If you are bothered by anything I do, please keep it to yourself I don't really care. I can either help the project or not.  I guess if you don't want me around, just say so.  At least be forthcoming about it.  Otherwise don't make passive aggressive or patronizing comments because I don't have the patience or the tolerance for it.

  • Confused 2
Link to comment
Share on other sites

As far as the Save Game not working for Fallout 4, I took a look at the code, and I see why.  Someone changed the code such that it imports the Skyrim SE save header for Fallout 4 which isn't what Lojack had, and what I later updated.

EDIT: After making a Template for Skyrim SE out of Jon's Template with alterations from okzach's template, the save games are more similar then I thought.

For example with my old code for Skyrim SE there was a spot that you read 17 bytes to skip unused data but that exact spot in Fallout 4 you skip 16 bytes. For Fallout 4 after you read in the unk3 which is one byte, you had to read two unknown bytes, and then a string which is the game version.  For Skyrim SE you don't do that.Now with that said, the 010 template I found more clearly defines the save header for Fallout 4. I'm not advanced enough to rewrite the Fallout 4 code by importing. Tomorrow I might be able to cut and replace the Fallout 4 code with the Sakyrim SE code, and then alter it to read correctly. I don't know when I will have time though.  This is the 010 template I found.I'm not sure I can break that down into simpler terms just yet because I barley download 010 that @Jon used to help decode Skyrim SE Save games. While he knows how to use it, I don't. This is my best attempt though. I don't know if that will work as is in 010.

Spoiler



struct FILE {
    struct HEADER {
        char magic[12];
        uint headerSize;
        uint Save Version;
        uint Save Number; 
        SSTRING16 playerName;  
        uint playerLevel;  
        SSTRING16 playerLocation;  
        SSTRING16 gameTime;  
        SSTRING16 playerRace;  
        ushort playerSex;  
        float Experience earned;
        float Experience required;
        FILETIME filetime <name="filetime">; # Explained to Hlp below
        uint screenshotWidth ;
        uint screenshotHeight ;
    } header;

    struct IMAGE {
        struct COLOR {
            ubyte R;
            ubyte G;
            ubyte B;
            ubyte A;
        } color[header.screenshotWidth*header.screenshotHeight] <optimize=true>;
    } image;

    struct OTHER {
        byte FormVersion;
        SSTRING16 Game Version;  
    } other;

    struct MASTERS {
        uint PluginListSize;
        struct PLUGINLIST {
            byte PluginListCount;
            SSTRING16 pluginList[count];  
        } pluginList;
    } masters;

    if (other.formVersion == 68)
        ushort unknown <name="unknown", fgcolor=cGreen, bgcolor=cGray>;

 

Also the save version for Skyrim SE is 12 and for Fallout 4 it's 15.

EDIT: FILETIME filetime <name="filetime">; # Explained to Hlp below.

WARNING: I just made another discovery in regards to Fallout 4 Save Games.  The original author of the template okzach, felt that there was a Form Version.  I am inclined to agree because the template works on Form Version 67 and reads until the end of the file. The new Save Games have a 68 for the Form Version.

PLEASE TRACK:  The Form Version for Fallout 4.  After the block of plugins there are two unused bytes in Form Version 68 that need to be read prior to altering the six offsets.

 

Link to comment
Share on other sites

This is what Skyrim SE should be. After Looking at it (And getting my game and test folder set up, which that and testing took 7 hours, you are welcome) I think the template below illustrates part of the issue.  For the Image you can't say if the Save Version is greater then 9 anymore. The Python code seems to indicate when the Save Version is not equal to 12.  That's not going to work either because the Save Version for Fallout 4 is 15, so since 15 is not equal to 12 metaphorically speaking, the code will explode.

Spoiler

 


struct FILE {
    struct HEADER {
        char magic[13];
        uint headerSize;
        uint saveVersion;
        uint saveNumber;
        WSTRING playerName;
        uint playerLevel;
        WSTRING playerLocation;
        WSTRING gameTime;
        WSTRING playerRace;
        GENDER playerGender;
        float   playerExp;
        float levelExp;
        FILETIME filetime <name="filetime">; # Explained to Hlp below
        uint screenshotWidth ;
        uint screenshotHeight ;
        if ( header.saveVersion > 9 ) and ( header.saveVersion < 15 )
            ushort compressionType <name="Compression Type">;
    } header;

    struct IMAGE  {
        struct COLOR {
            ubyte R;
            ubyte G;
            ubyte B;
            if ( header.saveVersion > 9 ) and ( header.saveVersion < 15 )
                ubyte A;
        } color[header.screenshotWidth*header.screenshotHeight];
    } image;

    if ( header.saveVersion > 9 ) and ( header.saveVersion < 15 )
        struct COMPRESSED {
            uint uncompressedSize;
            uint compressedSize;
            char data[compressedSize];
        } compressed <name="Compressed">;

    if ( header.saveVersion < 12 ) {
        // Form Version
        ubyte formVersion <name="From Version", fgcolor=cGreen, bgcolor=cGray>;

        // Plugin list
        uint32  pluginListSize <name="Plugin List Size">;
        struct PLUGINLIST {
            ubyte   count <name="Plugin File Count">;
            WSTRING pluginList[count] <optimize=false, name="Plugin Files">;
        } pluginlist <open=true, name="Plugin File List">;

        // File Map
        struct FILEMAP {
            uint32 refIDArrayCountOffset <name="RefID Array Count Offset", fgcolor=cBlack, bgcolor=cDkPurple>;
            uint32 unknownOffset <name="Unknown Table Offset", fgcolor=cBlack, bgcolor=cDkPurple>;
            uint32 globalData1Offset <name="Global Data 1 Offset", fgcolor=cGreen, bgcolor=cDkPurple>;
            uint32 globalData2Offset <name="Global Data 2 Offset", fgcolor=cGreen, bgcolor=cDkPurple>;
            uint32 changeFormsOffset <name="Change Forms Offset", fgcolor=cYellow, bgcolor=cDkPurple>;
            uint32 globalData3Offset <name="Global Data 3 Offset", fgcolor=cGreen, bgcolor=cDkPurple>;
            uint32 globalData1Count <name="Global Data 1 Count", fgcolor=cDkGreen, bgcolor=cDkPurple>;
            uint32 globalData2Count <name="Global Data 2 Count", fgcolor=cDkGreen, bgcolor=cDkPurple>;
            uint32 globalData3Count <name="Global Data 3 Count", fgcolor=cDkGreen, bgcolor=cDkPurple>;
            uint32 changeFormsCount <name="Change Forms Count", fgcolor=cYellow, bgcolor=cDkPurple>;
            uint32 unused[15] <name="Unused", fgcolor=cBlack, bgcolor=cDkPurple>;
        } filemap <open=true, name="File Map">;
    }

 

I am still comparing Save Game formats for Fallout 4 and Skyrim/Skyrim SE.  There is a chance you could use some of the Skyrim/Skyrim SE code for the Header and the Image.  Make sure you are checking for the Save Version as I have indicated.  However, after the Image in a Fallout 4 Save game you will need to include the struct I titled OTHER, and then you can start looking at the Masters. To do that you would make the Skyrim/Skyrim SE look for Form Version 9 or greater to include 12 for Skyrim SE and 15 for Fallout 4.

EDIT: Updated to include both Skyrim and Skyrim SE.

Link to comment
Share on other sites

Just edited the descriptions on Nexus sites, we had instructions for the Oblivion game telling them to extract into Skyrim Special Edition and one user was confused. The Skyrim one was also telling them to install to the special edition location too ..

Quote

Note that the 32-bit versions are required even if you are on a 64-bit operating system

Then:

  • For manual installations of either the Standalone or Python versions, extract the archive in your Skyrim game folder so that the Mopy directory appears in the Skyrim directory
  • For Python versions, run by double-clicking on Wrye Bash Launcher.pyw
  • For WBSA (Wrye Bash StandAlone), run by double-clicking on Wrye Bash.exe

Not all of them needed changing (FO4 and SSE were correct for the respective game) but I started editing all of them, so a few have bumped the last edit date unnecessarily.

 

One question on Nexus :

"So I made a slight boo boo. When dragging and dropping archived files into the installer tab I accidentally clicked the "Move to" option from the prompt and had checked the small box to not show the prompt any longer, but that was an accident. I had really wanted it keep making copies as I move the mods from their folder and into Bash as now instead it's simply moving the files completely out of their original folder and into Bash. How do I change this back so that it's always copying?"

Is this a case of having to delete one of the dat files ?, and if so I think it ought to be easier to undo via the settings icon

Link to comment
Share on other sites

8 hours ago, hlp said:

No. I don't remember solving it, but I did not spend much time on it and that was with FNV.

I was curious so I had to ask WrinklyNinja, I figured he would know.  He said, "The 010 reference says that FILETIME is "a 64-bit integer representing the number of 100-nanosecond intervals since 01/01/1601 12:00 AM" (a.k.a. the Win32 epoch). The value in the file is written as a little-endian integer, so the value is actually 0x01D2B5C79A40C0E0. I verified this using the tool here (though you're 1 second off).

You'd typically use an existing (eg. standard library) conversion function to get a textual date from that, rather than implementing an algorithm yourself, because calendars are non-trivial to get right. For example, FileTimeToString."

In case you ever want to use that.

Link to comment
Share on other sites

11 hours ago, Sharlikran said:

I had posted on GitHub some questions, comments, and just general rambling.  I was looking for a template Jon had provided not realizing the mistake I was making.  Then I explained here that the Skyrim SE Save Game code was imported into Fallout 4 when they are different. Which is why there is an error.  I wanted to help resolve the issue.  However, I work on things to help the community not to make any one individual happy or discontent. If you are bothered by anything I do, please keep it to yourself I don't really care. I can either help the project or not.  I guess if you don't want me around, just say so.  At least be forthcoming about it.  Otherwise don't make passive aggressive or patronizing comments because I don't have the patience or the tolerance for it.

Uh. Ok. I'm not sure if this is directed at me or not for what I asked on GitHub, but I wasn't trying to slap you down for making a mistake or chiding you somehow. I was asking a question because you were posting to a closed ticket, nothing more.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...