[WIPz] TES5Edit

[fadingsignal]

Uploaded new version.

Actually it was my fault, I copied imagespaces block, renamed to godrays, but forgot to update accepted FormID signatures.

 

Oh nice!  Thank you!  Well maybe in that case, I'll keep reporting back my findings just in case they are legitimate

 

So happy you're all working on this.

[fadingsignal]

Not sure if this is at all helpful, or is simply one of the record types that will be better suited to try and map post-CK, but working with the shader particle geometry (SPGD) I found a few values are now mapped differently, and here is what my tests in-game yielded:

• Gravity Velocity = This is the same
• Rotation Velocity = Doesn't appear to do anything in game that I could tell
• Particle Size X = Doesn't do anything
• Particle Size Y = shows NaN <-- being used or something new or pointing to a new record type?
• Center Offset Min = this is actually Particle Size X
• Center Offset Max = Doesn't seem to do anything
• Initial Rotation Range = This is actually Particle Size Y
• Number of Subtextures both X and Y don't seem to do anything
• Box Size doesn't seem to do anything
• Particle Density shows NaN
• MNAM - Unknown is just the old texture path, which is now the path to the material file (BGEM), which contains material metadata and texture paths (can be de-compiled to JSON using Gibbed's Fallout 4 tools, which the game will natively read without needing to compile back into binary.)

Hoping to get a wrangle on the particle density, if that's even exposed anymore.

[zilav]

I'd be careful about the MHDT field. We know that's been used in Skyrim for something dragon related, which makes sense since the data in that field of a CELL gets updated if you ask the CK to regenerate height data.

It would be unfortunate if Fallout4 requires that field to be always present, this will be compatibility hell when modding exterior worldspace. Every single mod that places a rock will conflict with other mods that actually change height data. For now without official tools I'm assuming it has the same role as RNAM in Skyrim - not important or eiher merged at runtime, so I remove it automatically. Hence the label "experimental"

 

 

Ok, I did a bit of testing/inspecting. FO4Edit has an issue when displaying text containing foreign accented characters. You need (probably among various others) the original french strings files to see it.

Example : formID 000CAAD1, RNAM - Prompt field.

US text : 'Same beliefs'

FR text : 'Mêmes idéaux' ,but instead is displayed as 'Même idéaux'.

UTF-8 is not supported for now. Actually none of xEdit versions could ever display anything outside of cp1252 since all internal string types are Ansi. I would like to help, but my knowledge and experience dealing with localization, encodings and code pages is nonexistent unfortunately.

 

Not sure if this is at all helpful, or is simply one of the record types that will be better suited to try and map post-CK, but working with the shader particle geometry (SPGD) I found a few values are now mapped differently, and here is what my tests in-game yielded:

Unless you could figure out definition somehow in a more reliable way (like asking F4SE devs or other knowledgeable people for SDPG form fields inside the engine) we'd better wait for GECK.

[Mator]

@zilav:

Regarding UTF8, I know there's some kind of tool out there that patches the TES5Edit executable to allow it to handle UTF8 strings properly.  That might help with FO4Edit for editing french strings.

 

Also, if other people have managed to make a tool that patches the executable, we should be able to change xEdit's source code to support UTF8.  It really should be extremely simple and should have no negative repurcussions (due to how UTF8 and ANSI overlap).

[zilav]

@zilav:

Regarding UTF8, I know there's some kind of tool out there that patches the TES5Edit executable to allow it to handle UTF8 strings properly.  That might help with FO4Edit for editing french strings.

 

Also, if other people have managed to make a tool that patches the executable, we should be able to change xEdit's source code to support UTF8.  It really should be extremely simple and should have no negative repurcussions (due to how UTF8 and ANSI overlap).

Probably even replacing all AnsiString with UTF8String would work... for Fallout4. But what about previous games? Did Oblivion use UTF-8 for french as Fallout 4? Blindly updating core modules now without the knowledge of how localization worked in older games could easily break something for other games. That's why I hesitated to make updates after TES5Edit release, and why I still hesitate now. Not sure why Elminster didn't add support for other languages right from the start, or maybe Oblivion and FO3 just used cp1252 for eveything. Then how did chinese users modded Oblivion I wonder if this is the case.

So a lot of questions and I have no answers.

[Nico coiN]

[snip] Did Oblivion use UTF-8 for french as Fallout 4? [snip]

Skyrim, Oblivion & Fallout New Vegas : no, I just checked. Fallout 3 : can't check atm, but it's probably the same as FNV. Not a real matter atm if it's only a display issue. All required typos fixes in all these text fields will be achieved directly in the GECK. And if we need to 'copy' them to another esp/esm with the click'n drop feature, then I guess FO4Edit will save them with the proper encoding managed by the game exe.

 

Apart from that, the desktop icon for FO4Edit is very blurry atm because there are not many different icon sizes. I made this one (containing 9 different sizes) with the original 256x256 one. Enjoy !

[fadingsignal]

Minor issue: When copying sound descriptors into another plugin, comparison values and function references on conditions are lost (set to 0 and null respectively) and cannot be edited, either directly or with drag/drop from the original plugin. 

 

See image here: http://i.imgur.com/wFqgccE.jpg

[Arthmoor]

It would be unfortunate if Fallout4 requires that field to be always present, this will be compatibility hell when modding exterior worldspace. Every single mod that places a rock will conflict with other mods that actually change height data. For now without official tools I'm assuming it has the same role as RNAM in Skyrim - not important or eiher merged at runtime, so I remove it automatically. Hence the label "experimental"

Sounds like we may not be understanding each other. It sound like you're planning to cause MHDT to get dropped from the worldspace and cell records. I'm not sure I'm getting the point of that since it's pretty clear the height data is relevant. Assuming it isn't a special case record that asserts itself whether any other files have it or not.

[zilav]

Minor issue: When copying sound descriptors into another plugin, comparison values and function references on conditions are lost (set to 0 and null respectively) and cannot be edited, either directly or with drag/drop from the original plugin.

Oops sorry, fixed. New version is up.

Other changes are:

- upgraded icon from Nico (by the way can you please png compress it? added 0.5MB to exe currently)

- hardcoded forms .dat file for FO4 from fireundubh, with missing AVIF records. It allowed to define "PRPS - Properties" subrecord used across several other records.

- minor definitions update

 

I'll check encodings later, first need to read how does it work internally. I was not joking when mentioned about zero experience with codepages   

[zilav]

Sounds like we may not be understanding each other. It sound like you're planning to cause MHDT to get dropped from the worldspace and cell records. I'm not sure I'm getting the point of that since it's pretty clear the height data is relevant. Assuming it isn't a special case record that asserts itself whether any other files have it or not.

All games prior to Skyrim has huge OFST subrecord in WRLD, xEdit deletes in automatically when loading plugins.

Skyrim added another one - RNAM. TES5Edit deletes it too.

Now Fallout4 adds even more -MHDT, WLEV, CLSZ. I set FO4Edit to delete them as well from WRLD. They are not deleted from CELL records.

[Nico coiN]

@Zilav : I don't know how you really want to implement the icon, so here are the resources. Archive contains all sizes embeded in previous ico file, and even handled with 2 different downscale Photoshop filters. Choose anything you need/want.

[Mator]

Probably even replacing all AnsiString with UTF8String would work... for Fallout4. But what about previous games? Did Oblivion use UTF-8 for french as Fallout 4? Blindly updating core modules now without the knowledge of how localization worked in older games could easily break something for other games. That's why I hesitated to make updates after TES5Edit release, and why I still hesitate now. Not sure why Elminster didn't add support for other languages right from the start, or maybe Oblivion and FO3 just used cp1252 for eveything. Then how did chinese users modded Oblivion I wonder if this is the case.

So a lot of questions and I have no answers.

 

I think the UTF8 patch was used for Skyrim and FNV as well.  Dunno about TES4.  I think just switching them to UTF8 would do the trick.  Perhaps we should just make a branch for this and then do some testing?

[McGuffin]

Hi there

 

Author of TesvTranslator and now Fallout4translator, here

I see that my fellow compatriote Nico Coin already spotted my update.

 

Thank you for working so fast on fo4edit, fortunately, this will ends the fear about bad esps soon

 

Here is what I found so far about Strings:

English uses codepage 1252

and all other languages I have tested (meaning all europeans released by steam), + Chinese and Russian are in UTF8

Strings format is exactly the same as before.

 

Also I have found all new fallout4 strings internal ref (meaning 100% of the english strings are applied in fallout4translator, without creating either orphan strings (strings without internal ref) or orphan records (record without lookup), and also no string with multiple call.

French/spanish/Deutch and few other languages strings produce 2 orphan strings, which is likely a bug for the game, since no 4bytes records recognize their references in the whole esm.

 

 

Since I spotted few  missing definitions in the current FO4edit here is my own definitions list:

#FieldId=RecordID=List (0: strings, 1: dlStrings, 2: ilStrings
#CharCase must match
#(*) indicate the string can't be null (needs at least one space).
#-proc1, -proc2 refer to internal procedures that defines some conditions for reading the data
Def_:DNAM=MGEF=0
Def_:NAM1=INFO=2*
Def_:SHRT=NPC_=0
Def_:CNAM=QUST=1
Def_:CNAM=BOOK=1
Def_:TNAM=WOOP=0
Def_:NNAM=QUST=0
Def_:NNAM=MESG=0
Def_:ITXT=MESG=0
Def_:RDMP=REGN=0
Def_:EPF2=PERK=0
Def_:RNAM=ACTI=0
Def_:RNAM=FLOR=0
Def_:RNAM=INFO=0*
Def_:BPTN=BPTD=0
Def_:MNAM=FACT=0
Def_:FNAM=FACT=0
Def_:DESC=LSCR=0
#Fallout4 
Def_:ONAM=AMMO=0
Def_:ONAM=LVLI=0
Def_:ANAM=AVIF=0
Def_:CNAM=DOOR=0
Def_:WNAM=INNR=0
Def_:FMRN=RACE=0
Def_:BTXT=TERM=0
Def_:ITXT=TERM=0
Def_:RNAM=TERM=0
Def_:UNAM=TERM=0
Def_:WNAM=TERM=0
Def_:DNAM=ALCH=0
Def_:ONAM=DOOR=0
Def_:TTGP=RACE=0
Def_:MPPN=RACE=0
Def_:NAM0=TERM=0
#
Def_:DATA=GMST=0-proc1
Def_:EPFD=PERK=0-proc2

#fallback, fields work for every records type
Def_:FULL=****=0
Def_:DESC=****=1
Def_:ATTX=****=0

Of course,  unused placeholder for other strings could also exist, but without corresponding strings associated there is no way to detect them atm.

I hope you will find this useful.

[fadingsignal]

Oops sorry, fixed. New version is up.

Other changes are:

- upgraded icon from Nico (by the way can you please png compress it? added 0.5MB to exe currently)

- hardcoded forms .dat file for FO4 from fireundubh, with missing AVIF records. It allowed to define "PRPS - Properties" subrecord used across several other records.

- minor definitions update

 

I'll check encodings later, first need to read how does it work internally. I was not joking when mentioned about zero experience with codepages 

 

You are awesome, thank you so much.

[Bigtalon]

Ok, I keep getting some error that it can't find my Skyrim files, but I downloaded version 3.1.2, and your thread says 3.1.3 contains the files for Fallout 4, so maybe I'm downloading the wrong version, or I need to install it to my fallout 4 or skyrim directory to use it ? I have both so that's not a problem, just not sure why it won't open, unless it's because I d/led the wrong version

[Arthmoor]

You need to download 3.1.3 from the link in the first post. 3.1.2 is the current public release on Nexus. The ones here are test versions and are only ever linked from the OP.

[Bigtalon]

You need to download 3.1.3 from the link in the first post. 3.1.2 is the current public release on Nexus. The ones here are test versions and are only ever linked from the OP.

Got it to work ty

[zilav]

Ok, I did a bit of testing/inspecting. FO4Edit has an issue when displaying text containing foreign accented characters. You need (probably among various others) the original french strings files to see it.

Example : formID 000CAAD1, RNAM - Prompt field.

US text : 'Same beliefs'

FR text : 'Mêmes idéaux' ,but instead is displayed as 'Même idéaux'.

Link doesn't work.

[Bigtalon]

MODT hasn't been decyphered yet I'm guessing ? Not coming up in any of the listings

[Nico coiN]

Link doesn't work.

Link restored. BTW, a manipulation error on my side made the icon png source files vanish. Do you want me to redo and reupload them ?

[zilav]

Link restored. BTW, a manipulation error on my side made the icon png source files vanish. Do you want me to redo and reupload them ?

if you can get it's size below the current 500Kb then sure

[ThatBenderGuy]

Are you keeping the original post updated with the latest build or are you posting the upadtes throughout this thread?

[hlp]

The latest build is always on the first post.

[Roy Batty]

Can you put up a mirror somewhere, Google drive is doing the "too many downloads in the last 24 hours" dance.

[expired6978]

MSWP - BNAM - Base Material

MSWP - SNAM - Swap Material

 

OMOD - DATA

pseudo-code

if formVersion < 53
   skip(4)
if formVersion >= 48
   includeCount = UInt32
if formVersion >= 48
   modifierCount = UInt32
if formVersion < 52
    skip(4)
if formVersion >= 48
    unk1 = UInt8
    unk2 = UInt8
if formVersion >= 63
    formType = UInt32 (e.g. ARMO)
else if formVersion >= 53
   formType = UInt8
if formVersion >= 90
   unk3 = UInt8
if formVersion >= 107
   unk4 = UInt8
keywordID = UInt32 (formID)
keywordCount = UInt32
for n = 0 to keywordCount
   keyword = UInt32
if formVersion >= 53
   count = UInt32
   for n = 0 to count
       item1 = UInt32
       item2 = UInt32
for n = 0 to includeCount
    unk1 = UInt32
    if formVersion >= 42
        unk2 = UInt8
    unk3 = version < 49 || UInt8
    unk4 = version < 49 || UInt8
for n = 0 to modifierCount
    valueType = UInt8 (0 - Int, 1 - Float, 2 - Bool, 4 - IDAndInt, 5 - Unknown, 6 - IDAndFloat)
    skip(3)
    functionType = UInt8 (0 - Set, 1 - Multiply, 2 - Add)
    skip(3)
    index = UInt16
    skip(3)
    switch(valueType)
        case 0
            value1 = SInt32
            value2 = SInt32
            break
        case 1
            value1 = float
            value2 = float
            break
        case 2
            value1 = UInt32
            value2 = UInt32
            break
        case 4
            formID = UInt32
            value2 = SInt32
            break
        case 5
            value1 = UInt32
            value2 = UInt32
            break
        case 6
            formID = UInt32
            value2 = float
            break
    factor = float



The modifier index is what type of modifier it is, here's a list of indices:

enum ArmorFormProperty // ARMO
{
    Enchantments = 0,			// type ID
    BashImpactDataSet = 1,
    BlockMaterial = 2,
    Keywords = 3,			// type ID
    Weight = 4,				// type Float
    Value = 5,				// type Float
    Rating = 6,				// type Int
    AddonIndex = 7,
    BodyPart = 8,
    DamageTypeValues = 9,		// type IDAndFloat
    ActorValues = 10,			// type IDAndFloat
    Health = 11,			// type Int
    ColorRemappingIndex = 12,
    MaterialSwaps = 13		// type ID
};

enum NPCFormProperty // NPC_
{
    Keywords = 0,
    ForcedInventory = 1,
    XPOffset = 2,
    Enchantments = 3,
    ColorRemappingIndex = 4,
    MaterialSwaps = 5
};

enum WeaponFormProperty // WEAP
{
    Speed = 0,
    Reach = 1,
    MinRange = 2,
    MaxRange = 3,
    AttackDelaySec = 4,
    //5,
    OutOfRangeDamageMult = 6,
    SecondaryDamage = 7,
    CriticalChargeBonus = 8,
    HitBehavior = 9,
    Rank = 10,
    //11,
    AmmoCapacity = 12,
    //13,
    //14,
    Type = 15,
    IsPlayerOnly = 16,
    NPCsUseAmmo = 17,
    HasChargingReload = 18,
    IsMinorCrime = 19,
    IsFixedRange = 20,
    HasEffectOnDeath = 21,
    HasAlternateRumble = 22,
    IsNonHostile = 23,
    IgnoreResist = 24,
    IsAutomatic = 25,
    CantDrop = 26,
    IsNonPlayable = 27,
    AttackDamage = 28,
    Value = 29,
    Weight = 30,
    Keywords = 31,
    AimModel = 32,
    AimModelMinConeDegrees = 33,
    AimModelMaxConeDegrees = 34,
    AimModelConeIncreasePerShot = 35,
    AimModelConeDecreasePerSec = 36,
    AimModelConeDecreaseDelayMs = 37,
    AimModelConeSneakMultiplier = 38,
    AimModelRecoilDiminishSpringForce = 39,
    AimModelRecoilDiminishSightsMult = 40,
    AimModelRecoilMaxDegPerShot = 41,
    AimModelRecoilMinDegPerShot = 42,
    AimModelRecoilHipMult = 43,
    AimModelRecoilShotsForRunaway = 44,
    AimModelRecoilArcDeg = 45,
    AimModelRecoilArcRotateDeg = 46,
    AimModelConeIronSightsMultiplier = 47,
    HasScope = 48,
    ZoomDataFOVMult = 49,
    FireSeconds = 50,
    NumProjectiles = 51,
    AttackSound = 52,
    AttackSound2D = 53,
    AttackLoop = 54,
    AttackFailSound = 55,
    IdleSound = 56,
    EquipSound = 57,
    UnEquipSound = 58,
    SoundLevel = 59,
    ImpactDataSet = 60,
    Ammo = 61,
    CritEffect = 62,
    BashImpactDataSet = 63,
    BlockMaterial = 64,
    Enchantments = 65,
    AimModelBaseStability = 66,
    ZoomData = 67,
    ZoomDataOverlay = 68,
    ZoomDataImageSpace = 69,
    ZoomDataCameraOffsetX = 70,
    ZoomDataCameraOffsetY = 71,
    ZoomDataCameraOffsetZ = 72,
    EquipSlot = 73,
    SoundLevelMult = 74,
    NPCAmmoList = 75,
    ReloadSpeed = 76,
    DamageTypeValues = 77,
    AccuracyBonus = 78,
    AttackActionPointCost = 79,
    OverrideProjectile = 80,
    HasBoltAction = 81,
    StaggerValue = 82,
    SightedTransitionSeconds = 83,
    FullPowerSeconds = 84,
    HoldInputToPower = 85,
    HasRepeatableSingleFire = 86,
    MinPowerPerShot = 87,
    ColorRemappingIndex = 88,
    MaterialSwaps = 89,
    CriticalDamageMult = 90,
    FastEquipSound = 91,
    DisableShells = 92,
    HasChargingAttack = 93,
    ActorValues = 94
};