First off I want to thank Dragon32 for helping me try to sort out this mess but I think this one needs to be asked here.

Ok, first let me start by saying i'm sure we all are aware that the TesTool Merge_objects option can only handle so much before it CTDs.

Now I know, beyond a shadow of a doubt, that most of us here run a ridiculous amount of mods that all need to have their respective objects merged.

And since there is no other program out there besides TesTool to merge objects then how do those of us with alot of mods that need object merging merge objects? There has to be a workaround.

Some smart guy must have figured something out somewhere somehow!

Let me ask a few questions that I know others have wondered about so that we can have this written down someplace and maybe even solve something.


1. Does one NEED to have Merged_Objects.esp? (if they have mods that need object merging)

2. Can you merge multiple merge_object.esps using testool and still get the desired result? CS? TESAME?
Note: I tried the CS but it crashes when attempting to merge.
Note Note: TESAME merged them fine but there was something wrong with the file according to Wyre Mash (some error when using "Repair all").
Note Note Note: The resulting Merged_Objects file using TesTool when merging multiple files results in a file MUCH smaller than the TESAME file. That can't be right?

3. How does one know if a particular mod requires object merging? If I somehow figure it out and "object merge" twelve mods but I missed one is the Merged_Objects.esp now worthless?

4. Can I take TesTool, pour gasoline on it, set it on fire, launch it into the atmosphere and then shoot it with a missle and STILL be able to enjoy Morrowind??????

So.....what do you think?


--joelR

Get this: TES Plugin Conflict Detector

Load all of your active mods into it and check for conflicts. If more than one mod affects the same object, then you ought to look into those more. For example, if you add one mod that adjusts the prices of all weapons, and another one that changes the damage ratings on all of them, you will need to merge them. I would suggest doing it manually by making a patch that incorporates all of the changes in the objects that are overwritten. That way you know that the changes you want are there....and only those changes.

Ok good. I can do something with this. I was hoping for a simpler solution but at least I can try this route. I dont know squat about making a patch and all that but I suppose I have to start someplace.



Addendum: Ok I got to thinking about this and I think that making patches is a little too advanced for the likes of me. What do people that arent so good with things like this do?

TESTool will only generate merged data when there are two or more object ids from separate mods with changes. If you try and merge multiple merged objects esps, the resulting Merged_Objects.esp will only retain data for the same ids that had changes in two or more of those previously merged esps. That's way you noticed a significant decrease in file size - only matching ids were kept.

You need to use another method to merge multiple merged objects esps - TESPCD like Fliggerty suggested, or Enchanted Editor. Either way you're best off doing the merging manually. I prefer using EE as it has an easy check-box method to copy records from one esp to the other, clipboard to hold the data, and a safe editing mode to help review what changes you're attempting before applying.

In some cases TESTool cannot merge object ids, seems to mostly occur when more than two esps modify the same object. This is indicated in the summary window and TESTool.log after merging. You should thoroughly review that log and then manually add the changes to the Merged_Objects.esp or to a new esp if you want all the changes to the object to show up in-game.

Loading multiple Merged_Objects.esp (e.g. Merged_Objects01.esp, Merged_Objects02.esp, Merged_Objects03.esp, etc...) is fine as long as the same object id does not appear in more than one of them. Otherwise, you just have the same problem you did before merging - the last loaded change wins.


When I first started managing mods, I just went with the resulting Merged_Objects.esp TESTool generated.

Because you have so many mods you're trying to merge which TESTool apparently is incapable of handling, the only other suggestion I can make is to merge them in separate groups:
-start with two groups, each half of your mods
-rename each Merged_Objects.esp so it won't get overwritten the next time you run the merge tool
-do NOT merge the resulting merged esps together

You might also want to exclude mods you know for certain do not conflict with any other mods since they do not need to be merged (again, TESPCD is invaluable in this regard).


Otherwise, you'll just have to be content with last mod wins.

Well you asked for smart guys, and you got the really clever people (lots over my head), however:
As good as TESTool was when it was conceived and implemented, these days us mugs are told that:
1/ Wrye Mash is what you should use to merge leveled lists;
2/ Never, ever merge dialogue;
3/ Merged objects probably aren't needed.

This is only from my reading of the boards, but it seems to work. But I'm more than happy to be howled down by the experts if they see a need.

Alright so after running TESPCD and generating the conflict report can I safely assume that the files under the "Overriding esp file" and "esp file" tabs are the ones that need to be "merged" and the rest in my list that do not show can be unchecked before making a new merged objects esp?

I noticed that some of the conflicts are patches. I assume that these arent "real" conflicts?


PS: What is the definition of an "Object" in MW?

Since Wrye Mash is the best thing to merge leveled lists, and since Wrye Mash is the best thing to do many things, we should ask Wrye to implement a merge objects function in his excellent tool.

Wrye is fairly retired these days I think

Affirmative.

Correct, unless the readme for the patch instructs you to merge (highly unlikely tho). Usually patches need to override the source, so merging their changes back with what they're supposed to be fixing/replacing is undesirable.

Anything you find in the Object window of the CS.

1) Yes. The Leveled List merger of TESTool is flawed.

2) Probably, but essentially yes: Better safe than sorry.

3) Negative, IMHO. For a large amount of mods, I find the Merged Objects function invaluable. Esp. if you use, like I do, some kind of item balancer (weight, weapon damage, price, etc.) together with mods like Wrye's name patches. Or a head replacer, a mod which adds an item to the NPC and another one which changes some stats. Without Merged Objects, I would never see any effect of some of my plugins...
Of course, I could do it manually in the Enchanted Editor. But why should I?


Wrye did his Bash program, his UESP wiki editing and various other ES-related activities while being, according to his sig, 90+% retired.
But yes, he's been more quiet lately.

Apart from the fact that Mash has been around for quite a while now, and he's never added such a feature, so it's rather unlikely that he will now.

i found testool invaluable for the reports it prints, and for cleaning altered cells of the residual entries. make sure to restrict dialog and cells though. merged objects works well if you know what to expect, but it can't handle very large files, but is wonderful for small merges.

merge dialog i never used. wyre mash is the best for levelled lists.

TESTool *could* handle lots of mods, the reason for the crash is (at least according to other people, I haven't tried to verify this myself) that the text buffer which holds the log overflows. If that is the case, then there are several solutions to the problem (inrease text buffer size, write directly to disk instead of to the buffer, write less text or stop logging if the buffer is full, etc.) which perhaps could be implemented relatively easily. However, we don't have the source code for TESTool. Has anyone ever tried to ask Ghostwheel for it?

------------------------------------------------------------

"Merging objects isn't needed" - I disagree. Of course it depends on the context, but it's easy to see how not using merged objects can break things. For example, lets say you have a mod that adds scripts to some objects in order to do some quests. Now let's say you're loading another mod after it which changes some unrelated data of the same objects. Although the second mod doesn't add scripts on its own, it will break the first mod since it will de-attach the scripts. Using merged objects fixes this.

------------------------------------------------------------

How to circumvent the crash problem in TESTool: This has already been answered. Basically there are three ways to do this:

a) Remove mods from the list until TESTool doesn't crash. Personally I find this method unsatisfying since it bears the risk of missing out on mergeable changes. However it's easy to perform and doesn't require any knowledge of additional tools, so it's the best option for people who simply don't want to dive too deep into the intricacies of Morrowind's data structures.

b ) Split your modlist into separate parts and make a merged_objects.esp for each part. This still has the risk to "lose" mergeable changes between those parts. However, with some work, one can usually find mods that can be split away and which don't interfere too much with others. For example, if you're using "book rotate" and "book jackets", then these two mods will create one entry for every single one of the hundreds of book objects in the game. So splitting off these two and making a seperate merged_objects.esp for them will give you a lot more space in youe "general" merged_objects.esp. Likewise, a mod that rebalances weapon stats and a mod that adds scripts to weapons (like "weapon rotate") will produce lots of entries and can be split away. Of course, in both cases you do run the risk that some other mod happens to change the same object, and you'll probably lose the change of this mod because TESTool wasn't aware of this mod when it created the merged objects for your books/weapons.

You can use TESPCD to find out which parts of your modlist are best to split away. You want to look for mods that have *many* conflicts which each other and *few* (ideally no) conflicts with anything else.

c) Create merged objects for separate parts of your modlist as mentioned above and then merge these files manually. This is possible, however as tetchy already said, you need to be aware that TESTool assumes that all mods which are active when creating the merged_objects.esp are also active when playing the game. So if TESTool doesn't find anything to merge for a particular object, then this object will not find its way into the merged_objects.esp. This can cause a problem in the following situation:

- You merge objects from "book rotate" and "book jackets" into "merged_books.esp"
- You merged objects from a weapon rebalance and "weapon rotate" into "merged_weapons.esp"
- You merge objects from the rest of your mods into "merged_general.esp"
- You merge objets from these three files into "merged_final.esp"
- You play the game with only merged_final.esp and remove the three other files

This will NOT have the desired effect because merged_final.esp only contains data for the *conflicts* between the other three files. The merged_books.esp will contain hundreds of entries (because there are hundreds of mergeable conflicts between "book rotate" and "book jackets"), but only a handful of those will also conflict with something in the other two esps. And only this handful of entries will make its way int merged_final.esp. If you switch off merged_books.esp, you will lose all the other entries which didn't find their way into merged_final.esp. (For TESTool this is perfectly logical since it expects you to have merged_books.esp active anyway, so it doesn't see any need to copy any of its objects into merged_final.esp unless it finds a conflict with another mod which can be resolved this way.)

So you can either merge the files mentioned above manually in the EE, or with TESAME. Notice that this operation is a merging of *mods*, not *objects*. It's important to fully understand that these are different operations. Merging objects with TESTool analyzes mods for conflicting object entries, then tries to reconcile these conflicts by merging the entries, and creates an esp which only contains data to solce these conflicts. Merging *mods* (with TESAME or the CS) takes *everything* from two mods and puts it together into one single mod.

For me, the best solution to the problem is the following:
1. First split away parts of your modlist as mentioned above
2. Create separate merged_objects.esp for each part, and one merged_objects.esp for the rest
3. *Do* run "Merge objects" on the files produced by the operations in the previous step
4. Use another tool (like TESAME) to merge all these files produced in (2) into one, and the, as the last step, merge this file with the file produced in (3). Make sure that the file produced in (3) gets priority so that it overwrites all conflicting data from the other file (you can probably do this by selecting it as the last file to merge, but double-check this).

This process should create one merged_objects.esp with most mergeable changes in your modlist. It won't catch every single one, but it catches most of them while still keeping the amount of effort needed manageable imho.

I'd still suggest to have a look at the various logfiles produced by the above operations to spot any undesired actions.

Also, I still think that having the source code of TESTool would be preferable since a working code-based solution would render the *whole* process descibed above moot. I'd also support anyone trying to programming a replacement for TESTool without the limitations we're hitting here, but this won't be easy since TESTool *is* doing quite a good job with its merging, it even manages to understand and merge data on a the subrecord level. But if anyone is up for the task and wants to have someone with a bit of knowledge of Morrowind's data structure fpr testing and advice, just ask.

------------------------------------------------------------

"Could Wrye add object merging as a feature to Mash?" - I think the chances for this are extremely slim. It's simply too much work concerning that Wrye is quite retired (although he's always around when I ask him a question, for which I'm eternally grateful) and that there *is* a working solution (TESTool) which only breaks down when using lots of mods.

------------------------------------------------------------

"What is an object in the TES context?" - In TES data, there are objects and references. Objects are like blueprints, while references are actual "things" built according to these blueprints.

Example: barrel_01 is an object. It defines, among other things, the looks and the contents of a certain type of barrel.

If you drop such a barrel_01 into a cell in Morrowind's Construction Set, then you've created a *reference*. This reference inherits all data from the object it refers to - i.e. the looks or contents of the new reference will be those specified in the object barrel_01. The new reference also has a set of *individual* data, for example its position and orientation, which it does not inherit from any object. (Objects do not have position or orientation data, it wouldn't make sense since objects are not anything tangible in the gameworld, but just abstract blueprints.)

If a mod moves some barrel_01 in Arrille's warehouse a few ticks to the south, then the mod only changes this specific reference, this one, unique barrel, because each reference has its own data for position and orientation. If, however, a mod changes the mesh of barrel_01 to something more detailed, then this change applies to the *object* barrel_01, because data about the looks of the barrel is stored on the object level. This means that the change (here: a new mesh) will apply automatically to all other references to barrel_01 in the game world, because all references simply refer to their barrel_01 object when the game looks for the data of how to draw them. (Note that the initial content of barrel_01 *is* stored on the object level, it's a common pitfall for beginning modders to change the contents of a barrel object, not realiting that by doing this they'll also change the contents of *all* other references to the same barrel object.)

Note that a lot of documentation about Morrowind is inconsistent with regard to the terms "object" and "reference". Specificall, the term "object" often gets used when "reference" would have been appropriate. A couple of years ago, this made it very difficult for me to understand the differences (I misunderstood the meaning of the objectindex for a while because it's actually a *reference* index).

With regard to TESTool: TESTool works on the object level. This means that if one mod changes the mesh of barrel_01, and another mod changes the contents of barrel_01, then TESTool can reconcile this conflict by merging these two changes together. TESTool does *not* work on the reference level, so if one mod moves a barrel from Arrille's tradehouse a bit to the left, and another scales it to double site, then TESTool cannot do anything about this conflict, since it occurs on the reference level, not the object level.

Luckily, this also means that you can modify your merged_objects.esp for a game in progress. Usually changing data of a mod while a game that uses this mod is in progress is not advised, but nearly all problems that this has (doubling etc.) are problems with *references* and how the game recognizes them. The merged_objects.esp never contains any references, it only contains object data. So manipulating the merged_objects.esp is much safer than manipulating any other mods.

Hmm, seems I rambled quite a lot. Anyway. I hope there was something useful somewhere inside. If something isn't clear, or if someone would like further info, just ask.

Wonderful post, Psyringe. Suggested for Yacoby's Archive so we will always have a copy:)

[Edit: I guess, if one had the "headroom" one could always run with multiple merged objects ESPs. Using your example: "merged_books.esp", "merged_weapons.esp", "merged_general.esp" and "merged_final.esp". As long as "merged_final.esp" loaded last then one would have the desired effect]

Ask and Ye Shall Receive.



This is all great stuff. Psyringe you are a fount of information. And yes I agree with Dragon32 that your post should be saved. That was full of good solid info.

I have alot of options now as well as some valuable knowledge when it comes to modding. Who knows maybe one day I might actually try to make a mod?

Now that I know what needs to be "object merged" using TESPCD I can finally feel more secure about that Merged_Objects file.



--joelR

Yes, the effect of running the individual files in this order should be identical to the effect of merging them in the way I suggested. Actually that is roughly what I'm having in my current game, which has four different esps with merged objects (because back in 2006 when I set this game up I had less knowledge about merging and TESTool's inner workings, and I haven't felt the need to merge these four files into one so far).

Btw, while I certainly appreciate it when my posts are valued, I'd like to point out that in parts I only repeated what others had already said, so if this thread is worth saving, then my post certainly isn't the only reason. Also, with regard to TESTool, my post is lacking some info about (for example) how to deal with esps that are patches to other mods (tricky subject), so there's definitely room for improvement. Any effort to fill the gaps is certainly welcome.