Ok, in the past couple of weeks there have been a LOT of discussions regarding converting ESPs into ESMs... And to be honest, I don't understand a lot of it.. so I am creating this thread in an attempt to get some discussion on this topic going.

1) Firstly, What are the benefits of having an ESM over an ESP?

-- In my view of things, there are three potential benefits of having an ESM.
i) Allows for multiple ESPs to communicate with each other as long as they are both associated with the ESM. Good for those who want to release multiple mods and want the mods to know if the player has one of the other relevent mods installed.
ii) Allows for other modders to be able to use your added content without using the mod which they are associated with. (example, you create a big mod.. which includes new weapons and armors.. other modders with to use your weapons and armor, but not the mod)
iii) Allows for plugins or patches.. say your mod creates a brandnew dungeon which is placed into the world - you could then have a handfull of ESPs which the player would choose from which determine various qualities of the dungeon.

2) Secondly, Is it better to have a single ESM file; or to have both, an ESM and an ESP file for a mod?
i) Obviously, having just one file makes it easier for a player to install and uninstall....
ii) but if that one ESM file contains a lot of Vanilla entries, then it will be overwritten by any other mod which also modifies these entries.



I guess my purpose in posting this is purely to generate discussion in a way which is understandable to new modders and allows for different approaches to this issue.

I'd hate to see a new player say "oooh, ESM seems better than ESP- therefore my mod, which changes levelled lists and alters the weight of all items in the game, will be an ESM because it is better." -- Because that modder would be severely disappointed to learn that not only would their mod not be better, it would almost certainly be completely overwritten by most of the popular mods out there... which could lead to frustration and giving up on modding.

I have noticed that lod isn't working with esp's when you have other mods as well. This would be solved (yet to try this out) when converting it to esm.

This is exactly the kind of thing which I would like more information on..!! test and post back specifics..

I'm interested in the general merits of esm's over esp's.

I'm skeptical of doing this just for the sake of it or on the expectation that this will make MODs somehow more reusable by other modders. One obvious advantage is that you can load esp's that are dependent on having the esm's in place. This is handy if there likely to be several such esp's made and used together, otherwise you are as well off just modding an esp to produce a new expanded version. I would say that there maybe some disadvantages to using esm's generally, e.g. requiring that your game be started from scratch, and so would not advocate using them with no good reason.

However, currently there are some re-load issues when you do a 'quick-load' of a game to the same area. In particular it seems some quest variables may get re-initialized unless you restart OB and reload. Using an esm may force your quests and persistant objects to be more reliably scriptable. (?)

Starting the game from scratch? Do you mean that you can't use an esm in an existing save game? If so, could you elaborate? I've never encountered this issue before and would be interested in the conditions that precipitate it.

I have to confess as to not knowing much about using esm's, which is why I'm intested in this thread. My understanding that esm it has to be in place form the beginning of a (new) game but I'm not sure where I got this info. from and it may be simply that typically an esm would be used when changing the vanilla game set up, e.g. when scripting base objects. Please feel free to correct me and give us the benefit of your experience.

I only used it once for the variable damage spell script. The idea was that the script implemented and debugged a portion of code that might be desireable to others so I made it an esm. I tested it by creating an esp that used it and then using an existing character to test the spell. I think I even forgot to unload the two mods so my sister ended up playing on for quite a while with them loaded. Neither of us noticed any problems.

I think I read on the Open Cities discussion thread that you can't use esms that modify existing content. Only to add new content. (They were unable to make Open Cities an esm because it was trying to replace the existing cities.)

^^^^
Thanks for this snippet. Unfortunately it does not add much to the discussion of why one would use an esm over an esp. Did you have to restart OB with the esm in place or did you just select as usual? (I assume the latter.) From this example I would assume that the only benfit was that coding in the CS was easier in that your esp could reference objects added by the esm? However, having never tried it, I'm not sure what happens when you are working on two esp's at the same time in the CS?

Anyone?

I've worked on two ESPs at the same time - when both are associated with the same ESM... didn't notice any problems on loading an existing game.

I've worked on both the ESM and an ESP which was dependent upon the ESM at the same time in the CS.. didn't notice any problems on loading an existing game.

But in both cases my playtesting wasn't very in depth - just spot testing for what I had changed.

And as for the Editting the ESM in CS - actually I converted the ESM to an ESP, editted that, Saved that, Converted back to ESM, then saved the ESP which was dependent upon it.

I'm not quite sure what you mean by this^^^. I feel like we might be talking about two different things. I thought you were wondering if esm files would not work with existing save games. The comment above was to illustrate that I was able to load the esm into an existing save game without any apparent problems.



More or less. I wanted to make the spell script reusable. It was a fair amount of work to implement and debug and I though that anyone else who would want to deal variable amounts of damage in any application (traps, weapons, or straight-up spells) might appreciate having a script that took care of the fussy parts (with my script, you set quest vars to the desired damage amounts, type, target, casting object (optional), and magicka cost (also optional) ). As you can see I went to a lot of effort to make it very general purpose. (And yes, I know you published a technique to deal variable damage, etc. I read the article after working out the mod.)

Because the mod was specifically intended as a modder's resource, it made much more sense to release it in esm format so that multiple plugins could be made from it without conflicts. This is a rare, but very good reason for using esm files.

As for the "two esp's at the same time" question, I assumed that multiple esps would be able to reference the same esm in the same way that they do for the main oblivion.esm file. I'll be done with my finals soon and will verify this when I get a chance.

Exactly; an .esp can reference or change anything in an .esm that it's dependent on.
The caveats to using an .esm are that you can't include changes to existing landscapes without royally screwing up the terrain, and more generally you shouldn't use an .esm to alter any records from Oblivion (or another .esm).
Apparently .esms may not even be necessary any more, if Malbulga's and Wrye's experiments with .esp dependencies are anything to go by. There's a recent thread about it.

You know what's really annoying? THIS problem http://www.elderscrolls.com/forums/index.p...501&st=160# which involves one of my characters games 'eating' anything that is dropped in the gameworld, saved and reloaded after Sunday...the issue got resolved when I converted OOO to an esm, but it sucked, because there was nothing from Oblivion.esm in the gameworld. So still not a fix.

Any ideas about my reloading = vanishing items (but only after Sunday) problem with this one character btw? Or how I could get around the esm conversion issue?

I've only used ESM for one purpose so far... for the Book Placement mod, I wanted to reduce the amount of duplicate data that the game had to load, so I separated out McLean's scripts and the one new book added and put those items in an ESM. This ESM can be called to assign the scripts by my Book Jackets mod or the vanilla books esp or anything else. I know at least MrDarkSim has made use of it in his book mods.

I'd like to see Harvest[Containers] doing the same thing, so that it could return to a modular format without so much fuss involved in the maintanence. The scripts would still exist in only one file so that's the one that would need maintaining, while players could decide again which types of objects they wanted to use.

Thinking out loud... Expect some meandering with no clear conclusion...

Types of mod interaction:
1) Resource mod. Is specifically intended to be used as a resource for other mods. Modifies nothing in vanilla Oblivion. Is very stable. Possible examples: Eye/hair collection; room tilesets for dungeons; useful scripts; gameplay resource like Necessities of Morrowind.

2) Signalling mod. A mod that's specifically built to pass information between mods. This is "before the fact" integration. Morrowind example: Tokens and globals used for signalling between Vampire Embrace and other mods. I don't know of an Oblivion example.

3) Patch mod. A mod which is specifically designed to patch another mod(s). E.g., add additional portals to Frostspire, or remove the starting messages from the various official mods.

4) Glue mod. A mod specifically intended to glue two mods together. Typically this is relatively small compared to the master mods. E.g.: Dev_akm's FranOOOMMM leveled list mod. This is "after the fact" integration.

It's only recently that we've realized that esps can modify other esps, thus making 3 and 4 viable for esps as well as esms. (Qualification: PhoenixAmon reports that ParasiteX(?) figured it out months ago and used it to create a popular vampire patch mod, but didn't make a lot of noise about it, so it didn't become widely known.) Though this is still somewhat in testing, it looks good so far.

Types 1 and 2 should be esm mods, while types 3 and 4 should be esps. Given the possibility of esp-esp patching and glueing, it is not necessary for the master mods to be esms.

There is the consideration of which is better: pre-integration or post integration (i.e., signalling or glueing). The problem with signalling is that it requires a fair amount of coordination between different modders -- and most major modders are monomaniacs. So, realisitically glueing will happen much more easily than synchronizing. (Even in Morrowind, where signalling is relatively easy, there was very little of it. And by very little, I mean almost zero. However, glue patches were moderately common.)

To give some Morrowind examples: 1) MCA and NOM conflicted over beds. Solution was an MCA-NOM glue patch included with the MCA release. 2) A global for pc gender is used by several different mods. There was never an agreement on a standard global. 3) PC race number for custom races (used for dialog) might conflict. There was never a standard solution. 4) LCV Schedules conflicted with VampireEmbrace and other mods that had regular NPCs become followers. Cortex (VE) and I (LCV) implemented a signalling solution. 5) For new home/land mass mods to work well with NOM, they needed to include NOM resources (scripted wells, etc.). Some major home/land did, but most did not. (I did add a feature to Mash which allowed players to NOM-ify a mod, but I don't think that it was widely used.)

So that's my experience... Mod integration (to the degree that it happens) tends to be a second generation effort, often done by someone other than the original modder.

So, what's that leave? I think that if esp-esp interaction continues to work, then the only reason to esm-ify a mod is to split off the mod's resources so they can be used without otherwise using the integration of the mod into the world. E.g., to have OOO objects without having OOO otherwise in the world. Hmm... I don't know, seems like a lot of work for an only moderately desirable goal.

Splitting a mod into two pieces (esm and esp) has pros and cons.
* Pro: It's easier to build on top of an esm. (However, Bash's espify masters command helps quite a bit.)
* Con: You use two mod slots instead of one.
* Pro: You can use the resource without using the esp. (But I'm not sure how desirable this is.)
* Con: Two mods are more difficult to update and maintain than one.
* Con: Greater chance of bad mixes, e.g., of an esp being used with the wrong version of the esm.
* Pro: Memory consideratations. I think I'm hitting the ram limit (1 Gb) on my machine now that I've got the newer version of OOO (vs my original 1.23) plus KOTN plus other official mods. Havin separate resource mods might make that less of a problem (assuing that I don't load the esp of course).

Standard library esm: Rather than having umpteen different resource mods, have just a few (maybe just one) to cover stuff that people really want shared. However, I'm not mod savvy enough to know what would go into it. As I suggested before, maybe tilesets, a NOM type game resource?

Like I said, I've just been kind of thinking aloud on this one.

Good thoughts there, Wrye.

Some other things to consider.

* The CS. Having clean ESMs (with few if any changes to 00 records) is the only practical way to build "glue" ESPs. There's just no way around this yet. Sure, it's not necessary to distribute the ESMs if you're willing to hex-edit the MAST record after you're done working in the CS (to fool the game engine into thinking an ESP is really a master). But this means you have to create your own clean ESMs for any mods you want to enhance/extend/glue, and so does anyone else who wants to work with the same master like you did, etc. This adds up to a lot of wasted effort in the long run.

* Maintenance. Especially ease of maintenance for option plugins. Both Fran's and MMM make very effective and extensive use of optional ESPs to modify certain aspects of the master. This makes those mods dramatically easier for the creator to maintain, and at least MMM is already benefiting greatly from a bunch of community-created add-ons. You can probably do the same thing with the hex-edit bait-and-switch method, but why would you want to if you have the choice?

* Longevity. You will eventually get burned out and move on. It happens to all modders (maybe a few exceptions, but not many). Do you want other modders who may carry on the torch for you after you're gone to be forced to copy your work and re-release it, or would you rather that they mostly just create option plugins to extend and improve your original work?

* Size. Silgrad Tower and several other very large projects have discovered size limits to ESPs. This limit doesn't seem to apply to ESMs.

* Teams. Large mod projects like ST rely on ESMs and then use merge-to-master to fold the new work of each team member back into the baseline for each phase of the project. This is essentially the same process Bethesda uses (although we're obviously lacking some of the tools they have) and there's a good reason for it. A sound development methodology like this is critical for any large team project to succeed (and it's also beneficial even for smaller team projects).

JOG's thievery esm or Retro's Crafting esm might fit that category. They provide a base that the author has already built one mod on and expects to build on further with more mods in the same series. Other mods could reasonably hook into the same esm, at least in the case of the crafting mod.


I suspect that ParasiteX didn't report that he'd done something new because he didn't know it. If you don't know something's thought to be impossible, you don't think much of it when you accomplish it. Although it's possible that he reported what he'd found and no one listened. Since I discovered what he did in that mod, I've mentioned it a few times on the forum, and it never caused a wave.

Similarly I reported that mods couldn't interact on this forum a week or so after Oblivion was released, and that post sat without reply for months and finally evaporated. Maybe a month later, you reported the same thing and then everybody knew it. Sometimes it's not what's said, but who says it. (This isn't meant to be rude or argumentative... it's merely true. You were a known respected member of the community and nobody had ever heard of me, and people are more likely to respond to those they know. I do the same thing, and so can't be offended when others do it.)


This is the one I'm most focused on, lazy beggar that I am. If I can just convince Dejunai that I am more evil than he is, I may win out.

PhoenixAmon: Known, and I posted it on a couple of places and PM'ed some folks too I think. Anyway, credit now given! And, I think that you're right about PirateX not knowing it was impossible. However, remember too that at the time we were figuring out mod isolation, we didn't have the tools to easily try the bait and switch trick. (I.e., I kind of suspect that he used Bash to do the bait and switch trick -- so I get partial credit for it anyway )

By signalling I mean something mod that is specifically designed for signaling between mods -- i.e., that would otherwise not be necesssary. For LCV, one signal was a global that LCV watched in dialog, another was an "OffSchedule" token that an NPC might have. If an NPC has the token, LCV doesn't apply schedules to them. Neither of those was something that LCV needed for itself, they were both there purely to support integration with other mods.

I think what you describe with thievery or Retro Crafting is more of a resource mod -- i.e., it provides resources, which another mod can also use. (But I'm not familiar with the mod, so I could be wrong.)

But I think that I'm quibling over details here...

dev_akm: Well it's not necessary to hex edit anything anymore. Suppose I want to glue mods Alpha.esp and Beta.esp together...
1) Use Bash's "Copy to esm" command on both Alpha.esp and Beta.esp. This creates esm copies of both mods, while leaving the esp originals intact.
2) Build glue mod Gamma.esp with Oblivion.esm, Alpha.esm and Beta.esm as masters.
3) When ready to playtest Gamma.esp, use Bash's "Espify Masters" command on it.
4) After playtesting, if you want to re-edit Gamma.esp, use the "Esmify Masters" command on it.
Then repeat 3 and 4 as necessary.

A possible improvement would be to have a Bash command, which than rather applying "Espify Masters" directly to Gamma.esp, first copies Gamma.esp to "Gamma X.esp" and then espifies "Gamma X.esp". That would allow modders to skip step 4. The only problem with this would be with mods with BSAs and/or voice files, which are linked to the exact name of the mod. (There are ways around this, but it's a complication.)

Longevity Hmm... I'm not sure that I agree with your point. The esm/esp split is more imporant for allowing variations of a mod (no small benefit), than it is for supporting upgrades. I think that if someone came in and did a substantial upgrade, then it would probably be necessary (or at least strongly desirable) to update the esm, rather than just release a patch. My MW experience (updating LGNPC Seyda Neen, NOM and LCV) all supports that -- for each of those, changes were pervasive enough that updating the original mod was more desirable than releasing a patch.

Maintenance, Size, Teams All good points. No counterarguments there.

Summary We're certainly expert heavy on this topic! And in truth, I haven't done that much modding for Oblivion, so my expertise is less than others. I still think that patch and glue are easy enough to do for esp mods, and I would strongly lean to doing that. However, a lot good points about moving to resource esms have been made.

I am sorry if I sounded like I was talking about giving credit... I only meant to point out that sometimes things go unheard because we listen more closely to those we know there's a good reason to listen to. That's a good choice... a person can read only so much news, so you read the news you know is from a reliable source. You deserve loads of credit. There are plenty of things we simply can't do without Bash, and while someone like me can see a problem and describe what they see and complain about it, we can't explain why it's happening. And failing that, we can't hope to change it.

Actually when I first saw what ParasiteX had done, the first thing I did was look at Bash to try to figure out how. I hoped to avoid hex editing a single byte, and now from what you describe I can do just that.

Ah, yes, I see what you mean. The types of mods I describe are more for providing a framework for expansion by the author. They're not exactly meant as a public resource, but an internal one. Still, resource fits.

Just rambling a bit now...

I think ParasiteX's reason for use of the bait-and-switch is likely the best one... you want to provide interactivity with an existing mod as a third party. This happens after the fact, and you do not control the mod you want to interact with. If that author has not provided an esm, you have this option to provide compatibility.

You might also do the same thing when you do have control of the original mod, if the mod you want to use as master is something you've already circulated. If you update your old mod to esm so that you can make it interact with your new mod, you may cause problems for people who are already using it. If they upgrade to an esm version the name of the mod has changed, so variables stored in their savegame may not be carried over. Ownership of mod-added items may not be carried over. Rather than tell the users that they need to start from scratch if they want the add-on, you can use bait-and-switch so that they don't need a new version of the original mod.

It's taking me a long time to type this, because I keep thinking of permutations and "what would I do if...". If you're building a mod that you know you plan to produce in installments or if you're sharing a pure resource, these are the times when I think you'd be most likely to build an esm from the start. For a large mod like Fran's, once you go through the splitting process, I'm not sure if having the esm is useful anymore. You'd be putting so many modified vanilla forms into esp files that you'd have to use bait-and-switch for you add-ons anyway, so is separating them out really worthwhile? Having and esp master and an esm master rather than just an esp master? My brain's gotten stuck on that example, so I think I'll stop now.

Thanks. All good stuff to know. I was actually referring to 2 esp's being edited at the same time w/o an esm. But I assume that you cant reference stuff in one esp (in the CS) from another.(?) That is, they would both have to reference a 'parent' esm and that's basically why you would use one.(?) One o'these days I might get around to trying this stuff for myself - basically when I can work out a reason for using an esm wrt my mods.


I see. That would be a couple of reasons I guess, especially the idea of issuing a large MOD as components that all shared the same esm.

Just got caught up on reading most of this thread.

You guys should be a bit more explicit about 'the MOD isolation problem' and the solution discussed for esp's - there are no good links anywhere on this thread. It sounds like it could be very important or exceedingly neice. (I started threads on the subject of inter-mod (esp) communication a long time ago with recommendations for how to go about this but I've no idea if what's discussed here is in anyway related?)

Please add a little simpler discussion for those of us that haven't seen/followed these threads.

Only one of the mods you're working on can be Active in the CS. So you've got mod A and mod B open, B is the active one. When you have both open in the CS, you can use items from mod A in mod B. It feels like it's working. Everything appears just how you'd expect. Then you save mod B, and try to load it in the game or CS and discover it can't remember where it got those references. Nothing but errors for everything that was referenced from mod A. If you use Bash to add the "master" reference to mod A, I believe you'd resolve the errors. Now the engine would know where to look for the resources. But I haven't tried that yet.

EDIT: For a specific example, the first OB mod I did was add some new eye colors. Then I wanted to use a second mod to assign those eye colors to some NPCs. I opened the eye color mod in the CS but didn't make it active, and set about making a new mod assigning the eyes. It all looked great until I saved and closed the CS. It didn't work in the game at all. It couldn't find the eyes assigned. I can't remember whether I got error or if it just crashed, but I do remember that it plain didn't work.

I didn't see whether anyone had answered the question about esm's needing a new game. They don't. An esm can be added to a game in progress without a problem. Certain types of changes do require a new game (I think changing merchants' spawn lists is one of them) but that issue exists no matter what mod format you use.

Thanks a bunch phoenixamon. Didn't want to seem stupid but with reems of text my brain starts to panic or something - I've always been a bit dislecix.

I've used multiple MODs in the CS (I've only have 3) but just for copy/pasting - always been affraid to find out what happened if I actually tried to merge them. Good to know I did the right thing in playing it safe here. (I'm very goal orientied - I'll find a way to do whatever I need regardless of difficulty but never try stuff just for the sake of it otherwise.)

Good tips! I didn't know about the Espify Masters/Esmify Masters function, so I haven't tried it yet, but from looking at your readme page it looks like this would work great if you don't mind re-arranging your data folders a bit before you run it. I'm assuming it makes the switch automatically for each master (i.e., without prompting the user) based on the files it finds in your folder. For example, if I have:

Francesco's leveled creatures-items.esm
Mart's Monster Mod.esm
Mart's Monster Mod.esp
Oscuro's_Oblivion_Overhaul.esm
Oscuro's_Oblivion_Overhaul.esp

Then if I want to make a new plugin that uses these masters but bait-and-switch the Oscuro's_Oblivion_Overhaul.esm to use Oscuro's_Oblivion_Overhaul.esp instead (since the ESM has not been publicly released), then to use your function I would need to first remove Mart's Monster Mod.esp from the directory (since I don't want to change that master reference). Right?

I think you're better off leaving Step 4 in that process.

It would be really useful to also have an "Add Master" function in Bash. I currently have to do this in a hex editor (TESsnip has a bug that can sometimes mangle your mod), so it would be a very useful feature for anyone trying to follow my Creating Clean Masters tutorial. I guess this request would be better in your Wrye Bash thread, but it might not make much sense without the context of this discussion.



I agree that variations are a much bigger point, but consider the case of OOO for a moment. I've been supporting it while Sotobrastos is away, but I haven't been able to reach him to get permission to release some bugfixes. This means I have to release binpatcher files, which are a royal pain (takes forever to compile and confuses the heck out of the kids). It would've been massively simpler to just release an ESP with a few fixes in it. If I do it again I'll probably use the bait-and-switch method.

Also, although I agree with your point, longevity isn't just about upgrades and fixes. If you have a mod that adds some really cool new content/feature/tweak/whatever, chances are that eventually someone will come along with another mod that adds some other really cool new content/feature/tweak/whatever but is incompatible with yours on some small but critical points. If you're not still around to upgrade your mod, and you never released a clean master, then it's going to be significantly harder for somebody else to come along and create a bit of glue to fix this incompatibility. Not a huge problem for most mods, to be sure, but certainly worth thinking about.

Dual Editing
Open two instances of TESCS. Edit the resource master in one and the addon master in the other. Then save both and reload the addon when you need it to use the updated master. Definitely not as convenient as doing all your work in one mod, but it's doable.


Correct.

Adding Master Function in Bash:
Well ordinarily, if you add a master, you need to jack up the formids belonging to the mod itself by one modindex number. This is pretty nontrivial operation. ScripterRon figured it out for his tool, but I'm trying to avoid doing it. But if you're thinking of doing this in the context of splitting out material to a master resource esm, so that the modindices don't need to be jacked up... Hmm... Just adding another name to the master list is trivial -- I'm only concerned about safety for general Bash users. Anyway, if that's the goal something can be added to Bash one way or another.

Previous discussions for those looking for more background
* ESF: Mod-DeIsolation
* ESF: Dealing w Dirty Masters
* UESP: Mod Integration
* UESP:FormId
* ESF: FormId Problems

Yes, specifically, this is known as "Master Injection" and is needed when you split a mod into an ESM/ESP pair. No correction of FormIDs is needed. Just add the MAST record to the TES4 header and update the record count. All records that were previously local (i.e., new content) will now be considered change records for the new master you've injected. Any further new content you add to the ESP (when working in the CS) will now be given a higher modindex (+1).

BTW, Timeslip has just released a fixed version of TESSnip that shouldn't lose data anymore, but it would still be a lot easier to do it in Bash using something like the interface you already have for altering savegame masters. TES4plugin lets you remove masters, but does not let you add them.

dev, do you happen to know where I could read more about "Master Injection"? I keep seeing the term, but what you've just said is the closest I've yet seen to a description.

Also wanted to say thanks to everyone for a good discussion. This thread has got me thinking a lot. I feel less sure now that I'd know what to say if I were writing a generic list of "best practices" but more sure that I'd be able to decide what to do in a specific situation.

Fine. Okay. "Add Master..." now available in Wrye Bash 0.36. (Note that it's on the context menu for the mod itself, not on masters list details.) It was easier to code that way.

Oh, and I threw in a merge leveled list feature too.

LOL. Sweet! You rock!



The only one I know of is my Creating Clean Masters [TUTORIAL]. If that doesn't explain it well enough, then let me know and I'll try to expand it.



Agreed. Good stuff all around, folks!

As I've hinted at the Bash merged list thread, I've developed a feature which will build alchmemical catalogs updated to the currently active mods. Here's how this will work:

Base Mod: Alchemical Notes.esp OR "COB Main.esm"
Bashed Lists, xxx.esp

The catalogs are scrolls which are defined in the base mod. Then, during the bashed list merging process, I rebuild the catalog text based on the actual loaded mods and insert it into Bashed Lists, xxx.esp so that it overrides the original version. Basic code is already done. I just need to clean it up and integrate it into the GUI. Here are a couple of excerpts:

So, policy question: I could just go ahead and have an "Alchemical Notes.esp" mod with maybe some minor quests, etc. to ge the mod. But I'd like to take this opportunity to start using a common library mod, which we've discussed earlier here. E.g., rather than having my own special mod, I'd like to use something like "COB Main.esm" (COB = Common Oblivion).

As far the alchemy catalogs are concerned, COB Main would only contain the catalog scrolls (a separate esp could be used to place the scrolls in the world). However, for this to be worthwhile, I want COB Main to become a general standard which can hold various "before the fact" integration data (basic leveled list definitions, etc.). I.e., it needs to be a group project.

Who's interested?

Nice! Great idea, man! I'll be happy to sign up for this effort.

Actually, this is very similar to what I was proposing in my original spec for a split/merge tool. The COB Main.esm would essentially serve as a template for other modders to use when they want to add stuff to leveled lists.

Martigen and I are already in the process of working on something very much along these lines in order to create a better MMM for OOO mod (more closely integrated than the current FranOOOMMM version).

The big question in my mind is whether this can be done with a single "template" (i.e., list structure) or if we will need several different ones to accommodate various design goals. It seems there are at least four popular approaches right now:

1. Fully Leveled Random (Fran/vanilla style lists)
2. Random Risk/Reward (MMM style)
3. Static Risk/Reward (OOO style)
4. Mixed Static/Random (compromise, ala FranOOOMMM!)

Perhaps if the base lists are structured correctly, we could manage to have only one baseline and then several optional variant ESPs that adjust the level settings and counts to match the player's preferred style.

I'm not sure this list includes the Critters model.

What do you think?

As an aside, Wrye, for the next time you update your site:

Latest version of wxPython is 2.8.0.1.
The unicode version definitely doesn't work, at least with Mash. (Haven't tested with Bash, no much point to it since my laptop can't run Oblivion anyway. Stupid shader-allergic Intel chipsets.)
Unicode error report.
Worse, I get that even when I tell the ANSI version to be the default wxPython. wxPython unicode has to be completely uninstalled before Mash or Bash may work.

Gez: I've replied on the Wrye Bash topic.

COB Main.esm: So, should I just go ahead and do it? (Might wait a day or so to see if anyone else has comments.) My first version would just have six to eight scrolls in it. Other stuff (e.g., leveled list templates) can be added later. (A strong consideration is that stuff should not go into COB main until we're pretty sure about it.)

Leveled List Templates
If you want to accommodate different leveling styles, then I think that the best solution is to have a single template with a consistent style (default to Oblivion style, I think), and then have esps that patch that default style to other styles as desired.

In doing this, I think that you could simplify the amount of patching work by leaving the sublists unleveled (e.g., all at level 1), and letting the highest level lists do the leveling work (or not do it as style dictates). (But this is not my area of expertise -- I'm probably either stating the obvious or missing some equally obvious flaw.) However, you set it up, you need a guidelines for modders who want to add stuff to leveled lists about how/where to add their items.

Now, suppose that a modder adds his cool new weapon set following vanilla Oblivion leveling rules. How do you morph that to a different set of leveling rules? Well, he could supply patch esps that patched the mod to each of the different leveling styles.

Or the end user could use Bash to automatically apply a leveling style while merging. E.g., Bash could take a given leveled list (after merging) and flatten, scale (current level x 0.75), cap (max level == 20), randomize, or partially randomize the levels as the user specified. It wouldn't be desirable for Bash to do this to ALL lists, but rather only to a specified set of lists. Note that for such scaling to work, the original lists need to be leveled in a consistent way. In particular, the source list can't be flat or randomized (have random levels, I mean). For bash's style morphing coding, scaling, randomization, etc. would all be pretty simple formulas. However, with custom python code you can mangle the lists in much more interesting ways of course.

For example: Random Population: Suppose that families of NPC lists are named in a consistent way, e.g. (Amazon10, Amazon20, Amazon50), (Bandit10, Bandit20, Bandit50), etc.. Bash can now recognize each of these as a family of lists. Now, suppose that a modder puts spawn points in cave "Foo", but rather then putting a particular type of spawn (e.g., "Amazon") at that point, instead uses empty lists named like (caveFooXX10, caveFooXX20, caveFooXX50). Now, Bash merge comes along and what it does is randomly assign a single spawn family to the caveFoo spawns (e.g., Amazon10 to caveFooXX10, Amazon20 to caveFooXX20, etc.). Now you have a cave randomly filed with a consistent type of NPCs. Do the same thing with campsites, etc. Do this and you'll shuffle your Oblivion experience every time you do a Bash Merge. Note that cave Bar a couple of cells over would have different spawn names (caveBarXX10, etc.) so the family that it received would be randomly different than cave Foo.

The last is something of an alternative to PirateLords's approach. The plus is that it's somewhat simpler and more flexible. The minus is that it requires using Wrye Bash merge whenever you want to swap populations (i.e., you have to step outside of the game world).

And if some people are foolish enough to refuse to use Bash, it's always possible to someone else to bash the lists together and release that.

Wrye: Yes, go ahead and make your COB. We can add to it as we go.

I'll give some serious design time to the template issue in the morning (just woke up for a snack, need to go back to bed).

Corepc: I'll also put some time in on the issue you raised about making neutral factions. This is a pretty big issue with merged lists and it might actually be an area where something like COB could help out. However, this would probably need to be a tool for mod-makers rather than mod-users I think. The problem is we almost certainly do not want to try automatically generating a massive integration mod that copies NPC records from all the merged mods.

Just random thoughts here. Perhaps a single NeutralFaction.esm could be shared among all the major mods and then you could use the batch edit function in ObEdit to easily assign this faction to all the right NPCs? If you got Fran, Martigen, Soto, etc. to all include this as a standard element with their mods, then their NPCs could already be members of those factions and all you'd need to do would be add yours as well. Wouldn't this work?

I really like this idea. I've been hoping that a nice "Oblivion Mod Standard" would come along for a while, to lessen the amount of conflicts people have with larger mods. Along those lines, having a nice "Community Expansion Mod", along the lines of the CEP for Neverwinter Nights would be phenomenal, more for a modders resource type idea, so they can put new items/areas/characters/scripts etc. into the game world as they wish all from one large mod so they don't have to have millions of small mods adding each seperate thing, with the possibility of overwriting other files. I've kind of been trying to do it myself with just the mods that I have, and boy is it a "fun" undertaking... Keep up the good work!

Shouldn't these factions be defined in COB, then?

Yes, I was just about to bring up factions as well.

My personal interest in the issue is that atm I'm making house mods. All my house mods have extra bedrooms/sleeping areas, in the event that the player has a companion, or more than one, and doesn't want to make said companion or companions sleep on a bedroll on the floor somewhere.

But I cannot, of course, know which companion any user might be using-- I don't necessarily know which companion(s) I might be using in any given game.... and all the companions are different factions, so I can't even designate a bed as useable by a "Companion" faction and hope that any companion, whether a CM Partner, or Gabrielle, or Vincente/Valeria, or Hunl'aa/Elwen, or Amoz (Dremora Companion), or a companion created by Unique Companions, or only Dexwood's Daughter, would recognize it.

If at least companions had a single faction, contained in such an common ESM, it would enable "me" or the modder or whoever, to make all the companion mods dependent on that ESM and set all the companions to that faction, which would 1) help companions get along even if they were from different mods, and 2) allow me to designate beds in my houses to that faction easily and quickly.

The more complex faction issues occurring in mods like MMM I don't know about, but I do feel that some elements of 'factionalization' should be under consideration for communalizing.

Dev - Yes the single Neutralfaction.esm just along these lines that could be used by modders. Just not by npc, but, for creatures, objects, basically anything that you can set ownwership over.. like motub said..

Give it some thought please

I haven't been replying because I don't feel I have clear understanding of what's being discussed at this point. Since I'm not getting a whole lot clearer on the matter, I'm just going to post my thoughts and see if you can set me straight.

On the issue of the COB.esm file, I'm not really sure whether I understand the idea. I imagine it becoming a large resource mod that adds items and can be called by other mods to add those items to the game world. The first example we have for this is Wrye's ingredient recipe scrolls. It might also have some creatures and some furniture and a collection of activators to emit creepy sounds. So say I want a new hounted house mod, and it pulls on some of the creatures and some of the creepy sfx. I don't choose to have a mod that uses the recipe scrolls. If this resource really becomes big, might I not be loading a lot of data that I'm never using because of my mod choices? And if I am, will it matter much?

I do understand the GenericFaction and CompanionFaction ideas, and those would both be excellent resources. Is this the basic idea behind the templates? I'm just not clear on what templates means.

COB Main.esm Contents: I think that we'll need to figure that out a little bit. I would like COB Main.esm to be kept relatively small, but in truth I'm not sure how much this matters (many esm entries don't use up that much space in themselves).

Here's what I'm thinking right now:
COB Main
* Alchemical scrolls (8 items): These will usually be overridden by a later esp (Bashed list.esp)
* Generic Factions
--* Companion faction
--* Neutral faction??
--* Amazon faction (etc.) (Maybe)
* Expanded Lists (Blank in esm, filled by creature mods, later merged by Bash)
--* Amazon10 (a low amazon)
--* Amazon30 (intermediate amazon)
--* Etc.
* Standard activators/scripts
--* Bed, well, scripts, templates (for HTS (Hunger, Thirst, Sleep) type mods)
--* Standard companion script??
--* Standard well, water bell activator (again for HTS type mods)

Most of that stuff is just an idea at this point. The only thing I'm sure about at this point is the alchemical scrolls. I'm fairly sure that COB Main would NOT include creatures -- though it would probably include standard creature lists. I'm not sure about something like standard tiling sets. I think maybe a separate "COB Tiling.esm" for that -- if such things ever appear.

You could argue that COB XXX could be quite large, which would be useful for people building house/area mods. E.g., more books, more ingredients, more furnishings. I'm just not sure where to draw the line between what COB adds and what various mods (OOO, etc.) add. And those other mods are already being "resourceified". I think that will come out of discussion.

Anyway, I'll set up a project/discussion area somewhere later tonight when I have some more time.

I've searched high and low in this thread, and found the info about what COB is, but where the hell did the acronym come from???

Edit: aah, finally found it.

This sounds good wrye,

I have been working on my own esm / esp for my mod and it is working well. In my mod I place my merchants/npcs, creatures, xmarkers directly into cells. These changes causes problem with in and esm but not in a esp format..So, I have had to strip and rebuild using mainly the editor with tes4plugin to strip out data and copy over to a new esp the world data and x markers and packages that point to these, but, I have everything work just fine has along has world cells changes are kept in and esp not an esm...I just re add the npc from the esm to the esp with the ai packages and replace at those locations that are needed..It worked out quite well..

Thanks for this thread and these ideas

Corepc

Okay, I've started a discussion page at UESP: UESP: Common Oblivion. In particular, note the Proposed Additions section of that page. If you want something added, please suggest it there. I've put in a brief discussion of Companion Faction, and I've just put in a header for the Neutral Faction -- I don't really understand what the point of this is or where it would be used. CorePC and motub, I hope you'll expand both of those sections.

There's a plus and minus about UESP wiki. The minus is that it's another place to go to and a slightly different editing tool to use (but still pretty easy, especially for plain text). The plus is that it's easier to keep the discussion in place and pretty focussed. E.g., on this forum, I might reply to three different suggestions in one post -- which means a reader has to go back and put the pieces together. Whereas on the wiki, all of the comments on a single issues (e.g., a companion faction) are kept tightly together. Plus, by indenting your replies to other folks, the flow of the discussion is made pretty clear.

Thank you Wrye, I will check the page out..

The main purpose of such a faction.esm would to be able to assign ownership over cells, npcs, items, whatever, that can be used by the faction and it could be used by all for a large community project..in short!!

Okay, folks, I'm going to head over to the UESP Wiki in a few minutes and see what's up there. I haven't had a chance to post much on this yet only because I've been quite busy with holiday preparations and other projects, etc. (more on that below).

KomodoDave and myself have just finished the beta version of an extension to ScripterRon's TES4 Plugin Utility that will let you automatically create a clean ESM/ESP split mod from an ESP. KomodoDave has done a fantastic job on this (with a little help from me) and it's looking really good already. I was able to split the OOO.esp into a clean master/plugin pair and load it into both the CS and in-game without any errors or savegame problems, so I'm really excited about this new tool.

I mention all this here mainly because the ability to easily create clean master files is critical to more widespread mod integration and compatibility, much along the lines of what we're already discussing here. KomodoDave and I will post more details on this new tool after I contact ScripterRon regarding how he would like to see it released (i.e., whether he wants to post it as a new version of TES4 Plugin Utility, which is basically what it is, or if he would rather have us branch the functionality into a separate tool release).

I'm sure many of you will be away visiting family and such over the holiday weekend, as I will be myself, so have a merry xmas everyone!

this should people out thank you KomodoDave and Dev_Akm, everyone have a jolly holiday!!!

Heh. Well there you go, I have posted to a Wiki and the sky has not fallen. Yet. I'm still looking up.

Wiki: Thanks y'all. Looks like it's starting out nicely. I moved stuff around slightly just the so the page design is kept consistent. Combat tokens like a good idea, but it's outside my area of expertise, so I won't have much to say on it. And thanks for the links dev_akm...


Well, ummm... Basically you just described what a "faction" is. Why would there need to be one "neutral" faction that would be shareable by multiple mods?

For companions mods, I get it. You got a bed, and you want whoever your current companion is to sleep in it. There are a lot of companion mods, and you want them all to have the same faction so they're all ready to sleep in that bed.

But what's the corresponding argument for a "neutral" faction? Why would it be useful for different mods to share this?

Now I can see arguments for various types of standard factions e.g., Amazons, or Daedric. E.g. mods other than OOO might define their own Amazon npcs, but otherwise not build on top of OOO. So a common Amazon faction would be useful. By why a generic "neutral" faction. What types of creatures/npcs would belong to it, etc.

YW

I think rather than a GenericNeutral faction, what Corepc is looking for specifically is a TravelingMerchant faction so that his merchants won't get killed by bandits and marauders. Yes?

Hehe. still have a split conversation. I just posted my thoughts on the neutral faction to the Wiki.

I don't speak Wiki, so be patient with me while I try to get my bearings over there.

Well If one common neutral faction was shared by serveral mods, then any one of companion mods could use this faction for other mods, anything that a faction can be added 2 could be shared, houses, npcs, chest, objects, those types of things..and the faction relationship can be altered through and esp

Example: Mod Maker take faction above and makes a new mod, using the common.esm faction, he then could alter the faction relationship with an esp and with thus creates the so called actions that are needed, if they wanted bandits to attack them, or amazons, or whatever from any mod..

this is what I have done in my esm / esp split for my merchant mod..Dev_AKM & Wrye

Thank you for this thread and USEP page

I've added "Bandit Protected Faction" at UESP. I think that this plus the Companion faction would cover what's been talked about here.

If that doesn't cover what you're envisioning, then I think that I'm still pretty fuzzy on what you're suggesting.

PhoenixAmon, you're doing fine. I just have to keep remembering that you're PA here, but Daleth there. Please pardon my slight reorganizations, it should stabilize before too long as people get used to the wiki and we figure out what the main areas of discussion are.

The reorganization is fine. I'm just feeling my way around and sorting out how to do clever things like links. I feel like I'm doing the Wiki equivalent of a "Hello World" excercise.

I have the read the UESP and that is exactly what I have envisioned, a faction that does not fall under attack from bandits, marauders, etc..Like they have paid them not to attack them..Expect daedra course, when the gates open up the mess up the economy and world!!!

thank you Wrye and Dev_Akm!!! have some

Corepc

Here's my general thoughts: 1) I don't want to kill Cob with too much process. It should be possible to add stuff fairly quickly. 2) On the other hand, I don't want to risk throwing stuff in that's a mistake, or for which there's going to be a lot of dispute over. And 3) I don't want to do too much work! So, with that in mind, when it looks like discussions are might be long, and/or is outside of my area of expertise (which may be much of the time), I'll ask other folks to take the lead.

Also, the wiki tends to be best for short to medium length discussions (and final conclusions/information pieces). So, other forums may be useful for various discussions.

So, with that in mind, I'm asking:
* Behippo to take the lead on OBSE support in Cob on the forum of his choice.
* Daleth/PhoenixAmon to take the lead on alchemy sorter on the forum of her choice.

When/if there's a consensus on these, I'll add whatever is suggest to Cob Main.esm (or maybe to another cob mod, if that makes more sense).

Sounds good, man. I'm still working on getting some other recruits in here.

I'm also going to attempt a mock-up of some leveled list variations as I get time over the holidays.

Being as I have no idea where to begin a discussion about this sorter business, I'm just going to start doing what I know how to do.

The idea seems to be to create a sorter that can pull all of the ingredients from your inventory and put them in a single container for storage. When retrieving them, you can either just open the container and take them or you can choose a desired potion effect and sort out only the ingredients that have that effect visible at your skill level.

There should be 2 versions: Vanilla and OBSE. The OBSE version would be in an ESP file that overrides the Vanilla version in the COB esm. If a modder attaches the vanilla script to a container in their house mod, and the player has the OBSE esp installed, he'll get the benefit of the OBSE script. The modder only has to create one version to provide both options.

If anyone disagrees with this summary, kick me (gently).

I suppose a lot of people aren't going to be around for the next week or so. I will because I'm boring. But for those who are doing the family thing, here's hoping your obnoxious Uncle Fred falls just ill enough that he can't make the party.