ProjectsWhat's NewDownloadsCommunitySupportCompany
Forum Index » S.T.A.L.K.E.R.: Clear Sky Forum » Mod discussion
Stalker Mod Change Script (Modify The Extracted Mod .ltx Files Automatically) - Discussion

1 2 | Next 10 events »| All Messages
Posted by/on
Question/AnswerMake Newest Up Sort by Descending
  17:59:38  4 June 2009
profilee-mailreply Message URLTo the Top
JakFrost
(Senior)
 
On forum: 11/30/2008
 

Message edited by:
JakFrost
06/04/2009 18:01:22
Messages: 112
Download Thread: [link]https://www.gsc-game.com/index.php?t=community&s=forums&s_game_type=xr2&thm_page=1&thm_id=20922&sec_id=21[/link]

This is the discussion thread for this script.
  18:27:23  4 June 2009
profilee-mailreply Message URLTo the Top
Don Reba
Bishop and Councilor of War
(Moderator)

 

 
On forum: 12/04/2002
 

Message edited by:
Don Reba
06/04/2009 18:50:01
Messages: 11601

---QUOTATION---
I found that Smart Mod Manager would not integrate the existing mod files with certain mods that I played because it would only replace the entire file instead of just the relevant sections and values that were different causing a lot of crashes and headaches.
---END QUOTATION---

Your assumption that SMM replaces entire script files is incorrect. It does, in fact, always integrate changes, as you say it should. Frankly, the quality of integration leaves much to be desired, but it is getting better.

In version 1.10, SMM will include ModScript, which covers the functionality of your tool and more, but needs more feedback until then:

https://gsc-game.com/main.php?t=community&s=forums&s_game_type=xr2&thm_page=1&thm_id=20843&sec_id=22
  18:57:21  4 June 2009
profilee-mailreply Message URLTo the Top
JakFrost
(Senior)
 
On forum: 11/30/2008
Messages: 112
Personal Need Spawns Creation
(i.e. Necessity is the mother of all inventions)

I know that this script is quite late in the game for STALKER games since the development of many mods has slowed down or even stopped but I felt like I needed to write it for my own usage. I was installing and changing mods in my own STALKER games so many times and then having to manually use a text editor to make the changes that I wanted something automatic. I've wasted more time manually changing mod .ltx files than the amount of time it took me to create this script to do it automatically. I only wanted to change a few values in the files and not add more content to the game so I thought it was easier and faster to just change them manually, boy was I wrong after I started installing different mods and then updating those mods with new versions, every time losing all my changes and having to do them all again manually.

Monolithic Whole File Mods

I was also dissatisfied people releasing Mods by including the whole file in the distribution when they were only changing a few numbers in the .ltx files. I always thought that if you are going to change a few small things in a file and not really add more data then you could have done this but submitting a difference patch instead of the whole thing. However, most people are not familiar with .diff files unless they are developers or advanced computer users.

I understand that these mods are mean for casual gamers, who mostly do not know anything about advanced computer topics, and also do not want to bother with manually modifying files. It is hard enough to have them actually read any readme.txt files and then have them modify the "fsdata.ltx" file to set the "$game_data$" values to "true | true" as it is.

Shortsightedness For Ease

However, the modders who released their minor mods that only change numbers and release their own copy of the regular game file really became shortsighted since they didn't realize that other mods that provide new content would be using those same files and the files with the new content would have to overwrite the regular game files from the first mod. This made the minor mods completely incompatible with the new content mods, but they did not have to be incompatible if they only released the information about the lines that they have changed so that their minor mod could be applied on top of the new content mod.

Difference Engine That Could Have Been

My guess is that if I came across the game STALKER earlier and came across these problems I would have written my script earlier and then maybe convinced some of the minor modders to release their mods as data files for my script. I have a feeling that if that happened I would probably end up writing up a whole difference engine for STALKER LUA .ltx mod files or using an existing engine allowing mods to be quite modular, by only releasing data files with their own changes, additions, and modifications, so that mods could be merged one over another by modifying existing game files instead of replacing them whole. But it is too late at this point to work on this kind of an engine for these games, at least until STALKER Prypiat comes out and we'll see what kind of a script system that game will use.

Minor Modders Welcome and Encouraged

The script that I created is really fed by a data file that you can specify so if you're a minor mod maker, where you only like to change a few values here and there inside the scripts to change the game here and there then I encourage you to create your minor mod as a data file to my script so that your mod users can apply your mod to any other mod out there that might be using the same files.

I would probably encourage you to provide your mod as both, a single file that is compatible with the existing mod distribution format used here and also Smart Mod Manager, but also as a data file for this script so that if another mod already uses the same file as yours, you can instruct the user to download this script and apply your data over after he installs the other mod. It is a good way of ensuring that your mod users have a choice of using your mod if another mod conflicts with it because of the same file.

The data files are quite easy to create and you can use full Extended Regular Expression syntax for searching and replacing values inside the data file giving you a lot of flexibility in writing a single data like that would change hundreds of entries without having you hunt down and type in those hundreds of individual entries.

Data File Line Examples - Future-Looking and Forward-Compatible

One advanced example from my own Data File is the one that I wrote for limiting the number of grenades that spawn on new enemies. This single line affects half a dozen files, and dozens of entries inside those files. It is really a very efficient way of doing this.

You can see the wildcard "*.xml" entry to affect multiple files. Along with the Extended Regular Expression syntax "[^[:space:]]+" to affect any fields that start with the word "grenade_". This example means to select all characters that are not "^" space characters such as space, tab, vertical space, etc. So that it affects all fields such as "grenade_f1" and "grenade_rgd5". However, it would also apply to any other types of grenades that another modder might include in the game in his own mod such as "grenade_jumper" and "grenade_claymore" if you were applying this data file to a game that already includes another mod. This wildcard matching behavior is very flexible because it becomes future-looking and forward-compatible with other mods since you do not have to know exactly the names of other grenades that might be in those files, but you still really want the enemy to have a lot less of them.


STALKER-STCS;Grenades Less Available To Enemies;gamedata\configs\gameplay\character_*.xml;spawn;grenade_[^[:space:]]+;1, prob=0.1 \\n



A more powerful example of Extended Regular Expressions is the trader modifications for more ammo that I wanted to modify for the STALKER Clear Sky - Total Faction War mod that I played and enjoyed very much. The traders in the 0.6.9 version of this mod only had 1 or 3 boxes of ammo available and when I was playing the Freedom faction that users 5.56 NATO weapons I could never find enough ammo and could never buy enough to actually use those weapons, requiring me to convert my guns to 5.45x39 Russian so that I could at least scrounge up ammo from dead bodies since I also couldn't buy enough.

In the example below you see the section field that includes a pretty flexible and powerful wildcard match "supplies_(generic|res_[[:digit:]]+)". This match includes the very useful "OR" character "|" allowing the wildcard to applying to different sections that have the same "supplies_" prefix such as "supplies_generic" and "supplies_res_" while also including a wildcard match "[[:digit:]]+" to match all the "supplies_res_3" "_9", "_12", "_14", and "_16" sections all in a single line. This one line applies to half a dozen "trade_trade_*" files while making dozens of changes in each file itself. This is pretty flexible and powerful while at the same time being very easy and convenient for the modder since he only has to create one line to make all those changes to affect all the different files, and also easy for the user to implement since these changes are fairly future-looking and forward-compatible so that if a new version of the mod makes a few changes to the ammo lists that each trader provides this data file will still work correctly and apply the changes to those files also.


STALKER-STCS;Ammo More Available At Traders;gamedata\configs\misc\trade\trade_trader_*.ltx;supplies_(generic|res_[[:digit:]]+);ammo_[^[:space:]]+;100, 1.0



The two examples above demonstrate the power and flexibility of a non-monolithic system for implementing minor mods for users that are future-looking and forward-compatible across changes and updates to mods.

Minor Modders The Future Is Up To You

Since I'm pretty much finished with the current STALKER games and mods I leave this script and data files as an example for future minor moders for when the next game STALKER Prypiat comes out. If by some chance the new game users the same engine and LUA .ltx files in similar configurations as the previous games then this script might even be fully compatible with the future game as long as you include the proper Registry Key reference in the data file for the Game field such as I might guess "STALKER-PRIP" or something like that, and then just put in a reference to the other fields in the data file. Use this script as an example.

Future Improvements - Auto File Extraction for Missing Mod Files

One thing that I would like to see added is an automatic extraction function to this script so that if this script is processing a data file and it notices that the file to modify is missing that it could automatically extract the latest version of that file from the game DB files, place it in the proper folder in the "gamedata\" folder and then apply the mod there. That would be great but I would need tools to selectively extract specific files from inside the game DB folders. I don't believe that the current tools allow that, since I believe that they extract all the files from the DB files and do not allow single file extraction.
  19:30:17  4 June 2009
profilee-mailreply Message URLTo the Top
Don Reba
Bishop and Councilor of War
(Moderator)

 

 
On forum: 12/04/2002
 

Message edited by:
Don Reba
06/04/2009 19:31:17
Messages: 11601

---QUOTATION---
I was also dissatisfied people releasing Mods by including the whole file in the distribution when they were only changing a few numbers in the .ltx files. I always thought that if you are going to change a few small things in a file and not really add more data then you could have done this but submitting a difference patch instead of the whole thing. However, most people are not familiar with .diff files unless they are developers or advanced computer users.
---END QUOTATION---


Again, Smart Mod Manager computes the differences automatically. It seems to me, that you are solving a problem that does not exist.
  19:48:10  4 June 2009
profilee-mailreply Message URLTo the Top
JakFrost
(Senior)
 
On forum: 11/30/2008
Messages: 112

---QUOTATION---
I was also dissatisfied people releasing Mods by including the whole file in the distribution when they were only changing a few numbers in the .ltx files. I always thought that if you are going to change a few small things in a file and not really add more data then you could have done this but submitting a difference patch instead of the whole thing. However, most people are not familiar with .diff files unless they are developers or advanced computer users.
Again, Smart Mod Manager computes the differences automatically. It seems to me, that you are solving a problem that does not exist.
---END QUOTATION---



Maybe that is the case with your mods but in my experience I could not get Smart Mod Manager to integrate a simple Medkit Heals Bleeding single file mod with Total Faction War for Clear Sky without crashing the game and requiring me to do a manual edit of the file in question. If this problem did not exist then my guess is that your new ModScript project is solving the same problem that does not exist.

I understand that my script and your ModScript project do the same thing and if your ModScript project is updated to include wildcard matches for files and regular expression syntax for section, field, and value names then your project really covers the functionality of this script. I don't have a problem with your ModScript project at all since I wrote the script for my own usage and I've shared it for other people to look at and use if they need the same features and flexibility of regular expressions for their minor mods. I really have no intention of building up features into it beyond what it does not because it does what I need it to do.

Since your Smart Mod Manager works well for my other mods, and with the inclusion of ModScript it does the same as long as the features like regular expressions and wildcard filename matches are added then that would be great. I wouldn't mind at all deprecating my own project and recommending all potential minor modders to use your ModScript format instead. I'd make a link in my own download and discussion threads to point to ModScript so that it becomes the de facto standard for script mods and to avoid any modders from using my hopefully soon to be deprecated script and data file format.

I'm all for having a single, unified mod system in the STALKER community and your Smart Mod Manager with ModScript should be it, provided that it offers the most and best features that modders want.
  23:18:36  4 June 2009
profilee-mailreply Message URLTo the Top
Don Reba
Bishop and Councilor of War
(Moderator)

 

 
On forum: 12/04/2002
 

Message edited by:
Don Reba
06/04/2009 23:42:15
Messages: 11601
That SMM still cannot merge some seemingly simple cases just goes to show that diffs are not the Silver Bullet. Still, when in past people described problems they were having, we were able to solve them by improving SMM's merging algorithms. For instance, since version 1.8, it is not sensitive to differences in spacing or comments in Ltx files.

ModScript does not solve the same problem as your tool. I am mainly concerned with enabling modders to expose to their users adjustable features of their mods. Creation of mods that let the user customize the game is another application. It is not the intention, however, to create another way of making mods. There is plenty of small mods as it is, just look at those by Nandersen and {imperialreign} for good examples.
  12:56:15  5 June 2009
profilee-mailreply Message URLTo the Top
JakFrost
(Senior)
 
On forum: 11/30/2008
 

Message edited by:
JakFrost
06/05/2009 12:56:33
Messages: 112
New Script Version - Necessity Calls Again

Once again necessary calls and I answer. I wanted to include the ability to have the SEVA suit be upgradable fully in the regular game so that you don't have to join the Bandit faction. I saw a file mod for the "stalkers_upgarde_info.ltx" file that did it and I did a WinMerge check against it to find that it lets the Stalker Mechanic in Agroprom do the upgrades. The change was simple with changing two lines from "false" to "true" to allow the upgrades but it also required the addition of another line "scientific_outfit" to the "[agr_stalker_base_mechanic]" section.

Well my script could only change lines but not add them, so I sat down last night and rewrote it entirely to allow it to do additions, changes, and deletions for lines. I finished it and the script works but there is just a tiny little bug with the windows shell command processor "cmd.exe" forcibly escaping any strings with the escape character "^" causing some small issues. It is late now so I'm going to work on this issue and resolve it tomorrow.
  23:19:48  5 June 2009
profilee-mailreply Message URLTo the Top
JakFrost
(Senior)
 
On forum: 11/30/2008
Messages: 112
Escape Character Problem Fixed

I fixed the problem with the escape character so now the script works very well with all the special regular expression characters in the data file.

One annoying thing remains is that all the section, field, value fields in the data file are always treated as regular expressions so they require all special characters to be escaped when populating these fields inside the data file. This makes it a pain in the ass for future modders to remember to escape all characters even though they might be doing a simple replace that contains any of the "\^$.?*+[](){}|&" and "/" characters.

I might have to think of a way to allow regular expressions to be /forward-slash quoted/ and regular strings to appear normally. This would solve the problem of having to escape everything.

Script Refactorization Complete

I was able to refactor some sections of the script and merge some of the query section commands. It worked out well.

I also changed all funky numerical references for variables "%~5" to actual name references "%value%" since I rewrote all the parts of the script dealing with regular expression section, field, and value variables to make them double-quote safe.

Additional File Formats .ltx, .xml, .script

Currently this script only handles .ltx files that are based on the config file format similar to INI files and contain "[section]" and "field = value" type of data. There are also .xml and LUA .script files that contain some data that could be modified but right now this script does not understand these formats. I believe that it would be possible to implement additional SED scripts to deal with these data formats in the future. The difficulty with this update is how difficult it would be to write SED scripts that will be able to deal with these file formats without trying to use advanced programming techniques and variable data formats that would not work in the simple shell language and would require the porting of this script to a more powerful language.

Automatic File Extraction

Right now the script depends on the user to ensure that the file to be modified has already been extracted from the game's DB files or that is already exists because another mod provided the file so it is ready to be modified. However, I think that it would be possible to add simple logic to this script to detect if a required file is missing, then to automatically extract all the DB files that contain "config\" or "scripts\" folders, find the required file, and copy it to the proper "gamedata\" folder, then modify it.

Next Release

Since there is no rush, I'm going to see about implementing the automatic file extraction feature for this script to save the user the trouble from having to extract and find the files himself that need to be modified. I think that this should not be a big problem to do script wise.

The addition of .xml and .script file support is something that I'll probably look at sometime later after I get this release out because I have a feeling that I'm going to look at some other mod that modifies one of the .script or .xml files and I'm going to have that need to update the script again for my own usage. Well, at least necessity is good in this respect.
  12:17:44  6 June 2009
profilee-mailreply Message URLTo the Top
JakFrost
(Senior)
 
On forum: 11/30/2008
Messages: 112
New Release - 2009-06-06

2009-06-06 - Public release.
/ Data file format modified.
+ Add, Change, Delete actions added.
+ Automatic Mod File Extraction added.
+ Automatic Gamedata Mod Enable in "fsgame.ltx" file added.
+ SEVA Suit Upgrade included in data file with Add and Change actions.


The new release is up. I implemented some additional features that I needed such as the Add, Change, Delete actions to get the SEVA Suit Upgrade to work. I also added the automatic file extraction and "fsgame.ltx" file edit.

I'm quite happy with this script at this point. It could still use support for .xml and .script files but I think that I'll leave those features alone for now since implementing a good parser for those more difficult formats would probably be easier to do in a different language such as C# and .NET Framework than in the Windows shell. I already had to battle a hoard of bugs with the Windows shell to get even this functionality to work and I documented the bugs and my workarounds inside the script itself for any curious on lookers. The code is pretty clean, well structured, and commented so it should be understandable but it uses a lot of advanced coding techniques for Windows shell pushing it to the limit and past it so it is a bit weird in some places.

Enjoy. It's time to get a little break away from coding.
  18:21:55  7 June 2009
profilee-mailreply Message URLTo the Top
JakFrost
(Senior)
 
On forum: 11/30/2008
Messages: 112
New Release - Small Fix

Released a new version since I had to fix the Modify label since it did not refer to SED properly from the Utilities folder. Also tested the new SED 4.2-1 release from GnuWin32, but it still suffers from the --in-place problem.

2009-06-07 - Public release.
/ Fixed reference to SED.exe in the Modify label.
. Tested new SED 4.2-1 release, but --in-place bug still exists.
Staying with SED 4.1.5 for this script.
 
Each word should be at least 3 characters long.
Search:    
Search conditions:    - spaces as AND    - spaces as OR   
 
Forum Index » S.T.A.L.K.E.R.: Clear Sky Forum » Mod discussion
 

All short dates are in Month-Day-Year format.


 

Copyright © 1995-2019 GSC Game World. All rights reserved.
This site is best viewed in Internet Explorer 4.xx and up and Javascript enabled. Webmaster.
Opera Software products are not supported.
If any problem concerning the site functioning under Opera Software appears apply
to Opera Software technical support service.