In discussions with my local peer group about my turn-based game project, the idea of using scripting within the game has come up.

Several scripting languages, like Lua, Python, and Ruby have been discussed. These would be used to process scripts that define unit movement, attack resolution, UI structure, etc.

Does anyone have prior experience doing this with their games?

If so, was it worth it? Did it give more power to your mod community? Was it easy? Would you do it again?

Thanks!

I had used Lua in my day job for games, it was used in our editor for describing objects, it was used as the data format for the editor and because it was script it made it simple to write automatic merging for version control for shared projects, it was also used as a general scripting language in game too.. totally flexible, free, has a good community, would I use it again ? yep, no doubt about it, which scripting language you should chose though is an entirely different thing. but I give using script the big thumbs up for almost any project.. there is always a use for it (I used to be skeptical about it though belive it or not, I guess that's change for you :) )

I know you are asking for personal experiences, but I would suggest reading Game Scripting Mastery. It is an awesome book on game scripting and totally opened my eyes to how scripting should be done. Until I read that book, I kind of made up my own scripts without too much thought or confidence in what I was doing. If I remember correctly, the book covers Lua and existing script languages, but it also walks you through creating your own.

Basically, if you can script it, you can make your game more flexible, and so mod communities can definitely make things. At the very least, maps can have triggers, and providing an editor for your players will allow them to create their own. New units, new bonuses, new traps, etc, can all be done if you script the hell out of your game.

And of course, rather than recompile each time to tweak values in playtesting, you can just recompile a single script related to those values. Saves you a bunch on development time.

This is all based on the info I read in that book and othe articles on the subject (again, not personal experience). I naturally plan on having scripts for any sufficiently complex game project I take on.

Indeedy, there is also scope for if you use them to define a level, you can always give your users free expansion packs and what not, and they would only have to download bytes worth of data, keeping your bandwidth down, and customers happy.. doens't open Zelda use Lua, I think that may use Small, but same concept, lots of people have built levels and stuff.. biggish community.. it's all great :)

Scripting is incredibly useful. I use it to define effects, levels, the console takes in script, great for writing test harnesess/regression tests.

An example of how scripting makes life easy: in my file format the loader runs a script which, depending on certain parameters - hw available, difficulty settings etc, tells the loader what assets to load. I could have hardcoded this logic but putting it as a script in the WAD file makes it so much more flexible. Having the console accept lua script is incredibly useful too.

Lua is my best friend when it comes to development :)

grimreaper

By happy coincidence, I stumbled across this short interview about the use of Python (http://pygame.org/interview/stevemoret.shtml) in Greyhawk:Temple of Elemental Evil (http://www.greyhawkgame.com/) earlier today.

--- Rod

We're using scripting very heavily at my (not game related) job. This has been one of the benefits of working here, because before I didn't realize how powerful scripting can be. It's REALLY nice to be able to modify things without having to recompile. Development is also much, much faster because you don't have to deal with data types, pointers, and a slew of other things you have in a language like C/C++. Essentially what I'd recommend to anyone, is to write the code that has to be fast in C/C++ and write most of the game-specific code in a scripting language. This is what I'm leaning towards doing for my next game.

Just one bit of advice -- don't use TCL. That's what we're stuck with and it sucks. Some people might like the syntax, but to me, having to do things like:

set result [expr 1 + 1]

to add two numbers is very annoying. I don't understand what the problem is with the world-renowned standard:

result = 1 + 1

Some other things to keep in mind are: Lua is nice because it's small. Python will cost you about a meg in runtimes, but has a big standard library. That book Game Scripting Mastery is very good and it covers Lua, Python, and TCL if I remember correctly. Most of it is dedicated to making your own language though, which is fun but how many of us have the luxury to spend time on that. Still worth reading though, IMO.

My current project, Feyna's Quest (http://eldwood.com/feyna/), actually went through several stages. I started out writing the game in C++ with Python as the scripting language, but ended up having to drop the Python scripting because it was too slow. (For what it's worth, it wasn't actually Python which was too slow, but the Python/C++ bindings.) Later I found that I still needed some sort of scripting language, so I wrote my own. This turned out to be the right decision. It only took a week to get it to work - shorter than it would have taken me to rewrite the Python/C++ bindings - and it gives me all of the power I need, including some things which would have been very difficult to do in plain Python.

For my next project, I'm planning on writing the game in Python and using a slightly modified Python as the scripting language. This is actually harder than it sounds like - there is only minimal support for modifying Python from within Python, and keeping the engine and the scripts separate is difficult in such an environment. We'll see how it goes.

When we started our current project we were hardly aware of in-game scripting and its benefits and given the nature of the project, essentially a 3d shoot-em-up, we felt that scripting was something we could live without.

Eventually we ended up isolating character AI and tendering in isolated code using OOP with a factory providing new character instances by name. I think that for our purpose, this is suitable, but for anything more complex I'd definitely go for scripting.

O man. Mutant Storm would have taken us twice as long if it wasnt for LUA. We had loads of different enemies, each tweeked endlessly untill perfect. LUA basically ment our tester/artist (me!) could make changes without hassling our coder. Make something faster, slower, bigger or move differntly as well as just about everything else we could think of could be done very easily by the end.
It was a lovely way to work, so much so that our current title is even more heavily script based. This time EVERYTHING is controlled using scripts from the front end right down to every explosion and particle effect. Absolute control.

I simply couldnt go back now!

Pick and choose when you need to use it.

If you're the coder and you're the one who's going to be writing and editing the scripts then don't use a scripting language.

If you're going to pass it on to a designer or artist then ask yourself do you need scripting or just a txt file with attributes?

@luggage

Don't forget that besides making it designer-friendly, a scripted game engine has another major benefit: no additional compilation time, especially if it's a large project in c++.

For instance, if I do a change at the lowest layer in my own project, compilation take sin excess of 10 minutes! Luckily enemy AI and rendering code is in a separate high-level module so it takes only seconds! :)

compilation take sin excess of 10 minutes
OMG!

I guess this could be the most relevant (only?) argument of the C++ vs C wars or maybe a total lack of understanding on how to use precompiled headers - they're none to obvious and the default settings are worse than choosing "none". It has to be something avoidable.

I have a project here which is truly massive. It's a complete commercial rendering engine bristling with features, plus combined physics engine, portalling, ray casting/collision, memory manager and masses of other stuff - the full monty. Stainless used to make all their games with it and I now use it for my own efforts having removed all the PS2/XBox etc platform specifics.

There are hundreds of small files and hundreds of big files and truly masses of small headers. There's a shitload of #ifdef compile switches for the old cross-platform stuff that I've not tidied up yet, both in a lot of source and practically all the headers.

All files have .CPP extension but only light useage is made of the extra stuff available and it's not too OOP based generally. Dunno how much of that is a factor.

Total recompilation time from a clean ? About 15 seconds on a 3Ghz Intel.

OMG!

I have a project here which is truly massive. It's a complete commercial rendering engine bristling with features, plus combined physics engine, portalling, ray casting/collision, memory manager and masses of other stuff - the full monty. Stainless used to make all their games with it and

...

Total recompilation time from a clean ? About 15 seconds on a 3Ghz Intel.

Even more important: can you share it? It'll wont take much more than 15 seconds to upload... :) :)

grimreaper

10 minutes? I've had compilation times more the range of 10 hours.

Note: Using extensive template metaprogramming speeds up programming but slows down compliation dramatically.
gcc is slow.
Using on a 500Mhz computer with only 256MB RAM is slow.
Building two executables in two configurations on two compilers slows down compilation times by a factor of eight.

I've tried precompiled headers and they've given me trouble and at some point the only solution for a particular problem was to disable them. I guess I'm not too familiar with them really. Anyway, these days it is rare that I have to do a low level change.. most of the code work is in a top level project - enemy behaviour and rendering. Also, incremental compilation goes a long way into speeding up things - I rarely do a rebuild in actual fact.

One notable exception was retro-fitting a memory manager. Testing it and applying it to the existing game entities was a chore and it still doesn't handle object pool overruns perfectly.

There are a couple of things that I like to do to reduce compilation time.

The first is to take out the changeable headers from the precompiled header file. PCHs are meant to be compiled only very rarely, so I have things like the Windows, Boost, and STL headers in them.

The next thing that I do, is to ensure that my system is as loosely coupled as possible. I program in a very object oriented manner (in fact I find it hard now to program in the procedural style of my old C days.) The way I tend to do this is by using interfaces or facades and hide the implementation classes behind them. That way when I change my renderer code from DirectX to OpenGL (for example) only the renderer code is rebuilt, not the entire engine, or even any code that calls the renderer for that matter.

I always try and look at my code to see where patterns (that make sense of course) would reduce the complexity. Programmers are (can be) masters of complexity and take great pride in creating a wonderfully complex and artistic piece of code to solve a problem. Unfortunately, this complexity often leads to code that has links to dozens or hundreds of other functions, and when something is changed, the whole system has to be rebuilt.

I tried to use these rules in the last major project that I worked on. I think it exceeded half a million lines of code, and it never took more than a few minutes to rebuild on a 500MHz Pentium, unless I did a complete rebuild. Then it was time to get a cup of coffee.

The ironic thing is that I now drink a lot less coffee that I did back when I first started programming!

I fully agree with you that using an interface-implementation pattern would not only improve the design from an architectural point of view, but also speed up compilation.

However, the downside is that it introduces a whole host of virtual functions even when not necessary e.g. no polymorphismis required. I know that these days, the overhead of virtual functions is negligible but over a large number of calls it might add up considerably. In fact I go to great lengths to inline where possible and only make member functions virtual if I'm going to override them in derived classes.

In actual fact, there are places in the code where the interface-implementation pattern is in use, notably, the 3d engine (deisgned to support both OpenGL and DirectX drivers - ok, the latter is not there yet :)), the entity managers, collision managers etc. But, as I said, it is used only where strictly required.

I haven't bothered counting the lines of code yet, but the -compressed- size of the core code, game engine, editor and game itself (excluding 3d engine and sound engine yet to be built) is about 1MB so far.

I really like Lua. I used Lisp for a few months now, and after that it is quite hard to get back to C, but Lua makes the return bareable. The language is dynamically typed, has garbage collection, functional and exceptionally clean. In my current project all of the game specific code is in scripts (with the user interface a few other pieces of code scrapped from previous projects in C).

My current system is really nice - the program won't even exit for most of the errors, allowing me to either run debug scripts and see their results on an in-game console, or reseting everything and restarting the game after making changes in the script sources. In fact, I can pause a game and enter console mode, run some debug scripts and continue from where I left it. All the game runs are recorded, and I can replay the recording without quitting the game as many times as I want, giving me all the opportunity to track errors. Also my C code is choke full of asserts, so any mistakes are rapidly exposed, and I can fix them without exiting the running game.

I can't do break-point based/step by step debugging for the Lua code, but I can save all Lua code lines that get executed along with the values of local variables in a file (but it is very slow, and I only use this for specific areas of code). Also, I lack the data watching of the C debugger. Instead, I rely on debug scripts that display any data I want, as needed.

Lua supports debugging. You can get the stack, local variables, and set call and line hooks very easily. Check out luadebug.h.

A post on the lua.org forums/mailing list usually elicits several high-quality responses.

grimreaper

In that case it would be worthwhile hooking up LUA with the in-game console and perhaps even create the LUA equivalent of ASSERT(), VERIFY() and TRACE() hooked to native code in C... feeling all tingly inside! :)

However, the downside is that it introduces a whole host of virtual functions even when not necessary e.g. no polymorphismis required. I know that these days, the overhead of virtual functions is negligible but over a large number of calls it might add up considerably. In fact I go to great lengths to inline where possible and only make member functions virtual if I'm going to override them in derived classes.


Definitely true. Virtual functions are a little slower than non virtual. However, a virtual function is really just a jump table, and a jump is generally a pretty quick instruction. Also a good compiler will do some decent optimising anyway.

But... I'd rather have a clean design first, and then do my optimisation later. The clean design will have a greater effect on the performance and quality of the code during development and after release, than any attempt at premature optimisation will. In my experience, early attempts at optimisation generally target the wrong areas, and tend to make the code very messy. You generally do less work optimising code, when the software is complete.

I frequented a few agile programming mailing lists for a while, and they had this little mantra:

Make it work.
Make it right.
Make it fast.

In that order.

:)

FYI, regarding virtual functions.. without multiple inheritance, a virtual function call requires 2 memory loads before the jump.. with multiple inheritance, it requires 3 memory loads before the jump..

(Recalling what I vaguely remember from "Inside the C++ Object Model"..)

Definitely it's better not to worry about optimization until later, but it's not a bad idea to have these technical details in the back of your mind when designing the code in the first place.

Now back to your regularly-scheduled scripting language thread..

ObScriptingLanguage - been learning Python lately and am loving it. I will definitely use it for tools.

Can't remeber who said about think about your app to see if you need scripting, tbh I would design my app around having scripting always.. whether it needs it or not, as if you do use it, and you have a title that sells damn well, it wont take hardly any time to knock out sequels, that said, implimented properly, you would have a foundation for a generic game engine. so you can create several titles using the same code base but chaning the script... just something you should really ponder on.

I used Ficl on my current game, but I would no longer recommend it, even if you were comfortable in FORTH (which you're not). FORTH as an embedded scripting language means you wind up tripping over your own feet too often.

My next game will use Python as its scripting language. A totally minimal Python interpreter (no lib/*.py, no .exe, no COM) zips to 475k; with all the goodies it's 1,340k. 7Zip brings that down to 392k and 980k respectively. I figure, with the stuff I need, and with 7Zip, I'll eat 500k on Python. So, yes, it's big, but oh it's so good. Python makes my heart sing with joy.

If you dont need all those Python libaries, I suggest lua. It statically links to the exe and it's so small it takes no space at all. You can also include some additional libraries - all are very small.

grimreaper

Yes, scripting is tremendously useful. At the previous place I worked, we made this generic game engine, for PC and PS2, which didn't really have much game specific code in it. Instead, all gameplay related stuff was done in LUA, and we successfully made playable prototypes of a 3d platform game, a GTA style game, a puzzle game and a point & click adventure. All ran with the same executable, only different scripts.

Also, our level editor saved the levels as LUA files, which, when executed, pushed all the data to the engine. So, in a way, we used LUA as our generic data format as well.

That being said, sometimes I think it can be better to use a data-driven approach, especially if non-technical people are to be tweaking values etc. For my current indie project, I use XML files to store data, but there's no reason why LUA couldn't be used to do the data-driven approach as well as script-driven, as in the example above.

On compile times: I once worked at a place where we never (*ever*) included header files from a header file. Only cpp files where allowed to include header files. It was incredible to see how much this reduced compile times, but it was kind of a pain to always include everything in the right order in your cpp file (but on the other hand, it made you learn everything about the projects class hierarchy).

Hurrah! Language debate! Shall I say something about Java?

Cas :)

No, I think you should try to control yourself. You don't have a control problem, do you? :D

Seriously, I once worked for a company where we used Java as a scripting language, and had a virtual machine running inside the game engine...

Can't remeber who said about think about your app to see if you need scripting, tbh I would design my app around having scripting always.. whether it needs it or not, as if you do use it, and you have a title that sells damn well, it wont take hardly any time to knock out sequels, that said, implimented properly, you would have a foundation for a generic game engine. so you can create several titles using the same code base but chaning the script... just something you should really ponder on.

Spot on.
Our current title is NOTHING like Mutant Storm, but because we considered future developments while making MS, the system in place ment we could prototype in a matter of days. Without LUA, im not sure any of this would have been possible.
Space Tripper didnt use LUA, meaning a long winded process getting MS up and running. We wont make that mistake again.