diff --git a/devblog/build_blog.py b/devblog/build_blog.py index 064cc90a77..b85799042d 100644 --- a/devblog/build_blog.py +++ b/devblog/build_blog.py @@ -145,9 +145,11 @@ def md2html(): calendar[date.year].append(post) # make sure to sort all entries by date - for year in calendar: + for year in sorted(calendar): calendar[year] = list(sorted(calendar[year], key=lambda post: -post.date_sort)) + calendar = {year: calendar[year] for year in sorted(calendar, reverse=True)} + # pair pages with years/filenames for year, posts in calendar.items(): diff --git a/devblog/html/devblog2012.html b/devblog/html/devblog2012.html new file mode 100644 index 0000000000..198fcdbc88 --- /dev/null +++ b/devblog/html/devblog2012.html @@ -0,0 +1,925 @@ + + + + + + Devblog: Evennia Python MUD/MU creation system + + + + + + + + + + +
+
+

+ +

+

Evennia

+

The Python MU* Development Library

+
+ +
+
+ +
+
+
+

+ + + + Evennia Dev Blog

+ + Random Evennia- and MUD-related musings of Griatch, the Evennia lead developer. + + + +
+

+ Evennia changes to BSD license + +
- October 28, 2012
+

+

As of today, Evennia changes to use the very permissive BSD license.

+

Now, our previous "Artistic License" was also very friendly. One main feature was that it made sure that changes people made to the core Evennia library (i.e. not the game-specific files) were also made available for possible inclusion upstream. A good notion perhaps, but the licensing text was also quite long and it was clear some newcomers parsed it as more restrictive than it actually was.

+

... And let's be honest, it's not like I would have come hunting down anyone not complying fully with the Artistic license's terms. Changing to the much simpler and more well-known BSD license better clarifies the actual licensing situation.

+

After all, far too many older MUD-code bases are weighted by a legacy of licensing issues. Anything we can do to avoid this is better in the long run. Indeed we hope this change in licensing will remove eventual licensing doubts for new adopters and have more people join and contribute to the project.

+ +
+

+ +

+
+
+ +
+
+

+ Community interest + +
- October 5, 2012
+

+

+

It's fun to see a growing level of activity in the Evennia community. The last few months have seen an increase in the number of people showing up in our IRC channel and mailing list. With this has come a slew of interesting ideas, projects and MUD-related discussion (as well as a few inevitable excursions into interesting but very non-mud-related territory - sorry to those reading our chat logs).

+

One sign of more people starting to actually use Evennia "for real" is how the number of bugs/feature requests have been increasing. These are sometimes issues related to new things being implemented but also features that have been there for some time and which people are now wanting to use in new and creative ways - or systems which noone has yet really used "seriously" before. This is very encouraging, especially since a lot of good alternative solutions, variations and edge cases can be ironed out this way. So keep submitting those Issues, people!

+

The budding Evennia community consists of people with a wide variety of interests, skillset and ambition.

+

There are quite a few people who sees Evennia as a great stepping stone for learning Python, or for getting experience with creating a bigger programming project in general. Some are skilled programmers in other languages but we also have a few with only limited prior coding experience. From the experience in chat, it's really quite striking how fast members pick up the ropes. I'd like to think our documentation is at least partially helping here, but of course it helps that Python is inherently very easy a language to learn and use in the first place.

+

Not all are participating with the goal of building a specific game. The general flow of patches and clone repository merges have also picked up. We have some users which are primarily interested in a coding challenge, to help with fixing bugs and features, or which uses Evennia as a starting point for exploring various web- and technical solutions that may or may not be a part of Evennia in the future.

+

The proposed Evennia game projects are just as varied as its users - and none are yet open to the public. As is common with these things, it's of course hard to determine who actually has the time and discipline to bring their plans to fruition. But I should really start to keep some sort of record of who works on what, I'm terrible with remembering this stuff ... so below is just some sort of summary of my impressions, not a comprehensive listing.

+

As can be expected, most proposed Evennia projects concern relatively standard MUD-style games. A few people are into building traditional hack-and-slash variety games, but most want to expand on the concept considerably. There was even one user exploring using Evennia for a RobotWars kind of experience (where you "program" robot programs in a custom language and battle them). Another project (Avaloria, also blogging on the MUD-dev rss feed) aims for a sort of base-building/strategy mechanic combined with more traditional MUD elements. There are at least two zombie-survival concepts floating around and a few large-scale procedural-content-driven science-fiction text games. One user has apparently a working Smaug->Evennia importer.

+

It seems that most Evennia users want to offer some sort of roleplaying environment, or at least a "roleplay-friendly" one. Currently we have at least two MUCK admins who aim to convert their existing, running games to Evennia. Whereas the initial idea was to implement parsers for MUCK's MUF language, it seems the conclusion has now shifted to it being faster and easier to just rewrite the MUF-coded functionality in Python (and maybe use something like Evlang for player scripting instead). Several people have announced their interest in creating "RPI"-style games (Armageddon seems to be a big inspiration here), but there was also a MOO admin and even a writer of Interactive Fiction who dropped into the mailing list to see if Evennia could be used for their style of game.

+

How many of these projects actually reach a point of maturity remains to be seen. But that people are wanting to use the system and is really putting it through its paces is encouraging and very helpful for general Evennia development.

+ +
+

+ +

+
+
+ +
+
+

+ Combining Twisted and Django + +
- August 31, 2012
+

+

Combining Twisted and Django

+

+

Newcomers to Evennia sometimes misunderstand it as being a "Django mud codebase somehow using Twisted". The correct description is rather that Evennia is a "Twisted-based mud server using Django". Allow me to elaborate.

+

A mud/mux/moo/mu* is per definition a multi-user online game system. All these users need to co-exist on the server. If one player does something, other players shouldn't have to (noticeably) wait for that something to end before they can do anything. Furthermore it's important for the database schema to be easy to handle and upgrade. Finally, in a modern game, internet presence and web browser access is becoming a must. We combine two frameworks to achieve this.

+

Two frameworks combined

+

Twisted is a asynchronous Python framework. "Asynchronous" in this context means, very simplified, that Twisted chops up code execution into as small bits as the code lets it. It then flips through these snippets rapidly, executing each in turn. The result is the illusion of everything happening at the same time. The asynchronous operation is the basis for the framework, but it also helps that twisted makes it easy to support (and create) a massive range of different network protocols.

+

Django implements a very nice abstract Python API for accessing a variety of SQL-like databases. It makes it very convenient to maintain the database schema (not to mention that django-South gives us easy database migrations). The fact that Django is really a web framework also makes it easy to offer various web features. There is for example an "admin site" that comes with Django. It allows to modify the database graphically (in Evennia's case the admin site is not quite as polished as we would like yet, but it's coming).

+

Here are some highlights of our architecture:

+
    +
  • +

    Portal - This is a stand-alone Twisted process talking to the outside world. It implements a range of communication protocols, such as telnet (traditional in MUD-world), ssh, ssl, a comet webclient and others. It is an auto-connecting client to Server (below).

    +
    • +
  • +

    Server - This is the main MUD server. This twisted server handles everything related to the MUD world. It accesses and updates the database through Django models. It makes the world tick. Since all Players connect to the Server through the Portal's AMP connection, it means Server can be restarted without any players getting kicked off the game (they will re-sync from Portal as soon as Server is back up again).

    +
    • +
  • +

    Webserver - Evennia optionally starts its own Twisted webserver. This serves the game's website (using the same database as the game for showing game statistics, for example). The website is of course a full Django project, with all the possibilities that entails. The Django admin site allows for modifying the database via a graphical interface.

    +
    • +
  • +

    Webclient - There is a stand-alone MUD web client existing in a page on the default website. This uses Twisted to implement a long-polling ("comet") connection to a javascript client. As far as Evennia's concerned, this is just another outgoing protocol served by the Portal.

    +
    • +
  • +

    Other protocols - Since it's easy to add new connectivity, Evennia also offers a bunch of other connectivity options, such as relaying in-game channels to IRC and IMC2 as well as RSS feeds and some other goodies.

    +
    • +
+

On the joining of the two

+

An important thing to note about Twisted's asynchronous model is that there is no magic at work here: Each little snippet of code Twisted loops over is blocking. It's just hopefully not blocking long enough for you to notice. So if you were to put sleep(10) in one of those snippets, then congratulations, you just froze the entire server for ten seconds.

+

Profiling becomes very important here. Evennia's main launcher takes command arguments to run either of its processes under Python's cProfile module. It also offers the ability to connect any number of dummy Players doing all sorts of automated random actions on the server. Such profile data is invaluable to know what is a bottleneck and what is not.

+

I never found Twisted asynchronous paradigms much harder to understand than other code. But there are sure ways to write stupid blocking code that will come back and bite you. For example, much of  Evennia's workload is spent in the Server, most notably in its command handler. This is not so strange; the command handler takes care of parsing and executing all input coming from Players, often modifying the game world in various ways (see my previous post for more info about the command handler).

+

The command handler used to be a monolithic, single method. This meant that Twisted had to let it run its full course before letting anyone else do their turn. Using Twisted's inlineCallbacks instead allowed for yielding at many, many places in this method, giving Twisted ample possibilities to split execution. The effect on multi-user performance was quite impressive. Far from all code can  be rewritten like this though.

+

Another important bottleneck on asynchronous operations is database operations. Django, as opposed to Twisted, is not an asynchronous framework. Accessing the database is a blocking operation and can be potentially expensive. It was never extremely bad in testing, to be honest. But for large database operations (e.g. many Players) database access was a noticeable effect.

+

I have read of some people using Twisted's deferToThread to do database writes. The idea sounds reasonable - just offload the operation to another thread and go on your merry way. It did not help us at all though - rather it made things slower. I don't know if this is some sort of overhead (or error) in my test implementation - or an effect of Python just not being ideal with using threading for concurrency (due to the GIL). Either way, certain databases like SQlite3 doesn't support multiple threads very well anyway, and we prefer to keep giving plenty of options with that. So no deferToThread for database writes. I also did a little testing with parallel processes but found that even slower, at least once the number of writes started to pile up (we will offer easy process-pool offloading for other reasons though).

+

As many have found out before us, caching is king here. There is not so much to do about writes, but at least in our case the database is more often read than written to. Caching data and accessing the cache instead of accessing a field is doing much for performance, sometimes a lot. Database access is always going to cost, but it does not dominate the profile. We are now at a point where one of the most expensive single operations a Player (even a Builder) performs during an entire gaming session is the hashing of their password during login. I'd say that's good enough for our use case anyway.

+

Django + MUD?

+

It's interesting that whereas Twisted is a pretty natural fit for a Python MUD (I have learned that Twisted was in fact first intended for mudding, long ago), many tend to be intrigued and/or surprised about our use of Django. In the end these are only behind-the-scenes details though. The actual game designer using Evennia don't really see any of this. They don't really need to know neither Django nor Twisted to code their own dream MUD. It's possible the combination fits less for some projects than for others. But at least in our case it has just helped us to offer more features faster and with less headaches.

+ +
+

+

Image: Franco Nero as the twisted gunslinger Django

+ +

+
+
+ +
+
+

+ Taking command + +
- August 16, 2012
+

+

Commands are the bread and butter of any game. Commands are the instructions coming in from the player telling the game (or their avatar in the game) to do stuff. This post will outline the reasoning leading up to Evennia's somewhat (I think) non-standard way of handling commands.

+

In the case of MUDs and other text games commands usually come in the form of entered text. But clicking on a graphical button or using a joystick is also at some level issuing a command - one way or another the Player instructs the game in a way it understands. In this post I will stick to text commands though. So open door with red key is a potential command.

+

Evennia, being a MUD design system, needs to offer a stable and extensive way to handle new and old commands.  More than that, we need to allow developers pretty big freedom with developing their own command syntax if they so please (our default is not for everyone). A small hard-coded command set is not an option.

+

Identifying the command

+

First step is identifying the command coming in. When looking at open door with red key it's probably open that is the unique command. The other words are "options" to the command, stuff the open command supposedly knows what to do with. If you know already at this stage exactly how the command syntax looks, you could hard-code the parsing already here. In Evennia's case that's not possible though - we aim to let people define their command syntax as freely as possible. Our identifier actually requires no more than that the uniquely identifying command word (or words) appear first on the input line. It is hard to picture a command syntax where this isn't true ... but if so people may freely plug in their own identifyer routine.

+

So the identifyer digs out the open command and sends it its options ... but what kind of code object is open?

+

The way to define the command

+

A common variant I've seen in various Python codebases is to implement commands as functions. A function maps intuitively to a command - it can take arguments and it does stuff in return. It is probably more than enough for some types of games.

+

Evennia chooses to let the command be defined as a class instead. There are a few reasons. Most predominantly, classes can inherit and require less boiler plate (there are a few more reasons that has to do with storing the results of a command between calls, but that's not as commonly useful). Each Evennia command class has two primary methods:

+
    +
  • +

    parse() - this is responsible for parsing and splitting up the options part of the command into easy-to use chunks. In the case of open door with red key, it could be as simple as splitting the options into a list of strings. But this may potentially be more complex. A mux-like command, for exampe, takes /switches to control its functionality. They also have a recurring syntax using the '=' character to set properties. These components could maybe be parsed into a list switches and two parameters lhs and rhs holding the left- and right hand side of the equation sign.

    +
    • +
  • +

    func() - this takes the chunks of pre-parsed input and actually does stuff with it.

    +
    • +
+

One of of the good things with executing class instances is that neither of these methods need to have any arguments or returns. They just store the data on their object (self.switches) and the next method can just access them as it pleases. Same is true when the command system instantiates the command. It will set a few useful properties on the command for the programmer to make use of in their code (self.caller always references the one executing the command, for example). This shortcut may sound like a minor thing, but for developers using Evennia to create countless custom commands for their game, it's really very nice to not have to have all the input/output boilerplate to remember.

+

... And of course, class objects support inheritance. In Evennia's default command set the parse() function is  only implemented once, all handling all possible permutations of the syntax. Other commands just inherit from it and only needs to implement func(). Some advanced build commands just use a parent with an overloaded and slightly expanded parse().

+

Commands in States

+

So we have individual commands. Just as important is how we now group and access them. The most common way to do this (also used in an older version of Evennia) is to use a simple global list. Whenever a player enters a command, the identifier looks the command up in the list. Every player has access to this list (admin commands check permissions before running). It seems this is what is used in a large amount of code bases and thus obviously works well for many types of games. Where it starts to crack is when it comes to game states.

+
    +
  • +

    A first example is an in-game menu. Selecting a menu item means an instruction from the player - i.e. a command. A menu could have numbered options but it might also have named options that vary from menu node to menu node. Each of these are a command name that must be identified by the parser. Should you make all those possible commands globally available to your players at all times? Or do you hide them somehow until the player actually is in a menu? Or do you bypass the command system entirely and write new code only for handling menus...?

    +
    • +
  • +

    Second example: Picture this scenario: You are walking down a dark hallway, torch in hand. Suddenly your light goes out and you are thrown into darkness. You cannot see anything now, not even to look in your own backpack. How would you handle this in code? Trivially you can put if statements in your look and inventory commands. They check for the "dark" flag. Fair enough. Next you knock your head and goes 'dizzy'. Suddenly your "navigation" skill is gone and your movement commands may randomly be turned around. Dizziness combined with darkness means your inventory command now returns a strange confused mess. Next you get into a fight ... the number of if statements starts piling up.

    +
    • +
  • +

    Last example: In the hypothetical FishingMUD,. you have lots of detailed skills for fishing. But different types of fishing rods makes different types of throws (commands) available. Also, they all work differently if you are on a shore as compared to being on a boat. Again, lots of if statements. It's all possible to do, but the problem is maintenance; your command body keep growing to handle edge cases. Especially in a MUD, where new features tend to be added gradually over the course of years, this gives lots of possibilities for regressions.

    +
    • +
+

All of these are examples of situation-dependent (or object-dependent) commands. Let's jointly call them state-dependent commands. You could picture handling the in-game menu by somehow dynamically changing the global list of commands available. But then the global bit becomes problematic - not all players are in the same menu at the same time. So you'll then have to start to track who has which list of commands available to them. And what happens when a state ends? How do you get back to the previous state - a state which may itself be different from the "default" state (like clearing your dizzy state while still being in darkness)? This means you have to track the previous few states and ...

+

A few iterations of such thinking lead to what Evennia now uses: a non-global command set system. A command set (cmdset) is a structure that looks pretty much like a mathematical set. It can contain any number of (unique) command objects, and a particular command can occur in any number of command sets.

+
    +
  • +

    A cmdset stored on an object makes all commands in that cmdset available to the object. So all player characters in the game has a "default cmdset" stored on them with all the common commands like look, get and so on.

    +
    • +
  • +

    Optionally, an object can make its cmdset available to other objects in the same location instead. This allows for commands only applicable with a given object or location, such as wind up grandfather clock. Or the various commands of different types of fishing rods.

    +
    • +
  • +

    Cmdsets can be non-destructively combined and merged like mathematical sets, using operations like "Union", "Intersect" and a few other cmdset-special operations. Each cmdset can have priorities and exceptions to the various operations applied to them. Removing a set from the mix will dynamically rebuild the remaining sets into a new mixed set.

    +
    • +
+

The last point is the most interesting aspect of cmdsets. The ability to merge cmdsets allows you to develop your game states in isolation. You then just merge them in dynamically whenever the game state changes. So to implement the dark example above, you would define two types of "look" (the dark version probably being a child of the normal version). Normally you use your "default cmdset" containing the normal look. But once you end up in a dark room the system (or more likely the room) "merges" the dark cmdset with the default one on the player, replacing same-named commands with new ones. The dark cmdset contains the commands that are different (or new) to the dark condition - such as the look command and the changed inventory command.  Becoming dazed just means yet another merger - merging the dazed set on top of the other two. Since all merges are non-destructive, you can later remove either of the sets to rebuild a new "combined" set only involving the remaining ones in any combination.

+

Similarly, the menu becomes very simple to create in isolation (in Evennia it's actually an optional contrib). All it needs to do is define the required menu-commands in its own cmdset. Whenever someone triggers the menu, that cmdset is loaded onto the player. All relevant commands are then made available. Once the menu is exited, the menu-cmdset is simply removed and the player automatically returns to whichever state he or she was in before.

+

Final words

+

The combination of commands-as-classes and command sets has proved to very flexible. It's not as easy to conceptualize as is the simple functions in a list, but so far it seems people are not having too much trouble. I also think it makes it pretty easy to both create and, importantly, expand a game with interesting new forms of gameplay without drastically rewriting old systems.

+ +
+

+ +

+
+
+ +
+
+

+ Extending time and details + +
- June 26, 2012
+

+

For the fun of it I added an "Extended Room" contrib to Evennia the other night.

+

("Contribs" are optional code snippets and systems that are not part of the actual codebase. They are intended for you to use or dissect as you like in your game development efforts).

+

The ExtendedRoom is a room typeclass meant to showcase some more advanced features than the default one. Its functionality is by all means nothing revolutionary in MUD-world, but it was fun and very simple to do using only Evennia's basic building blocks - the whole thing took me some two hours to code, document and clean up for a contrib push. The "ExtendedRoom" contribution has the following features:

+
    +
  • +

    Season-based descriptions. The new Room typeclass will change its overall description based on the time of year (the contrib supports the four seasons, you can hack this as you please). It's interesting from an implementation point of view since it doesn't require any Script or ticker at all - it just checks on-demand, whenever it is being looked at, only updating if the season has actually changed. There is also a general description used as a fallback in case of a missing seasonal one.

    +
    • +
  • +

    Time-of-day-based descriptions. Within each Season-based description you can embed time-of-day based ones with special tags. The contrib supports four time slots out of the box (morning, afternoon, evening, night). In the description, you just embed time-dependent text within tags, like Morning sunlight is shining through the windows. Only time-relevant tags will be shown. This is a simple regular expression substitution, should be easy to expand on if one wants more fine-grained time slots.

    +
    • +
  • +

    Details. I took the inspiration of these from a MOO tutorial I read a long time ago. "Details" are "virtual" look-targets in the room. It allows you to add visual interest without having to add a bunch of actual objects for that purpose. Details are simply stored in a dictionary on the room. Details don't change with Season in this implementation, but they are parsed for time-of-day based tags!

    +
    • +
  • +

    Custom commands. The room is supported by extending two of the custom commands. The Details require a slightly modified version of the look command. There is also a new @desc for setting/listing details and seasonal descriptions. The new time command, finally, simply shows the current game time and season in the room.

    +
    • +
+

Installing and testing the snippet is simple - just add the new commands to the default cmdset (they will dynamically replace the same-named default ones), dig a few rooms of the new typeclass and play around! Especially the details do make building interesting rooms a lot more fun (I got hung up playing with them way too long last night).

+ +
+

+ +

+
+
+ +
+
+

+ Coding from the inside + +
- June 11, 2012
+

+

+

Some time ago, a message on the Evennia mailing list asked about "softcode" support in Evennia. Softcode, a defacto standard in the MUX/MUCK/MUSH/MOO world, is conceptually a "safe" in-game scripting language that allows Players to extend the functionality of things without having access to the server source.

+

Now, Evennia is meant to be extended by normal Python modules. For coding game systems and advanced stuff, there is really no reason (in my opinion) for a small development team to not use a modern version control system and proper text editors rather than entering things on a command line without formatting.

+

But there is a potential fun aspect of having an online scripting language - and that is player content creation. Crafters wanting to add some pizazz to their objects, builders getting an extra venue of creativity with their rooms - that kind of thing. I didn't plan to add softcode support to Evennia, but it "sounded like an interesting problem" and one thing led to another.

+

Python is of course an excellent scripting language from the start. Problem is that it's notoriously tricky to make it run safely with untrusted code - like that inserted by careless or even potentially malignant Players. Scanning the Internet on this topic is a pretty intimidating experience - everywhere you hear that it shouldn't be done, and that the various suggested solutions of a "sandbox" are all inherently unsafe. Python's awesome introspection utilities is its own enemy in this particular case.

+

For Evennia we are however not looking for a full sandbox. We want a Python-like way for Players to influence a few determined systems. Moreover, we expect short, simple scripts that can do without most of Python's functionality (since our policy is that if it's too complex or large, it belongs in an external Python module). We could supply black-box "safe" functions to hide away complex functionality while still letting people change things we expect them to want to script. This willingness to accept heavy restrictions to the language should work to our advantage, I hope.

+

Evennia actually already has a safe "mini-language" in the form its "lock system", and thus it was a natural way for me to start looking. A "lock string" has a very restricted syntax - it's basically function calls optionally separated by boolean operators, like this:

+
+

lockfunc1(*args) and lockfunc(*args, **kwargs) and not lockfunc2()

+
+

The result of this evaluation will always be a boolean True/False (if the lock is passed or not). Only certain functions are available to use (controlled by the coder). The interesting thing is that this string can be supplied by the Player, but it is not _eval_uated - rather it's manually parsed, from left to right. The function names and arguments are identified (as for the rest, only and/or/not are allowed). The functions are then called explicitly (in Python code, not evaluated as a string) and combined to get a result. This should be perfectly safe as long as your functions are well-defined.

+

For the potential softcode language, I first took this hands-on approach - manually parsing the string into its components. I got a pretty decent demo going, but the possibilities are much larger than in the simple lockstring case. Just parsing would be one thing, but then to also make sure that each part is okay to use too is another matter ... It would probably be doable, but then I got to supplying some sort of flow-control. The code starts to become littered with special cases which is never a good sign.

+

So eventually I drifted off from the "lock-like" approach and looked into Python's ast module. This allows you to view Python code as an "abstract syntax tree" (AST).  This solves the parsing issues but once you start dealing with the AST tree you are sort of looking at the problem from the other end - rather than parsing and building the script from scratch it more becomes a matter of removing what is already there (an AST tree can be compiled directly back into Python code after all). It nevertheless seemed like the best way forward.

+

Testing a few different recipes from the web, I eventually settled on an approach which (with some modifications compared to the original) uses a whitelist (and a blacklist for some other things) to only allow a given set of ast nodes and items in the execution environment. It walks the AST tree before execution and kills dangerous Python structures in one large swath. I expanded on this a fair bit, cutting away a lot of Python functionality for our particular use case. Stuff like attribute acces and assignments, while loops and many other Pythonesque things went out the window.

+

Around this highly stunted execution system I then built the Evennia in-game scripting system. This includes in-game commands as well as scriptable objects with suitable slots defining certain functionality the Player might want to change. Each Evennia developer can also supply any set of "safe" blackbox functions to offer more functionality to their Player-coders.

+

A drawback is the lack of a functional timeout watchdog in case of a script running too long. I'm using Twisted's deferToThread to make sure the code blocks as little as possible, but alas, whereas I can check timeouts just fine,  the problem lies in reliably killing the errant sub-thread. Internet experts suggest this to be tricky to do safely at the best of times (with threads running arbitrary code at least), and not wanting to kill the Twisted server is not helping things. I pondered using Twisted's subprocess support, but haven't gotten further into that at this point. Fact is that most of the obvious DOS attack vectors (such as the while loop and huge powers etc) are completely disabled, so it should hopefully not be trivial to DOS the system (famous last words).

+

I've tentatively dubbed the softcode system "Evlang" to differentiate it from our normal database-related "Scripts".

+

So is Evlang "safe" to use by untrusted evil Players? Well, suffice to say I'm putting it up with a huge EXPERIMENTAL flag, with plenty of warnings and mentions of "on your own risk". Running Evennia in a chroot jail and with minimum permissions is probably to recommend for the security paranoid. Hopefully Evennia coders will try all sorts of nasty stuff with it in the future and report their finding in our Issue tracker!

+

But implementation details aside, I must admit it's cool to be able to add custom code like this - the creative possibilities really do open up. And Python - even a stunted version of it - is really very nice to work with, also from inside the game.

+ +
+

+ +

+
+
+ +
+
+

+ Dummies doing (even more) dummy things + +
- May 30, 2012
+

+

+

This is a follow-up to the Dummies doing dummy things post. I originally posted info about this update on the mailing list some time back, but it has been pointed out to me that it might be a nice thing to put on the dev blog too since it's well, related to development!

+

I have been at it with further profiling in Evennia. Notably even more aggressive on-demand caching of objects as well as on-object attributes. I found from profiling that there was an issue with how object access checks were done - they caused the lock handler to hit the database every lock check as it retrieved the needed attributes.

+

Whereas this was not much of a hit per call, access checks are done all the time, for commands, objects, scripts, well everything that might need restricted access.

+

After caching also attributes, there is no need to hit the database as often. Some commands, such as listing all command help entries do see this effect (although you still probably wouldn't notice it unless you checked before and after like I did). More importantly, under the hood I'm happy to see that the profile for normal Evennia usage is no longer dominated by Django db calls but by the functional python code in each command - that is, in code that the end user have full control over anyway. I'd say this is a good state of affairs for a mud creation system.

+

In the previous "Dummies ..." post I ran tests with rather extreme conditions - I had dummy clients logging to basically act like heavy builders.  They dug rooms, created and defined objects randomly every five seconds (as well as walking around, reading help files, examining objects and other spurious things). In that post I found that my puny laptop could handle about 75 to 100 such builders at a time without me seeing a slowdown when playing. My old but more powerful desktop could handle some 200 or so.

+

Now, I didn't re-run these build-heavy tests with the new caches in place. I imagine the numbers will improve a bit, but it's just a guess. By all means, if you expect regularly having more than 100 builders on your game continuously creating 250 new rooms/objects per minute, do get back to me ...

+

... Instead I ran similar tests with more "normal" client usage. That is, I connected dummy clients that do what most players would do - they walk around, look at stuff, read help files and so on. I connected clients in batches of 100 at a time, letting them create accounts and logging in fully before connecting the next set of 100.

+

All in all I added 1000 dummy clients this way before I saw a noticeable lag on my small laptop. I didn't find it necessary to try the desktop at this point. Whereas this of course was with a vanilla Evennia install, I'd say it should be reasonable room for most realistic mud concepts to grow in.

+

With the rather extensive caching going on, it is interesting to know what the memory consumption is.

+

+

This graph shows memory info I noted down after adding each block of 100 players. The numbers fluctuated up and down a bit between readings (especially what the OS reported as total usage), which is why the lines are not perfectly straight.

+

In the end the database holds 1000 players (which also means there are 1000 Character objects), about as many rooms and about twice as many attributes.  The "idmapper cache" is the mapper that makes sure all Django model instances retain their references between accesses (as opposed to normal Django were you can never be sure of this). "Attribute cache" is a cache storing the attribute objects themselves on the Objects, to avoid an extra database lookup. All in all we see that keeping the entire database in memory takes about 450MB.

+

Evennia's caching is on-demand (so e.g. a room would not be loaded/cached until someone actually accessed it somehow). One could in principle run a script to clean all cached regularly if one was short on RAM - time will tell if this is something any user needs to worry about on modern hardware.

+ +
+

+ +

+
+
+ +
+
+

+ Shortcuts to goodness + +
- March 26, 2012
+

+

Evennia, being a MUD-design system, needs to take some special considerations with its source code - its sole purpose is after all to be read, understood and extended.

+

Python is of course very readable by default and we have worked hard to give extensive comments and documentation. But for a new user looking into the code for the first time, it's still a lot of stuff to take in. Evennia consists of a set of Django-style "applications" interacting and in some cases inheriting from each other so as to avoid code duplication. For a new user to get an overview could therefore mean diving into more layers of code than one would like.

+

I have now gone through the process of making Evennia's API (Application Programming Interface) "flatter". This has meant exposing some of the most commonly used methods and classes at a higher level and fully documenting exactly what they inherit av every layer one looks at. But I have also added a new module ev.py to the root directory. It  implements "shortcuts" to all the most commonly used parts of the system, forming a very flat API. This means that what used to be

+

from src.objects.objects import Object

+

can now be done as

+

from ev import Object

+

Not only should it be easier to find things (and less boilerplate code to write) but I like that one can also easier explore Evennia interactively this way.  Using a Python interpreter (I recommend ipython) you can just import ev and easily inspect all the important object classes, tab to their properties, helper functions and read their extensive doc strings.

+

Creating this API, i.e. going through and identifying all the useful entry points a developer will need, was also interesting in that it shows how small the API really is. Most of the ev interface is really various search functions and convenient options to inspect the database in various ways. The MUD-specific parts of the API is really lean, as befits a barebones MUD server/creation system.

+ +
+

+ +

+
+
+ +
+
+

+ Dummies doing dummy things + +
- February 22, 2012
+

+

It can often be interesting to test hypothetical situations. So the other day I did a little stress test to see how Evennia handles many players. This is a follow-up to the open bottlenecks post.

+

Evennia, being based on Twisted, is an asynchronous mud server. This means that the program is not multi-threaded but instead it tries to divide up the code it runs into smaller parts. It then quickly switches between executing these parts in turn, giving the impression of doing many things at the same time. There is nothing magical about this - each little piece of code blocks the queue as it runs. So if a particular part takes too long, everyone else will have to wait for their turn. Likewise, if Twisted cannot flip through the queue as fast or faster than new things get added to it, it will start to fall behind.

+

The good thing is that all code in the queue will run, although if the event loop cannot keep up there will be a noticeable delay before it does. In a mud, this results in "lagging" (in addition to whatever lag is introduced by your network connection). Running Evennia with a handful of users shows no effect of this, but what happens when we start to add more and more players?

+

My "dummy runner" is a stand alone Twisted program that opens any number of Telnet connections to the Evennia server. Each such "dummy" client is meant to represent a player connecting. In order to mimic actual usage, a dummy client can perform a range of actions:

+
    +
  • +

    They can create a new account and log in

    +
    • +
  • +

    They can look around and read random help files

    +
    • +
  • +

    They can create objects, name and describe them (for testing, the server is set to grant build privileges to all new players)

    +
    • +
  • +

    They can examine objects, rooms and themselves

    +
    • +
  • +

    They can dig new rooms, naming all exits and giving aliases

    +
    • +
  • +

    They can move between rooms

    +
    • +
+

The clients tick every 5 seconds, at which time there is a 10% chance each will perform an action from the list above (login first, then one of the others at random).  This is meant to spread out the usage a bit, like it would be with real people. Some of these actions actually consist of multiple commands sent to the server (create + describe + set etc), possibly simulating people using shortcuts in their client to send several commands at the same time.

+

Results

+

Note that I didn't do a proper objective benchmark. Rather, I logged in and ran commands to see, very subjectively, how the game "felt" with a number of different players. The lag times are rough estimates by putting time() printouts in the server.

+

First I tried with my development laptop, a Thinkpad X61s. It's about 5 years old by now and certainly not the fastest thing out there, mainly it's small and thin and has great battery life. I used a MySQL database.

+
    +
  • +

    1-50 dummy players didn't show any real difference from playing alone. It's not so strange; 50 players meant on average 5 of them would do an action every 5 seconds. My laptop could handle that just fine.

    +
    • +
  • +

    50-75 dummy players introduced a brief lag (less than a second) when they logged in. 5-7 players logging in at exactly the same time will after all mean a lot of commands sent to the server (they not only log in, they create a new character at the same time). Throughout these tests, clients logging in showed the greatest effect on lag. I think this is mostly an artifact of how the clients operate by sending all relevant login commands at the same time. Once all were logged in, only brief lag occurred at random times as coincidence had all clients do something expensive at the same time.

    +
    • +
  • +

    75-100 dummy players introduced longer lags during login, as on average 7-10 clients were now logging in at exactly the same time. Most commands were unaffected, but occasionally there were some noticeable "hiccups" of about a second depending on what the clients were doing.

    +
    • +
  • +

    100-150 dummy players - here things started to go downhill rapidly. Dummy client login caused seriously lagging and at 150 logged in players, there were 15 exactly simultaneous actions coming in. This was clearly too much for my laptop; it lead to a very varying lag of 1-10 seconds also for simple commands.

    +
    • +
+

Next I tried my desktop machine. This is also not bleeding edge; it's a 4-year old machine with 3GHz processor and 4GB RAM. Since I don't have MySQL on this machine, I used SQLite3, which is interesting in its own right since it's Evennia's default database.

+
    +
  • +

    1-75 dummy players didn't affect the feel of the game one bit.

    +
    • +
  • +

    75-100 showed some occasional effects starting to appear during dummy client logins.

    +
    • +
  • +

    100-150 dummy players didn't introduce more than a few hiccups of less than a second when approximately 15 clients decided to do something expensive at the same time.

    +
    • +
  • +

    150-200 introduces 2-3 seconds lag during client mass-logins, but once all clients had connected, things worked nicely with only brief random hiccups.

    +
    • +
  • +

    200-250 showed the lag getting more uneven, varying from no lag at all to 2-3 seconds for normal times and up to 5 seconds when clients were logging in.

    +
    • +
  • +

    250-300 dummy players showed login lag getting worse. The general lag varied from 0-5 seconds depending on what the other clients were up to.

    +
    • +
  • +

    300-350 dummy players, about double of what the laptop could handle,  the CPU was not able to keep up anymore. The system remained stable and all commands got executed - eventually. But the lag times went up very rapidly.

    +
    • +
+

Conclusions

+

So, based on this I would say 50-75 was a comfortable maximum number of (dummy) players to have on my laptop whereas the desktop could handle around 150 without much effort, maybe up to 250 on a good day.

+

So what does these numbers mean? Well, the dummy clients are rather extreme, and 100+ builders walking around building stuff every 5 seconds is not something one will see in a game. Also so many logging on at the same time is not very likely (although it could happen after a crash or similar). If anything the usage pattern of human players will be more random and spread out, which helps the server to catch up on its event queue.

+

On the other hand these tests were run with a vanilla Evennia install - a full game might introduce more expensive commands and subsystems. Human players may also introduce random spikes of usage. So take this for what it is - a most subjective, un-scientific and back-of-the-envelope measure.

+

All in all though, I would say that few MUDs these days see 30 concurrent online users, even less 100 ...

+ +
+

+ +

+
+
+ +
+
+

+ Commands and you + +
- February 17, 2012
+

+

Commands define how a Player interacts with a given game. In a text-based game it's not amiss to say that the available commands are paramount to the user experience. In principle commands could represent mouse clicks and other modernistic GUI sugar - but for this blog I'll stick with the traditional entered text.

+

Like most things in Evennia, Commands are Python classes. If you read the documentation about them you'll find that the command classes are clumped together and tacked onto all objects in-game. Commands hanging onto a Character object will be available all the time, whereas commands tacked onto a grandfather's clock will only be available to you when you stand in front of said clock.

+

The interesting thing with Commands being classes is that each Character gets a separate instance of each command. So when you do look 100 times in a row, it's always the same Look command instance that has its methods called by the engine. Apart from being an efficient way to handle things, this has a major neat side-effect:

+
+

You can store things on the Command object and whatever you store can be retrieved next time you execute that command.

+
+

I find this very cool mainly because I didn't really plan with this in mind when designing the command system - it was a happy side effect. A use I have thought of is to implement cooldowns. Say you have a powerful bash command. It makes a lot of damage, but you need time to recover between bashes. So when you do the bash command the Bash command object simply stores the current time on itself:

+
+

self.last_bash = time.time()

+
+

Next time the Player tries to use bash, all the command object needs to is to check if self.last_bash is set, and compare the time stored herein with the current time. No twisted tasks needed, no overhead. Very neat and tidy.

+

Another nice functionality (just added today in fact) is that Evennia can be set to store a copy of the last command object executed. What can one do with this? For starters, it allows for commands to check what a previous command was. This can be useful in itself, but since the next command actually have access to (a copy of) the previous command object itself, it will allow a lot more than that.

+

Consider a look command that remembers whatever object it is looking at. Since the Look command is a Python object, it simply stores the looked-at object on itself before returning the normal info to the Player. Next, let's assume we use a get command. If no argument is given to this get (no given object to pick up), the get normally returns an error. But it can now instead peek at the previous command (look) and see what that command operated on. This allows for nice potential constructs like

+
+
+
+

look [at] box

+
+
+
+
+
+
+

get [it]

+
+
+
+

Evennia does not use this functionality in its default command set, but it offers some very powerful possibilities for MUD creators to design clever parsing schemes.

+ +
+

+ +

+
+
+ +
+
+

+ Such a small thing ... + +
- February 15, 2012
+

+

Lately I went back to clean up and optimize the workings of Evennia's Attributes. I had a nice idea for making the code easier to read and also faster by caching more aggressively. The end result was of course that I managed to break things. In the end it took me two weeks to get my new scheme to a state where it did what it already did before (although faster).

+

Doing so, some of the trickier aspects of implementing easily accessible Attributes came back into view, and I thought I'd cover them here. Python intricacies and black magic to follow. You have been warned.

+

Attributes are, in Evennia lingo, arbitrary things a user may want to permanently store on an object, script or player. It could be numbers or strings like health, mana or short descriptions, but also more advanced stuff like lists, dictionaries or custom Python objects.

+

Now, Evennia allows this syntax for defining an attribute on e.g. the object myobj:

+
+

myobj.db.test = [1,2,3,4]

+
+

This very Pythonic-looking thing allows a user to transparently save that list (or whatever) to an attribute named, in this example, test. This will save to the database.

+

What happens is that db, which is a special object defined on all Evennia objects, takes all attributes on itself and saves them by overloading its setattr default method (you can actually skip writing db most of the time, and just use this like you would any Python attribute, but that's another story).

+

Vice-versa,

+
+

value = myobj.db.test

+
+

This makes use of the db object's custom get_attribute method behind the scenes. The test attribute is transparently retrieved from the database (or cache) for you.

+

Now, the (that is, my) headache comes when you try to do this:

+
+

myobj.db.test[3] = 5

+
+

Such a small, normal thing to do! Looks simple, right? It is actually trickier than it looks to allow for this basic functionality.

+

The problem is that Python do everything by reference. The list is a separate object and has no idea it is connected to db. db's get_attribute is called, and happily hands over the list test. And then db is out of the picture!. My nifty save-to-database feature (which sits in db) knows nothing about how the 3rd index of the list test now has a 5 instead of a 4.

+

Now, of course, you could always do this:

+
+

temp = myobj.db.test

+
+
+

temp[3] = 5

+
+
+

myobj.db.test = temp

+
+

This will work fine. It is however also clumsy and hardly intuitive. The only solution I have been able to come up with is to have db return something which is almost a list but not quite. It's in fact returning an object I not-so-imaginatively named a PackedList_._ This object works just like a list, except all modifying methods on it makes sure to save the result to the database. So for example, what is called when you do mylist[3] = 4 is a method on the list named setitem. I overload this, lets it do its usual thing, then call the save.

+
+

myobj.db.test[3] = 5

+
+

now works fine, since test is in fact a PackedList and knows that changes to it should be saved to the database. I do the same for dictionaries and for nested combinations of lists and dictionaries. So all is nice and dandy, right?  Things work just like Python now?

+

No, unfortunately not. Consider this:

+
+

myobj.db.test = [1, 3, 4, [5, 6, 7]]

+
+

A list with a list inside it. This is perfectly legal, and you can access all parts of this just fine:

+
+

val = myobj.db.test[3][2] # returns 7!

+
+

But how about assigning data to that internal nested list?

+
+

myobj.db.test[3][2] = 8

+
+

We would now expect test to be [1, 3, 4, [5, 6, 8]]. It is not. It is infact only [5, 6, 8]. The inner list has replaced the entire attribute!

+

What actually happens here? db returns a nested structure of two _PackedList_s. All nice and dandy.  But Python thinks they are two separate objects! The main list holds a reference to the internal list, but as far as I know there is no way for the nested list to get the back-reference to the list holding it! As far as the nested list knows, it is all alone in the world, and therefore there is no way to trigger a save in the "parent" list.

+

The result is that we update the nested list just fine - and that triggers the save operation to neatly overwrite the main list in the cache and the database.

+

This latter problem is not something I've been able to solve. The only way around it seems to use a temporary variable, assign properly, then save it back, as suggested earlier. I'm thinking this is a fundamental limitation in the way cPython is implemented, but maybe I'm missing some clever hack here (so anyone reading who has a better solution)?

+

Either way, the db functionality makes for easy coding when saving things to the database, so despite it not working quite like normal Python, I think it's pretty useful.

+ +
+

+ +

+
+
+ +
+
+

+ Evennia's open bottlenecks + +
- February 5, 2012
+

+

Since Evennia hit beta I have been mostly looking behind the scenes to see what can be cleaned up and not in the core server. One way to do that is to check where the bottlenecks are. Since a few commits, Evennia's runner has additions for launching the Python cProfiler at startup. This can be done for both the Portal and the Server separately.

+

I have created a test runner (soon to hit trunk) that launches dummy clients. They log on and then goes about their way executing commands, creating objects and so on. The result of looking at the profiling data (using e.g. runsnake) has been very interesting.

+

Firstly, a comparably small part of execution time is actually spent in Evennia modules - most is spent using Python built-ins and in the Twisted/Django libraries. This is promising in a way - at least there are no expensive loops in the Evennia code itself that sucks up cycles. Of course it also means we depend heavily on django/twisted (no surprise there) and especially when it comes to database access, I know there could be more caching going on so as to cut down on db calls.

+

Of the Evennia modules, a lot of time is spent in getting properties off objects - this is hard to avoid, it a side effect of Evennia's typeclass system - and this is cached very aggressively already. What also stands out as taking a considerable time is the command system - the merging of command sets in particular does a lot of comparing operations. This happens every time a command is called, so there are things to be done there. The same goes for the help system; it needs to collect all the currently active command sets for the calling player. Maybe this could be cached somehow.

+

More work on this is needed, but as usual optimization is a double-edged sword. Evennia is written to have a very readable code. Optimization is usually the opposite of readable, so one needs to thread carefully.

+ +
+

+ +

+
+
+ +
+
+

+ About this dev blog + +
- February 5, 2012
+

+

This is to be my Evennia dev blog, but it will also cover various other musings concerning programming in general and mud designing in particular. Whereas the Evennia mailing list remains the main venue for discussion, I will probably use this blog for announcing features too.

+

Some background:

+

Evennia is a Python MUD/MUX/MU* server. More correct is probably to call it a "MUD-building system". The developer codes their entire game using normal Python modules. All development is done with the full power of Python - gritty stuff like database operations and network communication are hidden away behind custom classes that you can treat and modify mostly like any Python primitives.

+

Since the server is based on the Twisted and Django technologies we can offer many modern features out of the box. Evennia is for example its own web server and comes with both its own website and an "comet"-style browser mud client. But before this turns into even more of a sales pitch, I'll just just direct you to the evennia website if you want to know more. :)

+

I, Griatch, took over the development of Evennia from the original author, Greg Taylor, in 2010.

+ +
+

+ +

+
+
+ +
+ + \ No newline at end of file diff --git a/devblog/html/devblog2013.html b/devblog/html/devblog2013.html new file mode 100644 index 0000000000..5b0b54c2d5 --- /dev/null +++ b/devblog/html/devblog2013.html @@ -0,0 +1,579 @@ + + + + + + Devblog: Evennia Python MUD/MU creation system + + + + + + + + + + +
+
+

+ +

+

Evennia

+

The Python MU* Development Library

+
+ +
+
+ +
+
+
+

+ + + + Evennia Dev Blog

+ + Random Evennia- and MUD-related musings of Griatch, the Evennia lead developer. + + + +
+

+ Imaginary Realities is back + +
- December 16, 2013
+

+

The Imaginariy Realities webzine was the place to go to for MUD game design articles in the late 90's. Last released in 2001, its articles are still worth the read for any game designers today.

+

But guess what - this venerable ezine has now returned! You can find the new issue here.

+

I think this is a good community initiative worthy of support. I contibuted two articles myself (one of which is about Evennia) and would like to thank the staff/editors who took their work very seriously and did sterling work on getting everything in shape.

+

Thanks also to the other authors who penned some very interesting articles. Great job guys!

+

My impressions:

+
    +
  • +

    KaVir elaborates in A modern interface for a modern MUD on the advantages of not sticking with an outdated UI just for the sake of it. Adding a more accessible presentation is not that hard and won't ruin your game but rather help it. Whereas I have read his argument about this before, this is a good summary to take to heart. Evennia's javascript web client is currently mainly a telnet clone; there's a lot of things we could offer to make it easier for users to offer a more graphical presentation.

    +
    • +
  • +

    Molly O’Hara, in her A well built zone is a work of art, outlines a list of useful things to keep in mind when designing a zone. While some of these vary with game type, others do not. I like the suggestion that scripting bugs need not be the most important aspect - syntactic errors can be handled by automated means as long as the design aspect of the zone is solid.

    +
    • +
  • +

    A journey through Paradice [sic] is Matthew Chaplain's entry on designing a dice-roller using the telnet protocol. Some interesting things here, including making creative use of the telnet character-mode and VT100 control sequences. This ties a bit into KaVir's article, in that the interface used for "modern" MUDs are often voefully missing out on a lot of possibilities.

    +
    • +
  • +

    Blind accessibility: challenges and opportunities by Matthew “Chaos” Sheahan, is based on interviews with a blind mudder and a game admin having implemented lots of support for seeing-impaired players. This was a really interesting article since I've been pondering myself what could be done from Evennia's core side to help players support players using screen readers. Most seem to be down to configuration options though, and avoiding making colour or ascii art the only sources of information. These are all things Evennia devs implement depend on their game. We may offer some good contribs to build from though.

    +
    • +
  • +

    Evennia: an introduction - this is mine. It not-so-briefly sums up stuff about Evennia and the more important systems it relies on.

    +
    • +
  • +

    Getting a roleplaying scene going - another article of mine. This is a light-hearted list of tropes for getting a RP scene going on an RP-mud. It's based on things I've tried or seen myself in play.

    +
    • +
  • +

    Darcie “Natilena” Laur laments on the often opaque newbie guides in Introducing new players and redesigning MUD School. It describes how she tested (and improved) her own MUD's starter area while testing it on her kids. It made me think more on having Evennia offering easier ways to dump text logs in all sorts of situations. And we find out that kids have the attention span of zombie squirrels - something new learned every day!

    +
    • +
  • +

    Finally, The Hunger Game, or how I learned to break the ship from the bottle is Michael “Drakkos” Heron's epic about his journey developing and releasing his zombie-survival MUD Epitaph. Drakkos is a frequent blogger on the MUD-planet feed, so I knew some of this already, but it's a good read and contains some useful warnings and things-to-think-of for those thinking of starting their own MUD project. We already give some of the same advice (albeit with fewer words) in our wiki but I'm seriously considering linking to Drakkos post from there as well - it gives a more comprehensive treatment and offers a real-world example of the long road to a released game.

    +
    • +
+

The issue also contains a call to help recover old MUDing resources that has gone offline since their heyday. Having browsed through the classic MUD-dev mailing list archives I can agree they are not quite as easily searchable as could be hoped. It would also be great to see the old comment posts tied to the articles of the old Imaginary Realities articles, if those could be found somehow.

+

This is actually one thing which I do miss with this first Imaginary Realities issue - a way for readers to comment on the articles. This would likely mean a much higher level of complexity and work though, so I can certainly see why it's not there - using the existing MUD forums is probably enough for now.

+

Anyway, I'm happy to see this thing getting off on a good start. I'm already looking forward to the next issue!

+ +
+

+ +

+
+
+ +
+
+

+ Out of band mergings + +
- November 28, 2013
+

+

+

As of today the development repository of Evennia, which has been brewing for a few months now, merged into the main repository. This update grew from one experimental feature to a relatively big update in the end. Together with the "many-character-per-player" feature released earlier, this update covers all the stuff I talked about in my Behind the Scenes blog post.

+
    +
  • +

    Evennia's webserver was moved from Portal to Server. This moves all database-modifying operations into the same process and neatly avoids race conditions when modifying a game world from various interfaces.

    +
    • +
  • +

    The OOB (Out Of Band) handler was implemented. This goes together with a protocol for telnet sub-negotiations according to the MSDP specification. The handler allows on-demand reporting whenever database fields update. It also offers regular polling of properties if needed. A user can customize which oob commands are available to the client and write whatever handlers are needed for their particular game. In the future we'll also add support for GMCP, but the lack of a central, official specification is off-putting (if there is a central document besides accounts of how individual games chose to implement GMCP, please let me know). For our own included web client, we'll likely just use JSON straight off.

    +
    • +
  • +

    Our channel system is now typeclassed. If you are not familiar with Evennia this won't mean much to you - In short it means developers will be able to customize their channel system much easier than in the past since a channel can be treated pretty much like any Python class (thanks go to user Kelketek who actually did the implementation).

    +
    • +
  • +

    We added the concept of Tagging, as a more generalized version of our old Alias system. Tagging is just what it sounds like - it allows you to tag all your objects in order to group them and easily (and efficiently) find them later. Tagging offers a powerful way to create what other code bases refer to as "zones". There are many other possible uses though, such as having effects appear only in certainly tagged rooms, indicate which Characters have joined a particular guild and so on.

    +
    • +
  • +

    Behind the scenes there were a lot of cleanups, along with minor API changes mentioned on the mailing list. A slew of older Issues were also fixed with this merge.

    +
    • +
+

Now, in the immediate future we'll be working on updating the documentation. Evennia prides itself with having a large body of documentation and this update comes with small changes to various existing systems. There are also a few bigger new features (such as OOB). No doubt the near future will also see other issues popping up as more people try things out.

+

Imaginary Realities update

+

Apparently the reboot of Imaginary Realities (to which I contribute two articles) has been pushed forward a week or two. Reason being, apparently, to finalize the actual presentation of the content. I already signed off on the last editorial fixes way before deadline, so I guess it's just to wait and see what comes of it!

+ +
+

+ +

+
+
+ +
+
+

+ A list of Evennia topics + +
- October 22, 2013
+

+

+

Some Evennia updates.

+

Development

+

Lots of work has been happening in the dev-clone of Evennia over the last few months.

+

As alluded to in the last blog, the main work has been to move Evennia's webserver component into the Server-half of Evennia for various reasons, the most obvious one to make sure that all database writes happen in the same process, avoiding race conditions. But this move lead to a rework of the cache system, which in turn lead to me having to finalize the plans for how Out-of-Band protocols should be implemented server-side. And once that was finalized, OOB was pretty much implemented anyway. As part of making sure OOB trackers were updated correctly at all times meant reworking some of the ways data is stored ... So one thing led to another making this a bigger update than originally planned.

+

I plan to make a more detailed post to the mailing list soon with more technical details of the (relatively minor) API changes existing users should expect. The merging of the clone into the main repo is still a little way off, but adventurous users have already started testing things.

+

Google Code

+

I like Google Code. It's easy to manage and maintain, it has a good wiki and Issue system, not to mention that it allows the use of Mercurial. But in the beginning of September, suddenly all links to our user's clone repositories were gone from the front of the project page_._ Not only that, creating new clones just didn't work anymore.

+

Now any site can have bugs, and we made an issue for it (other projects were similarly affected). But nothing happened for the longest time - at least two months given that we didn't report it right away. Just recently the functionality came back but there is no confirmation or comments from Google (our issue is not even closed).

+

That such a fundamental feature can go unheeded for so long is disturbing to me, driving home the fact that Google is certainly not putting much priority in their code hosting.

+

Community

+

Some furious activity in the IRC chat lately, with new people dropping in to chat and ask about Evennia. For example, an enthusiastic new user learned not only about Evennia but also Python for the first time. It was a lot of fun to see him go from having no programming experience except mush softcode to doing advanced Evennia system implementations in the course of a week and offering good feedback on new features in two. Good show! The freedom you get upgrading from something like softcode to Evennia's use of a full modern programming language was seemingly quite eye-opening.

+

Other discussions have concerned the policies around using clones/branches for development as well as the benefits of some other hosting solution. Nothing has been decided on this. There is however now also an official GitHub mirror of Evennia's main repo to be found here.

+

Imaginary Realities

+

The deadline for entering articles for the Imaginary Realities web zine reboot has passed. It's a good initiative to bring this back - the original (archived) webzine remains a useful mud-creation resource to this day. I entered two articles, one about Evennia and another about general mud-roleplaying. It will be fun to see how it comes out, apparently the first issue will appear Nov 13

+ +
+

+ +

+
+
+ +
+
+

+ One to Many + +
- May 13, 2013
+

+

+

As of yesterday, I completed and merged the first of the three upcoming Evennia features I mentioned in my Churning Behind the Scenes blog post: the "Multiple Characters per Player" feature.

+

Evennia makes a strict division between Player (this is an object storing login-info and represents the person connecting to the game) and their Character (their representation in-game; Characters are just Objects with some nice defaults). When you log into the game with a client, a Session tracks that particular connection.

+

Previously the Player class would normally only handle one Session at a time. This made for an easy implementation and this behavior is quite familiar to users of many other mud code bases. There was an option to allow more than one Session, but each were then treated equally: all Sessions would see the same returns and the same in-game entities were controlled by all (and giving the quit command from one would kick all out).

+

What changed now is that the Player class will manage each Session separately, without interfering with other Sessions connected to the same Player. Each Session can be connected, through the Player, to an individual Character. So multiple Characters could in principle be controlled simultaneously by the same real-world player using different open mud clients. This gives a lot of flexibility for games supporting multi-play but also as a nice way to transparently puppet temporary extras in heavy roleplaying games.

+

It is still possible to force Evennia to accept only one Session per Player just like before, but this is now an option, not a limitation. And even in hardcore one-character-at-a-time roleplaying games it is nice for builders and admins to be able to have separate staff or npc characters without needing a separate account for each.

+

This feature took a lot more work than I anticipated - it consitutes a lot of under-the-hood changes. But it also gave me ample opportunity to fix and clean up older systems and fix bugs. The outcome is more consistency and standardization in several places. There are plenty of other noteworthy changes that were made along the way in the dev branch along with some API changes users should be aware of.

+

So if you are an Evennia game developer you should peek at the more detailed mailing list  announcement on what has changed. The wiki is not updated yet, that will come soon.

+

Now onward to the next feature!

+ +
+

+ +

+
+
+ +
+
+

+ Churning behind the scenes + +
- January 29, 2013
+

+

+

At the moment there are several Evennia projects churning along behind the scenes, none of which I've yet gotten to the point of pushing into a finished state.  Apart from bug fixes and other minor things happening, these are the main updates in the pipeline at the moment.

+

Multiple Characters per Player/Session

+

Evennia has for a long time enforced a clean separation between the Player and the Character. It's a much appreciated feature among our users. The Player is "you", the human playing the game. It knows your password, eventual user profile etc. The Character is your avatar in-game. This setup makes it easy for a Player to have many characters, and to "puppet" characters - all you need to do is "disconnect" the Player object from the Character object, then connect to another Character object (assuming you are allowed to puppet that object, obviously). So far so good.

+

What Evennia currently doesn't support is being logged in with different client sessions to the same Player/account while puppeting multiple characters at the same time. Currently multiple client sessions may log into the same Player account, but they will then all just act as separate views of the same action (all will see the same output, you can send commands from each but they will end up with the same Character).

+

Allowing each session to control a separate Character suggests changing the way the session is tracked by the player and Character. This turns out to be more work than I originally envisioned when seeing the feature request in the issue tracker. But if my plan works out it will indeed become quite easy to use Evennia to both allow multi-play or not as you please, without having to remember separate passwords for each Character/account.

+

Webserver change to Server level

+

Evennia consists of two main processes, the Portal and the Server. The details of those were covered in an earlier blog post here. Evennia comes with a Twisted-based webserver which is currently operating on the Portal level. This has the advantage of not being affected by Server-reboots. The drawback is however that being in a different process from the main Server, accessing the database and notably its server-side caches becomes a problem - changing the database from the Portal side does not automatically update the caches on the Server side, telling them that the database has changed. Also writing to the database from two processes may introduce race conditions.

+

For our simple default setup (like a website just listing some database statistics) this is not a terrible problem, but as more users start to use Evennia, there is a growing interest in more advanced uses of the webserver. Several developers want to use the webserver to build game-related rich website experiences for their games - online character generation, tie-in forums and things like that. Out-of-sync caches then becomes a real concern.

+

One way around this could be to implement a framework (such as memcached) for homogenizing caches across all Evennia processes. After lots of IRC discussions I'm going with what seems to be the more elegant and clean solution though - moving the webserver into the Server process altogether. The Portal side will thus only hold a web proxy and the webclient protocol. This way all database access will happen from the same process simplifying things a lot. It will make it much easier for users to use django to create rich web experiences without having to worry about pesky behind the scenes things like caches and the like.

+

Out-of-band communication

+

This has been "brewing" for quite some time, I've been strangely unmotivated to finalize it. Out of band communication means the MUD client can send and receive data to/from the server directly, without the player having to necessesarily enter an active command or see any immediate effect. This could be things like updating a health bar in a client-side GUI, redirect text to a specific client window but also potentially more advanced stuff. I created the Evennia-side oob-handler over Christmas; it allows for client sessions to "sign up" for "listening" to attribute updates, do scheduled checks and so on. It's already in the codebase but is not activated nor tested yet.

+

On the protocol side (for serializing data to the client) I have a MSDP implementation ready for telnet subnegotiation, it should be simple to add also GMCP once everything is tested. A JSON-based side channel for the webclient is already in place since a long time if I remember correctly, it just need to be connected to the server-side oob-handler once that's finished.

+ +
+

+ +

+
+
+ +
+ + \ No newline at end of file diff --git a/devblog/html/devblog2014.html b/devblog/html/devblog2014.html new file mode 100644 index 0000000000..f0a5af0f17 --- /dev/null +++ b/devblog/html/devblog2014.html @@ -0,0 +1,735 @@ + + + + + + Devblog: Evennia Python MUD/MU creation system + + + + + + + + + + +
+
+

+ +

+

Evennia

+

The Python MU* Development Library

+
+ +
+
+ +
+
+
+

+ + + + Evennia Dev Blog

+ + Random Evennia- and MUD-related musings of Griatch, the Evennia lead developer. + + + +
+

+ Slowly moving through town + +
- October 2, 2014
+

+

+

After getting questions about it I recently added the Slow Exit contribution to the main repository as an example.

+

Delayed movement is something often seen in various text games, it simply means that the time to move from room to room is artificially extended.

+

Evennia's default model uses traditional MU* rooms. These are simple nodes with exits linking them together. Such Rooms have no internal size and no inherent spatial relationship to each other. Moving from any Room to any other is happening as fast as the system can process the movement.

+

Introducing a delay on exit traversal can have a surprisingly big effect on a game:

+
    +
  • +

    It dramatically changes the "feel" of the game. It often makes the game feel less "twitch" and slows things down in a very real way. It lets Players consider movement as a "cost".

    +
    • +
  • +

    It simulates movement speed. A "quick" (or maybe well-rested) character might perceive an actual difference in traversal. The traversal speed can vary depending on if the Character is "running" or "walking".

    +
    • +
  • +

    It can emulate travel distance. An Exit leading to "the top of the mountain" may take longer to traverse than going "inside the tent".

    +
    • +
  • +

    It makes movement a "cost" to take into consideration in the game. Moving back and forth over and over across a distance of multiple rooms becomes a much more daunting prospect with a time delay than if you could just zip along as quickly as you could press the button. This also has effects on map and quest design.

    +
    • +
+

Introducing delayed movement in Evennia is simple. But to explain the idea, let's first briefly explain how Evennia implements Exits.

+

A brief sideline: About Exits

+

An Exit in Evennia is a persistent Object sitting in a room. The Exit class is just like any Object except for two things - it stores a "destination" property and it houses a CommandSet on itself. This particular CommandSet holds a single command with the same name as the Exit object.

+

Commands and CommandSets are things I've covered in earlier blog posts. Suffice to say is that any number of command sets can be merged together dynamically to at any moment represent the commands available to the Character at any given time or situation.

+

What happens when an Exit bject is in the same room as a Character is that the Exit's command set is dynamically merged with that of the Character. This means a new command - which always has the same name as the Exit - becomes available. The result is that if the Exit object is called "south", the Character can use the command "south". By default all the command does is to call a hook method on the Exit object. This hook hooks simply moves the calling Character to the "destination" stored by the Exit. Done!

+

The nice thing with this is that the whole system is implemented without any special cases or custom hard-wired code. It also means that the entire Exit system can be changed and modified without ever touching Evennia's core.

+

Delaying Exits

+

To delay the traversal, the principle is simple - after the Exit command has triggered, wait for a little while before continuing.

+

Technically we define a new class of Exit, let's call it SlowExit, inheriting from the default Exit. We locate the spot where the Exit normally sends traversing objects on their way (this is a method called move_to()).

+

Since Evennia is based on Twisted, we use Twisted's intrinsic CallLater() function to delay the move for as many seconds we desire (in the contrib I use a thin wrapper around CallLater called delay()). The result is that the command is called, you get a little text saying that you have started moving ... and a few seconds later you actually move.

+

Once one understands how Exits work it's really quite straight forward - see the code on github for more details (it's got plenty of comments).

+

In the contrib are also some example utility commands for setting one's movement speed and to abort movement if you change your mind before the timeout has passed.

+

This simple start can easily be expanded as befits each individual game. One can imagine introducing anything from stamina costs to make travel time be dynamically calculated based on terrain or other factors.

+ +
+

+ +

+
+
+ +
+
+

+ Dance my puppets + +
- August 4, 2014
+

+

+

In many traditional multiplayer text engines for MUD/MUSH/MU*, the player connects to the game with an account name that also becomes their character's in-game name. When they log into the game they immediately "become" that character. If they want to play with another character, they need to create a new account.

+

A single-login system is easy to implement but many code bases try to expand with some sort of "account system" where a single login "account" will allow you to manage one or more game characters. Matthew “Chaos” Sheahan  beautifully argues for the benefits of an account system in the April issue of Imaginary Realities; you can read his article here.

+

Evennia and account systems

+

First a brief show of how Evennia handles this. We use the following separation:

+

Session(s) <-> Player <-> Objects/Characters(s)

+

The Session object represents individual client connections to Evennia. The Player is our "account" object. It holds the password hash and your login name but has no in-game existence. Finally we have Objects, the most common being a subclass of Object we call Character. Objects exist in the game. They are "puppeted" by Sessions via the Player account.

+

From this separation an account system follows naturally. Evennia also offers fully flexible puppeting out of the box: Changing characters (or for staff to puppet an NPC) is simply a matter of "disconnecting" from one Character and connecting to another (presuming you have permission to do so).

+

The Multisession modes of Evennia

+

This is the main gist of this entry since we just added another of these (mode 3). Evennia now offers four different multisession modes for the game designer to choose between. They affect how you gamers may control their characters and can be changed with just a server reload.

+

Mode 0

+

This is emulates the "traditional" mud codebase style. In mode 0 a Session controls one Character and one character only. Only one Session per account is allowed - that is, if a user try to connect to their Player account with a different client the old connection will be disconnected. In the default command set a new Character is created with the same name as the Player account and the two are automatically connected whenever they log in. To the user this makes Player and Character seem to be virtually the same thing.

+

Mode 1

+

In this mode, multiple Sessions are allowed per Player account. You still only have one Character per account but you can control that Character from any number of simultaneously connected clients. This is a requirement from MUSHes and some slower-moving games where there are communities of gamers who want to conveniently track the progress of the game continuously on multiple clients and computers.

+

Mode 2

+

In multisession mode 2, multiple Characters are allowed per Player account. No Characters are created by default in this mode, rather the default command set will drop you to a simplified OOC management screen where you can create new characters, list the ones you already have and puppet them. This mode offers true multiplaying, where you can connect via several clients simultaneously, each Session controlling a different Character.

+

Mode 3

+

This mode allows gamers not only to play multiple Characters on the same Player account (as in mode 2) but to also connect multiple Sessions to each Character. This is a multi-character version of Mode 1, where players can control the same Character via Player logins from several different clients on different machines in any combination.

+

It's interesting that some of these modes may seem silly or superfluous to people used to a certain type of MU* yet are killer features for other communities. It goes to show how different the needs are for users of different game styles.

+ +
+

+ +

+
+
+ +
+
+

+ Webby stuff + +
- June 30, 2014
+

+

+

Latest Evennia come with a range of improvements, mainly related to its integration with the web.

+

New and improved ways to expand the website/webclient

+

Thanks to the work of contributor Kelketek, Evennia's Django-based web system (website and webclient) has been restructured to be much easier to expand. Previously you had to basically copy the entire web/ folder into your game and modify things in-place. This was not ideal since it made it inherently harder to update when things changed upstream. Now Evennia makes use of Django's collectstatic functionality to allow people to plugin and overload only the media files and templates that they need. Kelketek wrote a new and shiny web tutorial explaining just how things work.

+

Websocket-based webclient with OOB

+

Evennia's webclient was an ajax-based one using a long polling ("comet") paradigm to work. These days all modern browsers support websockets though, a protocol that allows asynchronous server-client communication without the cludgery of long polling. So Evennia's new webclient will now use websockets if the browser supports it and fall back to the old comet client if it does not.

+

The new client also has full support for OOB (Out-of-band) communication. The client uses JSON for straight forward OOB messaging with the server. As part of this, I had an excuse to go back to clean up and make the OOB backbone of Evennia more complete. The server-side oob commands are borrowed from MSDP but the server side is of course independent of communication protocol (so webclient and telnet extensions can call the same server-side callbacks). I've not yet finalized the documentation for how to use the OOB yet, that will come soon-ish.

+ +
+

+ +

+
+
+ +
+
+

+ Bringing back Python memory + +
- June 15, 2014
+

+

+

Lately I've done work on the memory management of Evennia. Analyzing the memory footprint of a python program is a rather educational thing in general.

+

Python keeps tracks of all objects (from variables to classes and everything in between) via a memory reference. When other objects reference that object it tracks this too.

+

Once nothing references an object, it does not need to be in memory any more - in a more low-level languages this might lead to a memory leak. Python's garbage collector handles this for us though - it goes through all abandoned objects and frees the memory for usage by other things. The garbage collector will however not do its thing as long as some other object (which will not be garbage-collected) still holds a reference to the object. This is what you want - you don't want existing objects to stop working because an object they rely on is suddenly not there.

+

Normally in Django, whenever you retrieve an database model instance, that exists only in memory then and there. If you later retrieve the same object from the database, the model instance you have to work with is most likely a new one. This is okay for most usage, but Evennia's typeclass system (described in an earlier blog entry) as well our wish to store temporary properties on models (existing until next server restart) does not work if the model instance we get is always different. It would also help if we didn't have to load models from the database more than necessary.

+

For this reason, Evennia uses something called the idmapper. This is a cache mechanism (heavily modified for Evennia) that allows objects to be loaded from the database only once and then be reused when later accessed. The speedup achieved from this is important, but as said it also makes critical systems work properly.

+

The tradeoff of speed and utility is memory usage. Since the idmapper never drops those references it means that objects will never be garbage collected. The result was that the memory usage of Evennia could rise rapidly with an increasing number of objects. Whereas some objects (like those with temporary attributes) should indeed not be garbage collected, in a working game there is likely to be objects without such volatile data. An example might be objects that are not used some of the time - simply because players or the game don't need them for the moment. For such objects it may be okay to re-load them on demand rather than keep them in memory indefinitely.

+

When looking into this I found that simply force-flushing the idmapper did not clean up all objects from memory. The reason for this has to do with how Evennia references objects via a range of other means. The reference count never went to zero and so the garbage collector never got around to it.

+

With the excellent objgraph library it is actually pretty easy to track just what is referencing what, and to figure out what to remove. Using this I went through a rather prolonged spree of cleanups where I gradually (and carefully) cleaned up Evennia's object referencing to a point where the only external reference to most objects were the idmapper cache reference. So removing that (like when deliberately flushing the cache) will now make the object possible to garbage-collect.

+

This is how the reference map used to look for one type of Evennia object (ObjectDB) before the cleanup. Note the several references into the ObjectDB and the cyclic references for all handlers (the cyclic reference is in itself not a problem for reference-counting but they are slow and unnecessary; I now made all handlers use lazy-loading with weak referencing instead).

+

This is how the reference map looks for the same object now. The instance cache is the idmapper reference. There are also no more cyclic references for handlers (the display don't even pick up on them for this depth of recursion). Just removing that single link will now garbage-collect ObjectDB and its typeclass (ignore the g reference, that is just the variable holding the object in ipython).

+

We also see that the dbobj.typeclass <-> typeclass.dbobj references keep each other alive and when one goes the other one goes too - just as expected.

+

An curious aspect of Python memory handling is that (C-)Python does not actually release the memory back to operating system when flushing the idmapper cache. Rather Python makes it internally available so that it does not need to request any more. The result is that if you look at Evennia with the top command, its memory requirement (for example while continuously creating new objects) will not actually drop on a idmapper flush, it will just stop rising.  This is discussed at length in this blog, it was good to learn it was not something I did at least.

+

Apart from the memory stuff, there is work ongoing with fixing the latest batch of user issue reports. Another dev is working on cleaning up the web-related code, it should make it a lot cleaner to overload web functionality with custom code. One of those days I'll also try to sit down and finally convert our web client from long-polling to use web sockets now that Evennia suppports web sockets natively. Time, time ...

+ +
+

+ +

+
+
+ +
+
+

+ Imaginary Realities volume 6, issue 1 + +
- May 16, 2014
+

+

+

I'm a bit late with writing about it, but the latest issue of Imaginary Realities has been out for a month or so now. You can find it here.

+

Here is a brief summary of the articles in the latest issue.

+
    +
  • +

    In A Journey Through Paradice, Part II, Matthew Chaplan continues his description of the C++ codebase Paradice9, notably focusing on its input handling, which uses character-mode telnet to produce plenty of interesting effects in a custom terminal client. There are plenty of interesting features (or potential features) discussed. An example is the client knowing to store the receiver of a "reply" command the moment the command is entered rather than waiting for the player the press return (at which point someone else might have written to you and the reply-to target would have changed in a traditional setup). There is no denying the power of having a custom client for your game. And whereas I think some more secure protocol than telnet would maybe be better to use when you control both server and client anyway, it's really interesting to see the power and features you can achieve with it.

    +
    • +
  • +

    Building a Giant Mech in Evennia - this is my entry for this issue; a short little tutorial on designing a machine of mirth and mayhem in Evennia.

    +
    • +
  • +

    Richard “KaVir” Woolcock's Describing a Virtual World covers the different uses of dynamically created descriptions in text games. He summarizes the types, from the most common, fully static room description up unto the character-dependent procedurally generated descriptions in his own GodWarsII mud. It's a fascinating read since it not only goes into rooms but also how to build piecemeal and situation-aware character and object descriptions as well as procedural help and quest info. The techniques are not for every type of game of course. But a good and informative read for anyone interested in game design.

    +
    • +
  • +

    Dynamic room descriptions, by Jana, are also covering room descriptions although takes a more conservative conclusion than the full procedural contruction of KaVir's article. This covers the hybrid hand-written description - that is hand-written text that uses a markup language for optional or situation-dependent text. It makes for a good text on implementing such a system (an Evennia equivalent is extended_room.py found in our contrib folder.

    +
    • +
  • +

    Saddle Up - A Personal Story about Riding Your Demon to Success is a summary and inspirational story by Michael "Drakkos" Heron. It ties back to his work with Epitath and how it has affected and improved (and continues to improve) his personal and professional life. I like that he manages to include his game development work into his teaching and reasearch, a cool way to make use of your hobby. He has a point on the usability of a coding hobby like this: I myself have had lots of use and even landed project work based on my work with Evennia. One of our users landed his current job based on knowledge he learned working with Evennia. So there is definitely an advantage to mud-development outside the hobby realm.

    +
    • +
  • +

    The Successful Quest Builder by John "TheDude" Robinette and Joanna "Lorana" Liberty covers the construction of a Quest from the designer's perspective. Rather than focusing on gameplay considerations the authors here focus on the technical aspects; learning the codebase's tools and things to think about debugging and developing something that is enjoyable for the players.

    +
    • +
  • +

    The article Your MUD Should Have an Account System finally, is Matthew “Chaos” Sheahan's argument as to why a game should use a single login account system rather than the old way of creating a new account per player. Much of the argument is around converting an old-school code base into this configuration and how it's not as hard as one may think. I fully agree on his assessment (although I wonder just how "easy" it is to patch on such a system on an old-running mud). He even mentions Evennia as an example of a modern codebase having this functionality out of the box (yay!).

    +
    • +
+

Deadline for the next issue is announced to be May 31 2014 so don't be shy to contribute your own article. Richard Tew hints at in his introduction, finding people to write articles is the tricky part still.

+ +
+

+ +

+
+
+ +
+
+

+ Moving from Google Code to Github + +
- February 8, 2014
+

+

+

A few weeks back, the Evennia project made the leap from Google Code to GitHub (here). Things have been calming down so it's time to give a summary of how the process went.

+

Firstly I want to say that I liked Google Code. It did everything expected of it with little hassle. It had a very good Issue system (better than GitHub in my opinion) and it allowed us to use Mercurial instead of Git for version control (I just happen to like Mercurial better than Git, so sue me). Now, GitHub is getting to be something of a standard these days. But whereas our users have occationaly inquired about us making the move, I've been reluctant to do so.

+

The problem I did have with Google Code was that I got the increasing feeling that Google didn't care all that much about it. It worked decently, but it was not really going anywhere either. What finally made me change my mind though was an event just after summer last year. There was a bug in Google Code that made the links to online clones disappear. It was worse than that - creating new online clones of the main repo didn't work - people wanting to contribute using a clone just couldn't.

+

This is extremely critical functionality for a code-sharing website to have! I made a bug report and many other projects chimed in seeing the same issues. Eventually the links returned and everything worked the way it had. But it took several months before this critical bug was fixed. Even then Google didn't even bother to close my issue. This suggested quite strongly to me that Google Code is not really a priority even for its parent company. It was time to consider a move.

+

I was never personally a fan of Git. It is undoubtedly powerful, but I always felt its syntax way too archaic and the number of ways to shoot yourself in the foot way too many. But I do like GitHub better than BitBucket (I've used both in other projects), so that's where we nevertheless were heading.

+

Already last year I created an Evennia "organization" on GitHub and one of our  users first helped to set up a Git Mirror of our Mercurial repo. The idea was a good one - have a mirror on GitHub, allowing the transition to be more gradual. In the end this didn't work out though - there were some issue with the hg-git conversion and the mirror never didn't actually update. When I checked back and it was three months behind we just removed that first ill-fated version.

+

In the end I decided to not fiddle about with it, but to move everything over in one go.

+

Converting the repository

+

I set aside a new folder on my hard drive and cloned the original mercurial repo into a new sub folder. A good idea is to set up a quick Python virtual environment for easily getting updated dependencies of build scripts.

+

I initialized an empty Git repository and used a program called hg-fast-export to convert. As it turned out there were some finer details to consider when doing that:

+
    +
  • +

    The most obvious one was that the conversion initially failed, complaining about the Mercurial original containing "unnamed branches". These came from a contributor who did something to spawn off all sorts of weird branches with little purpose. I should not have merged those into main in the first place, but in those days I didn't know mercurial well enough to be  concerned. In the end I simply used mercurial's MQ extension to remove the unnamed (and unused) branches so the conversion could complete.

    +
    • +
  • +

    The second issue was that Mercurial is less stringent about its author strings than Git is. Git's author string is "name ". Over the years we have gotten contributions from people with all sorts of combinations of names, with or without an email address. So for this we had to supply a mapping file to the converter. It's basically a list of old_author_string = new_author_string and allows for grouping the various used names as needed (some of them were the same person using slightly different author strings).

    +
    • +
+

Once this was in place, the repo conversion worked fine. It was just a matter of changing the .hgignore file to a .gitignore file and change some code that made use of mercurial to get and display the current revision id.

+

Converting the Wiki, part one

+

Evennia's wiki consitutes our documentation, it's some 80+ pages or so by now. Definitely not something we want to loose. Google Code use a dialect of MediaWiki whereas GitHub's wiki supports a few other formats, like markdown or reST. I needed to convert between them.

+

Digging around a bit I found googlecode2github. This download contains python scripts for converting the wiki as well as Issues. I didn't really get the issues-converter to work, so I had to find another solution for that (see next section).

+

All in all, the initial wiki conversion worked decently - all the pages were converted over and were readable. I was even to the point of declaring success when finding the damn thing messed up the links. Googe Code writes links like this: [MyLink Text to see on page]. The script converted this to [[MyLink|Text to see on page]]. Which may look fine except it isn't. GitHub actually wants the syntax in the inverse order: [[Text to see on page|MyLink]].

+

Furthermore, in Google Code's wiki, code blocks were marked with

+

+{{{   
+
+ <verbatim code>   
+
+}}}
+
+
+

In markdown, code blocks are created just by indenting the block by four spaces. The converter dutifully did this - but it didn't add empty lines above and below the block, which is another thing markdown requires. The result was that all code ended up mixed into the running text output.

+

I could have gone back and fixed the converter script, but I suspected there would be enough small things to fix anyway. So in the end I went through 80+ pages of fixing link syntax and adding empty lines by hand. After that I could finally push the first converted wiki version up to the GitHub wiki repository.

+

Some time later I also found that there is a way to let GitHub wiki pages use syntax highlighting for the language of your choice. The way to do this is to enclose your code blocks like this:

+

+```python  
+
+  
+
+<verbatim code>  
+
+  
+
+``` 
+
+
+

This is apparently "GitHub-flavoured" markdown. So another stint into all the pages followed, to update everything for prettiness.

+

Converting Google Code Issues

+

I didn't want to loose our Issues from Google Code. I looked around a bit and tested some conversions for this (it helps to be able to create and delete repos on GitHub with abandon when things fail). I eventually settled on google-code-issues-migrator.

+

This is a Python script that gathers all the Issues from a given Google Code project. It then uses GitHub's API to re-post the issues. It retains the issue numbers and re-maps the Google Code Issue tags to GitHub's equivalent. It didn't retain most other formatting and whereas I ended up as the creator of all issues, the converter included the name of the original author as well as a link back to the original Google Code one. I found that to be quite sufficient for our needs.

+

Converting the IRC announcer

+

A lot of development discussion goes on in our IRC channel #evennia on Freenode. There is an announcer bot in there that I've written, that collates information from various sources and reports it in the IRC channel:

+
    +
  • +

    Repository updates

    +
    • +
  • +

    Wiki updates

    +
    • +
  • +

    Issue creation and updates

    +
    • +
  • +

    Mailing list/forum posts

    +
    • +
  • +

    Dev-blog updates (this blog)

    +
    • +
+

Say what you will about Google, but they are great at offering RSS feeds to all their stuff. So my IRC bot was basically a glorified threaded RSS reader that echoed changes to the channel as they came in. This had been working nicely for years.

+

GitHub does offer RSS feeds to -some- of their offerings, but it's a lot more patchy. I eventually had to do quite a bit of hacking to get everything reporting the way we were used to.

+
    +
  • +

    GitHub has its own IRC announcer bot that reports to IRC. The problem is that this will connect, send message and then disconnect. This causes a lot of spam in the channel. We neither can nor want to set +n on our channel to allow external messages either. The way I solved this was to expand my own custom IRC bot to sit in two irc channels. The GitHub announcer connects to only one of them (so this gets all the spammy connect messages). My IRC bot picks up the announcement and echoes it cleanly to our main #evennia channel.  It works really well.

    +
    • +
  • +

    Issues are handled by the GitHub announcer in the same way.

    +
    • +
  • +

    GitHub has no automatic way to report wiki updates. It doesn't even have a proper RSS feed. However, a user clued me in on using the pipes website to relay an RSS feed from github. I then configured my IRC bot to check that RSS and report it (I also changed the IRC colours to match the GitHub-announcer ones).

    +
    • +
  • +

    Mailing list and blog haven't changed, so those are still handled via RSS as before.

    +
    • +
+

All this done, the modified IRC announcement works well.

+

Closing the book on Google Code

+

At this point all the critical things were moved over. So after some heads-up warnings on the mailing list (and users helping to rewrite our documentation to use Git instead of mercurial) we eventually made the official move.

+

One thing I really dislike is when a project switches hosts and don't let users know about it in their revision history. So I made a mercurial-only last commit announcing that the repo is closed and giving the link to the new one.

+

The Google Code page doesn't go anywhere, but I changed the front page to point to GitHub instead. I even made an issue in the Issue tracker with a title telling people not to use that tracker anymore. Finally I re-pointed all the links on http://www.evennia.com to GitHub and made a mailing list posting. Move was officially complete.

+

Converting the Wiki, part 2

+

At this point were were officially moved over and I started to look into getting fancy with our documentation. We have for the longest time made automated translations of our wiki for compiling by ReadTheDocs.

+

Getting Google Code's special wikimedia syntax into reST (that ReadTheDocs uses) used to mean jumping through a few hoops. My hackish solution worked in two steps. First a custom python script (whose originating url I can no longer find, sorry) converted the Google Code wiki to HTML. Once this was done, pandoc converted the files from HTML to reST. The result was ... acceptable. There were some minor issues here and there but mostly the result was readable.

+

I figured that converting from the more standard Markdown of the GitHub wiki to reST should be a breeze by comparison. Not so.

+

The first hurdle was that the version of pandoc coming with my Linux distribution was too old to support Github-flavoured markdown syntax. I knew from before that Pandoc worked so I didn't want to start with something else. I had to download the several hundred MBs needed by the Haskell build environment and their package manager in order to get and compile all the dependencies and finally the latest version of pandoc. To their credit it was all a very streamlined experience, it just took quite some time.

+

The second hurdle came when finally looping pandoc to convert all wiki files. It turns out to be that the [[Text on page|address]] syntax I had manually corrected earlier is a special syntax offered by Gollum, the engine powering GitHub's wiki behind the scenes. None of the markdown-to-reSt converters I looked at (pandoc or otherwise) even recognized this syntax as a link at all. As it turns out, normal markdown actually expects its links in the format Text on page.

+

I was not going to go through and edit all those pages again. So my next step was to write a script to scan and replace all the [[...|...]] syntax in our wiki and replace it with the standard markdown one. After this the markdown files converted to reST quite nicely -- formatting-wise they look much better than the old wiki to HTML to reST chain I had to use from Google Code.

+

Problem was that when compiling these reST pages into HTML with Sphinx, no links worked.

+

Each individual page looked okay, just that the links were not pointing to anything reasonable. In retrospect this was not so strange. Pandoc knows nothing about the relationships between files, and clearly the simple naming scheme used for addresses is something the wiki softwares knows and Sphinx does not.

+

Some thinking lead to a custom Python script for renaming the link targets in the converted pages to their html page name. This needed to handle the fact that wiki links also allows whitespace. So the [Start](Getting Started) link would be converted to Start, which seems to be the format with which Sphinx will generate its pages.

+

One also needs to have a "toc" (Table of Contents) to tie all those pages together for the benefit of Sphinx. I just used a "hidden" toc, letting my converter script add this to the bottom of my normal index file. As long as it's included somewhere, Sphinx will be happy.

+

Originally I put the reST files in a subfolder of the GitHub wiki repo, I thought I could just point ReadTheDocs to that repo later. The GitHub wiki has a strange "feature" though. It seems to pick its wiki pages from wherever they are in the repo, no matter if they are in the root or in subfolders. Suddenly I was starting to see reST-style pages appear in the online wiki, and sometimes I would get the markdown version (the two would go out of sync). Very strange and confusing.

+

Since the files clearly "polluted" our wiki, I had to move the converted reST files to a separate branch of the wiki repository. This has the advantage of keeping all the support scripts and converter mechanisms separate from the normal wiki content. ReadTheDocs can luckily be set to read its information from another branch than master, so finally the latest converted wiki can again be read there!

+

That concludes what I think was the last main conversion effort. Phew!

+

Impressions so far

+

GitHub is nice. The merge requests and easy way to comment on them are really good. Since people are more familiar with using GitHub overall, it does seem to be a shorter step for people to make a fork and contribute small things. Doing the same in Google Code was probably not harder per se, just something less people were previously familiar with.

+

Due to the modular way Evennia is structured, people are recommended to make a fresh clone of the new Git repo and simply copy their plugin source files and database back into it. So far this seems to have gone smoothly.

+

The GitHub issue tracker is worse than the Google Code one. It has no good way to order Issues or list them in a more compact form (nor in a matrix). Not having good issue templates is really limiting; having to reply to issues only to ask for basic info they should include in their issue is an unnecessary time sink.

+

I also find that there is no clear way to announce an issue change (like "Needing more information"). Tags work partly for this, but setting them is not announced anywhere as far as I can tell - they are just there.

+

Most things also takes so much spaaace. Overall GitHub seems designed for people with big monitors. I have those, but most of the time I prefer working on my laptop. I'm sure it's a matter of habit, but Google Code is very compact by comparison. It gave a lot better overview of things. On GitHub I have to scroll everywhere and this is true both in the repo view, wiki and issues.

+

These small quips nonwithstanding, I think this move will serve us well. There is a good wibe of development and continuing improvement going on at GitHub. There's plenty of help and tutorials all over. Since so many people are using GitHub, problems are more likely to have been answered before. And of course we hope this will in effect help more people find Evennia and join the fun.

+ +
+

+ +

+
+
+ +
+
+

+ Looking forwards and backwards + +
- January 24, 2014
+

+

+

We are almost a month into the new year, time to look forward.

+

But first a look backwards. The year of 2013 was a year of big development projects and lots of quiet in the main repository in between. Two major updates were released during the year.

+

The first update, the "many sessions per player" update, originated in a feature request that I thought would be easy to implement but which led to far-ranging changes (and honestly, improvements) to how Players and Sessions interconnect. It was a lot more more work than I anticipated.

+

The second update was about moving Evennia's web server from the Portal level into the Server-level. The actual moving of the server was actually considerably easier than I thought it would be. But it turned out that a truckload of other things came along with it. Not only did the cache system have to change in order to accommodate the new webs erver, I had to also finalize the Out-of-band structure, since this made use of the cache system. And while I were at it, other fixes were done and ... the update grew and grew. When it finally merged late last year it closed plenty of issues, but it would probably have been better to structure it into more, small updates instead.

+

Anyway, 2014 promises continued (and hopefully more continuous and gradual) development of Evennia. The closest upcoming upheaval is our move from Google Code to GitHub in a few days (I'll probably do a blog about that once it's done). Apart from that we are currently in a fixing state, cleaning up and fixing remnant issues from the big mergers.

+

Another Issue of the MUD e-zine Imaginary Realities is coming too. I just contributed with an Evennia-related article. If anyone reading this blog has anything MUD-related to write about, do consider contributing before January 31, they need more articles! I don't think you need to be too advanced, anything from a mud-development anecdote to retells of good MUD gaming memories might be interesting I would think.

+ +
+

+ +

+
+
+ +
+ + \ No newline at end of file diff --git a/devblog/html/devblog2015.html b/devblog/html/devblog2015.html index e7fe9c1cfd..f016f8e645 100644 --- a/devblog/html/devblog2015.html +++ b/devblog/html/devblog2015.html @@ -35,30 +35,6 @@
-
  • 2020 (3) +
  • 2018 (4)
    • -
  • 2015 (1) +
  • 2017 (6) + + +
    • + +
  • 2016 (7) + + +
    • + +
  • 2015 (13) +
    • + +
  • 2014 (7) + + +
    • + +
  • 2013 (5) + + +
    • + +
  • 2012 (13) + +
    • @@ -333,7 +505,464 @@ def funcname(a, b, c, d=False): """

    - + +

    +
    +
    + +
    +
    +

    + MIT uses Evennia! + +
    - November 12, 2015
    +

    +

    +

    Evennia was recently used as a test bed to train an AI system to attempt to play a MUD as a human would - by only reading and understanding the text on the screen.

    +

    Researchers at MIT (Massachusetts Institute of Technology) recently presented the paper Language understanding for Text-based games using Deep reinforcement learning (PDF) at a conference on natural language processing. A summary is in the MIT press release.

    +

    I was contacted by these fine folks some time ago so I knew they had plans to use Evennia for their research. It's great to see they now have an article out on it! Evennia devs are also mentioned in the acknowledgements - so something for the Evennia dev community to be proud of!

    +

    MUDs are tricky

    +

    The main complication for an AI playing a MUD is that the computer has no access to the actual game state but must try to surmise how well it's doing only from the text given (same as a human would). The researchers compare the results from a range of deep-learning neural network algorithm that they train to play.

    +

    To test their AI, the researchers first used Evennia to build a simple training "Home World": a 4-room "house" where the simple goal is to find and eat an apple while refraining to go to sleep. The room descriptions used here were pretty formulaic although not trivial to give a challenge. This they used to train their AI system.

    +

    They then took this trained neural network and applied it to the real challenge, playing the Evennia Tutorial World. You can yourself try this out in our demo install or by just running a single command when starting Evennia. They call it "Fantasy World" in the article.

    +

    The tutorial world has hand-written descriptions and often describes the exits as part of the room text. The article actually makes a comprehensive analysis of the tutorial world, including the available game states and transitions as well as the number of words and number of commands per state. Interesting stuff in itself. I presume the scientists have modified their copy of the tutorial world to provide better metrics for their analysis.

    +

    A bridge too far

    +

    As far as I understand from the article, the AI does understand to use commands with one or two arguments (like eat apple or the move red-root right), but they note that actually finding the tomb of the fallen hero (the main quest of the tutorial) is too hard for the AI:

    +
    +

    [...]However, this is a complex quest that requires the player to memorize game events and perform high-level planning which are beyond the scope of this current work.

    +
    +

    So instead they evaluate the AI's performance on a more mundane task: Getting across the bridge to the castle. It's not clear to me if the AI actually plays more of the game too or if their test just exposes the AI to the bridge itself. I suspect it does play more due to the examples they use from other rooms; evaluating the bridge-crossing is just a clear-cut metric to use for "success".

    +

    The MIT press release claims that the AI is also scored on how much health/magic it has, but I don't see that mentioned in the article itself (and the tutorial world only has magic if you find the hero's tomb which they claim they cannot do).

    +

    The bridge in Evennia's tutorial world is actually a single "room" that takes multiple steps to cross. At every step the room description changes to describe the progress. Random texts will appear as the bridge sways in the wind and various environmental cues are heard and seen. There is also a small chance of falling off the bridge if one lingers too long on it.

    +

    So although all you really need to do is to walk east repeatedly, I can see why this can be a challenge to a neural network having no mental image of what a bridge is. It can only work off the text it's given at any given time.

    +

    In the paper, the algorithms are evaluated both on their ability to actually cross the bridge and on how optimal their solution was, for example by not issuing invalid commands to the situation.

    +

    Beyond the bridge

    +

    The results are that after being trained on the training house setup, the AI will eventually be able to cross the bridge. The particular algorithm proposed also perform slightly better than the comparison ones (and a lot better than simple randomness).

    +

    So from the perspective of the researchers this seems to be a success. Even so, this reinforces the fact that quite some way to go before an AI can actually play a real MUD successfully. Using MUDs for this type of research is a good idea though, and I do hope they expand and continue this line work in the future.

    +

    Who knows, maybe the AI will even find that ancient tomb eventually!

    + +
    +

    +

    Image: MIT news

    + +

    +
    +
    + +
    +
    +

    + Illustrations and soaps + +
    - October 11, 2015
    +

    +

    +

    I recently made an article presenting the infrastructure and main coding features of Evennia using a series of nifty diagrams.

    +

    It can hopefully help newcomers get a feeling for the grand structure of an Evennia server/game a little easier. It's published as a feature on Optional Realities and you can find the article here: Evennia Illustrated.

    +

    I also recently joined MU Soapbox, a forum predominantly discussing MUSH games, to answer some technical questions on Evennia. Unsurprisingly, some (although so far surprisingly few) MUSHers express concern about Evennia's explicit lack of softcode (that is, the ability for players to use a safe in-game language to code their game rather than to use external Python modules). Their argument is sound:  They are used to in-game coding as a way to easily express creatitivy and easily garner help from players.

    +

    Evennia's stance here clash a bit with those opinions: Our philosophy is that our command system is powerful enough to offer players any complexity of build commands they want. The design/coding of the game itself should be done using proper coding IDEs and modern version control tools.

    +

    There is no denying that compared to a softcode-coded game, a player-level contributor to an Evennia game needs some extra tools to create and contribute code over version control. The admin also needs to check such contributions for malicious code before merging it into their running game. Now, these are differences I actually consider advantages from a code-quality perspective. And for finding help, people on average are more likely to know Python than a custom softcode language. But here opinions differ and in a given game community those language adoption numbers can be severely skewed.

    +

    So far, the MUSHers that have adopted Evennia seems to have done so very much to get away from softcode. It will be interesting to see if things like Kelketek's in-development Group building Evennia contrib will be stirring some interest from those on the fence, or if coding their entire game in softcode is indeed an irreplaceable source of their gaming fun.

    + +
    +

    + +

    +
    +
    + +
    +
    +

    + Emoting System + +
    - October 2, 2015
    +

    +

    title: Emoting Systems, or how to chat up a girl

    +

    copyrights: Image: ©Griatch deviantart

    +

    +

    A few days ago I pushed an emoting contribution to Evennia. A "contrib" is an optional plugin system that is not part of core Evennia but is meant to be easily picked up and used in people's own designs.

    +

    If you are not familiar with what an emoting system does, it is a way to decribe the actions of a character in the game. The simplest form of emote is a single command (like the command dance leading to some canned response, or in the case of a graphical game, a dance animation). This contribution offers a more sophisticated system though, allowing input like the following:

    +

    emote /me smiles at /cheerful as he sits at her table. "Hello!" he says.

    +

    Now, this includes /keywords that relate to the objects in the room. So assuming there is a very cheerful girl in the room, this string will come out as

    +

    Griatch smiles at a very cheerful girl as he sits at her table. "Hello!" he says.

    +

    But she will actually see only my outward appearance (the short description) since she doesn't know me. So the cheerful girl (let's say her name is Sandra) would for example see

    +

    A man in flowing robes smiles at Sandra as he sits at her table. "Hello!" he says.

    +

    The emoting system has the following features:

    +
      +
    • +

      Short description replacement in emotes and in searches, as seen above. This means that you can do look cute and the system will know what you want to look at (in vanilla Evennia you'd need to use look Sandra).

      +
      • +
    • +

      Multi-word searching and disambiguation. If there is a cute girl and a cute puppy both in the same room, your referencing of /cute will  give an error listing the alternatives. You can then either include more words to make your reference unique or use an index (1-cute, 2-cute) to make it clear who you mean. This mimics normal object-key disambiguation in Evennia.

      +
      • +
    • +

      Recognition. You can assign your own aliases to people. If Sandra introduces herself you could assign her the name Sandra and henceforth be able to reference her as such and see that name appear. But you could also name her The girl calling herself Sandra if you didn't believe that's her actual name.

      +
      • +
    • +

      Languages. Everything within double-quotes is parsed as spoken language (like the Hello! above). By using writing this as (elvish)"Hello!", this could be spoken in another language and those who don't speak elvish would receive an obfuscated string.

      +
      • +
    • +

      Masking. A person wearing a mask can force people's recognition replacement to deactivate so that they are not recognized anymore.

      +
      • +
    +

    The emoting contrib comes as two files in evennia/contrib/: rpsystem.py and rplanguage.py. To use them fully, make your Characters and Rooms inherit from the supplied classes and/or add the new commands to the Character command set. Enjoy!

    + +
    +

    + +

    +
    +
    + +
    +
    +

    + Evennia on `podcast.__init__` + +
    - September 29, 2015
    +

    +

    +

    So a few weeks back I was contacted by the general Python podcast Podcast.init. They had stumbled on Evennia and were intrigued about MUD development still going on - and in Python even!

    +

    As it turned out at least one of the two hosts were an old-school MU* gamer and between the two of them they had plenty of good questions both on multiplayer text games in general and about Evennia technology in particular.

    +

    You can listen to my accent via one of the links below:

    + + +
    +

    +

    Logo ©Podcast.__init__

    + +

    +
    +
    + +
    +
    +

    + Pushing through a straw + +
    - September 24, 2015
    +

    +

    +

    Recently, a user reported a noticeable delay between sending a command in-game to multiple users and the result of that command appearing to everyone. You didn't notice this when testing alone but I could confirm there was almost a second delay sometimes between entering the command and some users seeing the result. A second is very long for stuff like this. Processing time for a single command is usually in the milliseconds. What was going on?

    +

    Some background

    +

    Evennia has two components, the Portal and the Server, running as two separate processes_._ The basic principle is that players connecting to an Evennia instance connects to the Portal side - this is the outward facing part of Evennia. The connection data and any other input and output will be piped from the Portal to the Server and back again.

    +

    The main reason for this setup is that it allows us to completely reset the Server component (reloading module data in memory is otherwise error-prone or at least very tricky to make generically work in Python) without anyone getting disconnected from the Portal. On the whole it works very well.

    +

    Debugging

    +

    Tracing of timings throughout the entire Server<->Portal pipeline quickly led me to rule out the command handler or any of the core systems being responsible - including both sides of the process divide was a few milliseconds. But in the transfer between Portal and Server, an additional 900 miliseconds were suddenly added! This was clearly the cause for the delay.

    +

    Turns out that it all came down to faulty optimization. At some point I built a batch-send algorithm between the Server and Portal. The idea was to group command data together if they arrived too fast - bunch them together and send them as a single batch. In theory this would be more efficient once the rate of command sending increased. It was partly a DoS protection, partly a way to optimize transfer handling.

    +

    The (faulty) idea was to drop incoming data into a queue and if the rate was too high, wait to empty that queue until a certain "command/per second" rate was fullfilled. There was also a timed routine that every second force-emptied the queue to make sure it would be cleaned up also if noone made any further commands.

    +

    In retrospect it sounds silly but the "rate of command" was based on a simple two-time-points derivative;

    +
    +

    rate = 1 / (now - last_command_time)

    +
    +

    If this rate exceeded a set value, the batch queuing mechanism would kick in. The issue with this (which is easy to see now) is that if you set your limit at (say) 100 commands / second, two commands can happen to enter so close to each other time that their rate far exceed that limit just based on the time passed between them. But there are not actually 100 commands coming in every second which is really what the mechanism was meant to react to.

    +

    So basically using a moment-to-moment rate like this is just too noisy to be useful; the value will jump all over the place. The slowdown seen was basically the DoS protection kicking in because when you are relaying data to other users, each of them will receive "commands" in quick succession - fast enough to trigger the limiter. These would be routed to the queue and the sometimes-delay simply depended on  when the queue cleanup mechanism happened to kick in.

    +

    Resolution

    +

    Once having identfied the rate measuring bug, the obvious solution to this would be to gather command rates over a longer time period and take an average - you will then filter out the moment-to-moment noise and get an actually useful rate.

    +

    Instead I ended up going with an even simpler solution: Every command that comes in ups a counter. If I want a command rate limit of 100 commands/second, I wait until that counter reaches 100. At that point I check when the time difference between now and when the counter was last reset. If this value is below 1, well then our command rate is higher than 100/second and I can kick in whatever queuing or limiter is needed. The drawback is that until you have 100 commands you won't know the rate. In practice though, once the rate is high enough to be of interest, this simple solution leads to an automatic check with minimal overhead.

    +

    In the end I actually removed the batch-sending component completely and instead added command DoS protection higher up on the Portal side. The Command-input is now rate limited using the same count-until-limit mechanism. Seems to work fine. People have no artificial slowdowns anymore and the DoS limiter will only kick in at loads that are actually relevant. And so all was again well in Evennia world.

    + +
    +

    + +

    +
    +
    + +
    +
    +

    + A wagon load of post summer updates + +
    - August 27, 2015
    +

    +

    +

    Summer vacations are over and work resumes in Evennia land! Here's a wagon-load of small updates on what's going on.

    +

    Ainneve

    +

    The Ainneve project, the creation of an official, open-source Evennia demo game, has gotten going. The lead devs of the project are keen to make this a collaborative effort and there is a lot of good discussion and code being written. After some slowdown at the end of summer it's bound to pick up again.

    +

    Ainneve's a rare chance to see a full MUD getting developed from scratch out in the open. The current issue list includes tags for difficulty and allows also newbie Python coders to jump in. Not to mention you have a chance to get valuable feedback on your work by seasoned coders!

    +

    So if you are at all interested in making a MUD, try out Python/Evennia or just get involved in a semi-big programming project, this is a great chance to do so.

    +

    Imaginary Realities

    +

    Since a few weeks, there is a new issue of Imaginary realities (vol 7, issue 3) is out. As usual I have an article in it. This venerable e-zine was revitalized to include articles on both MU* as well as roguelikes, Interactive fiction and others. Not only is this issue the most content-rich since the reboot, with this issue they have also spruced up their interface to make past issues easier to navigate.

    +
      +
    • +

      "A text MUD with a working ecology system" - in this article Molly O'Hara  details the concepts behind simulating a functioning ecologic network in a game. Interesting stuff and some parts of this is certainly worth considering for any open-world game design. I wonder at which level of detail the system become more complex than the players can appreciate though.

      +
      • +
    • +

      "Dispelling the gloom" by Tomasz Gruca is an interesting essay on the author's history growing up in the former Soviet Union and eventually finding text adventure games and interactive fiction, a passion he has apparently lately re-kindled. He makes the observation that the current "retro" trend of games have not really reached back to the text-based game world when it comes to mainstream acceptance.

      +
      • +
    • +

      "How integral are letters and text to ASCII gaming?"by Mark R. Johnson goes into the practical use of ASCII in traditional rogue-like games (beyond nostalgia). This is a meaty article that goes into both text-as-graphics as well as the use of text for aiding imagination and suggest subtle puzzles in some classic rogue-likes.

      +
      • +
    • +

      "Legend and the lore" (Hugo Zombiestalker) proclaims the death of the traditional point-and-click adventure game and but then moves on to try to distill just why those games nevertheless was so appealing to him and how it can be applied in modern game designs like zombie-survival MUD Epitath which he is a senior developer for. Plenty of good observations here!

      +
      • +
    • +

      "The bonds of mudding" by Clint Itan Kuranes Knapp writes about the community that can spring up on a long-running MUD, the interactions the friends and the relationships that could persist already long before "social media" became a buzz word. A warm text with plenty of anecdotes and examples and things to ponder for both designers and staff when wanting to cater for this type of player bonding.

      +
      • +
    • +

      "The mercurial temperament at the end of the world" (Drakkos) discusses NPCs and how they rarely are as interactive as one would want (the term "vend a chat" is a good one I think). He then goes on to how they have implemented their "Mercurial" system for NPCs in Epitath. This seems to be a state-AI system where NPCs have moods that affects what they say based on their circumstance and relation to other actors in the world. Sounds really cool and since he goes into some details on the implementation there is a lot to ponder here.

      +
      • +
    • +

      "Where do I begin?" by me, Griatch, discusses one of the more common questions we get in the Evennia chat - 'I want to make a MUD, but how do I begin?' This article starts before Evennia's Game planning wiki page - it discusses assessing your capabilities and resources in the form of programming skills, code bases and motivations to help you figure out what you can realistically accomplish.

      +
      • +
    +

    Evennia Web client

    +

    In the pipeline I have some updates to Evennia's websocket/JSON MUD-web client component. These are changes that are intended to make the webclient easier to customize and hook into Evennia output using only HTML/CSS. More details on this will be forthcoming when I have more solid stuff to show.

    + +
    +

    + +

    +
    +
    + +
    +
    +

    + Announcing the Evennia example-game project "Ainneve" + +
    - June 22, 2015
    +

    +

    +

    The Evennia example-game project is underway!

    +

    I was quite impressed with the response I got on the mailing list to my call for developing an Evennia example game (see my Need your Help blog post).

    +

    The nature of the responses varied, many were from interested people with little to no experience in Evennia or Python whereas others had the experience but not the time to lead it. It was however clear that the interest to work on an "official" Evennia game is quite big.

    +

    I'm happy to announce, however, that after only a week we now have a solid lead developer/manager, George Oliver. Helping him on the technical/architecture side is Whitenoise (who, despite a barren github profile, is a professional developer).

    +

    George put together a game proposal based on the OpenAdventure rpg, an open-source (CC-SA) ruleset that is also found on github. The example game is to be named "Ainneve" and its development is found in a in a separate repository under the github Evennia organisation.

    +

    All the relevant links and future discussion can be found on the mailing list.

    +

    George and whitenoise have already made it clear that they aim to not only make Ainneve a good example Evennia game for others to learn from and build on, but to make the development itself a possibility for people of all skill levels to get involved. So get in touch with them if you are at all interested in Python, Evennia and mud development!

    +

    So thanks to George and whitenoise for taking this on, looking forward to see where it leads!

    + + +
    + +
    +
    +

    + Need your help! + +
    - June 15, 2015
    +

    +

    This for all you developers out there who want to make a game with Evennia but are not sure about what game to make or where to start off.

    +

    We need an example game

    +

    One of the main critiques Evennia get from newbies is the lack of an (optional) full game implementation to use as an example and base to build from. So, Evennia needs a full, BSD-licensed example game. I'm talking "diku-like", something you could in principle hook up and allow players into within minutes of installing Evennia. The Tutorial world we already have is a start but it is more of a solo quest, it's not designed to be a full multiplayer game. Whereas Evennia supports other forms of MU* too, the idea is that the systems from a more "code-heavy" MUD could easily be extracted and adopted to a more freeform-style game whereas the reverse is not generally true.

    +

    The exact structure of such a game would be up to the person or team taking this on, but it should be making use of Evennia's api and come distributed as a custom game folder (the folder you get with evennia --init). We will set this up as a separate repository under the Evennia github organisation - a spin-off from the main evennia project, and maintained separately.

    +

    We need you!

    +

    Thing is, while I am (and, I'm sure other Evennia core devs) certainly willing to give considerable help and input on such a project, it's not something I have time to take the lead on myself. So I'm looking for enthusiastic coders who would be willing to step up to both help and take the lead on this; both designing and (especially) coding such an example game. Even if you have your own game in mind for the future, you still need to build most of these systems, so starting with a generic system will still help you towards that final goal - plus you get to be immortalized in the code credits, of course.

    +

    +

    Suggestion for game

    +

    Being an example game, it should be well-documented and following good code practices (this is something we can always fix and adjust as we go though). The systems should be designed as stand-alone/modular as possible to make them easy to rip out and re-purpose (you know people will do so anyway). These are the general features I would imagine are needed (they are open to discussion):

    +
      +
    • +

      Generic fantasy theme (lore is not the focus here, but it can still be made interesting)

      +
      • +
    • +

      Character creation module

      +
      • +
    • +

      Races (say, 2-3)

      +
      • +
    • +

      Classes (say 2-3)

      +
      • +
    • +

      Attributes and Skills (based on D&D? Limit number of skills to the minimal set)

      +
      • +
    • +

      Rule module for making skill checks, rolls etc (D&D rules?)

      +
      • +
    • +

      Combat system (twitch? Turn-based?)

      +
      • +
    • +

      Mobs, both friendly and aggressive, with AI

      +
      • +
    • +

      Trade with NPC / other players (money system)

      +
      • +
    • +

      Quest system

      +
      • +
    • +

      Eventual new GM/admin tools as needed

      +
      • +
    • +

      Small game world (batch-built) to demonstrate all features (of good quality to show off)

      +
      • +
    • +

      More? Less?

      +
      • +
    +

    I'm interested!

    +

    Great! We are as a first step looking for a driven lead dev for this project, a person who has the enthusiasm, coding experience and drive to see the project through and manage it. You will (hopefully) get plenty of collaborators willing to help out but It is my experience that a successful hobby project really needs at least one person taking responsibility to "lead the charge" and having the final say on features: Collaborative development can otherwise easily mean that everyone does their own thing or cannot agree on a common course. This would be a spin-off from the main Evennia project and maintained separately as mentioned above.

    +

    Reply to this thread if you are willing to participate at any level to the project, including chipping in with code from your already ongoing development. I don't know if there'd be any "competition" over the lead-dev position but if multiple really enthusiastic and willing devs step forward we'll handle that then.

    +

    So get in touch!

    + +
    +

    + +

    +
    +
    + +
    +
    +

    + Dreaming big? + +
    - May 30, 2015
    +

    +

    +

    Optional Reality's Dreaming Big Challenge has come to an end. This was a challenge about designing a "sales pitch" in 600 words or less for a text-based online roleplaying game. This is the kind of pitch you'd tease knowledgeable investors with -  if investors cared about multiplayer text games and you ended up in an elevator with one, that is.

    +

    There were 10 entries to the competition and as the results are now in (including very elaborate judge feedback!), I encourage you to go read all the entries. The focus was on originality and fresh ideas and maybe these short pitches represent a cross-section of the current trends or a whiff of where the wind is blowing for future games. Or maybe it will help inspire you to make a sales pitch of your own.

    +

    You can find all the entries linked from the second post of this thread. Enjoy!

    + +
    +

    + +

    +
    +
    + +
    +
    +

    + Things goin on + +
    - May 11, 2015
    +

    +

    +

    We are currently in a maintenance and cleanup phase of Evennia, where bugs are found and reported and things are getting more and more stable as people learn and use the new features we merged a few months back.

    +

    Overall though I must say the relatively big changes we did to the infrastructure (making Evennia into a full library and making a complete overhaul of the typeclass system behind the scenes) went over with surprising smoothness. There were a flurry of things to fix immediately after the devel-branch merger but no more than expected. For the big changes it really worked very well I think, with no big disaster stories. We have a few bugs lingering in the issue tracker that need to be addressed but nothing show-stopping.

    +

    I have been a bit busy with various projects off-MUD so to speak. I was contracted for making the cover and illustration for a book (this is not hobby art for once, but a professional commission which I was quite excited to be asked to do). I also author and draw a fantasy comic as part of another project.

    +

    I've not been slacking off on on the MUD side though: I have written and submitted an article for the revived Imaginary Realities e-zine (next issue should be out end of May/early June?) and another article (on Evennia) for the new Optional Realities MUD community website. I also contributed a game-design blurb for the latter's Dreaming Big contest, where you compete (for cash prizes, actually!) by submitting a game sale's pitch under 600 words.

    +

    The above mentioned Optional Realities website is so far picking up pace with quite good discussion in its forums (the similarity of name with Imaginary Realities is unfortunate, apparently they were not aware of the former when choosing it).  While targeted at discussions of RPI-style games (a sort of sub-genre of roleplay-enforced MUDs with perma-death), it already hosts several invited articles on game design and general game development that can be interesting for any MU* dev.

    +

    People should know by now that I like to support MUD community efforts when possible, and Evennia is thus  listed as an official "affiliate" to Optional Realities (which admittedly means little more than us linking to each other but still). The team behind OR is however also using Evennia for their own "Project Redshift" Sci-fi mud, so we hope to get plenty of good feedback as their project matures.

    + +
    +

    +

    Image: Truss from Gems NFX

    + +

    +
    +
    + +
    +
    +

    + Documenting Python without Sphinx + +
    - May 9, 2015
    +

    +

    +

    Last week Evennia merged its development branch with all the features mentioned in the last post. Post-merger we have since gone through and fixed remaining bugs and shortened the list at a good clip.

    +

    One thing I have been considering is how to make Evennia's API auto-documenting - we are after all a MUD creation library and whereas our code has always been well-documented the docs were always only accessible from the source files themselves.

    +

    Now, when you hear "Python" and "documentation" in the same sentence, the first thought usually involves Sphinx or Sphinx autodoc in some form. Sphinx produces very nice looking documentation indeed. My problem is however as follows:

    +
      +
    • +

      I don't want our API documentation to be written in a different format from the rest of our documentation, which is in Github's wiki using Markdown.  Our users should be able to help document Evennia without remembering which formatting language is to be used.

      +
      • +
    • +

      I don't like reStructuredText syntax. This is a personal thing. I get that it is powerful but it is also really, really ugly to read in its raw form in the source code. I feel the sources must be easy to read on their own.

      +
      • +
    • +

      Sphinx plugins like napoleon understands this ugliness and allows you to document your functions and classes in a saner form, such as the "Google style". One still needs reST for in-place formatting though.

      +
      • +
    • +

      Organizing sphinx document trees is fiddly and having had a few runs with sphinx autodoc it's just a mess trying to get it to section the Evennia sources in a way that makes sense. It could probably be done if I worked a lot more with it, but it's a generic page generator and I feel that I will eventually have to get down to make those toctrees myself before I'm happy.

      +
      • +
    • +

      I want to host the API docs as normal Wiki files on Github (this might be possible with reST too I suppose).

      +
      • +
    +

    Long story short, rather than messing with getting Sphinx to do what I want, I ended up writing my own api-to-github-Markdown parser for the Evennia sources: api2md. Using Python's inspect module and aiming for a subset of the Google formatted docstrings, this was maybe a day's work in total - the rest was/is fine-tuning for prettiness.

    +

    Now whenever the source is updated, I follow the following procedure to fully update the API docs:

    +
      +
    1. +

      I pull the latest version of Evennia's wiki git repository from github alongside the latest version of the main Evennia repository.

      +
      2. +
    2. +

      I run api2md on the changed Evennia sources. This crawls the main repo for top-level package imports (which is a small list currently hard-coded in the crawler - this is to know which modules should create "submodule" pages rather than try to list class contents etc). Under each package I specify it then recursively gets all modules. For each module in that package, it creates a new Markdown formatted wiki page which it drops in a folder in the wiki repository. The files are named after the model's path in the library, meaning you get files like evennia.objects.models.md and can easily cross-link to subsections (aka classes and functions) on a page using page anchors.

      +
      3. +
    3. +

      I add eventual new files and commit the changes, then push the result to the Github wiki online. Done!

      +
      4. +
    +

    (I could probably automate this with a git hook. Maybe as a future project.)

    +

    The api2md program currently has some Evennia-custom elements in it (notably in which packages it imports) but it's otherwise a very generic parser of Python code into Markdown. It could maybe be broken out into its own package at some point if there's interest.

    +

    The interesting thing is that since I already have code for converting our wiki to reST and ReadTheDocs, I should be able to get the best of both worlds and convert our API wiki pages the same way later. The result will probably not be quite as custom-stunning as a Sphinx generated autodoc (markdown is a lot simpler in what formatting options it can offer) but that is a lesser concern.

    +

    So far very few of Evennia's docstrings are actually updated for the Google style syntax (or any type of formatting, really) so the result is often not too useful. We hope that many people will help us with the doc strings in the future - it's a great and easy way to get to know Evennia while helping out.

    +

    But where the sources are updated, the auto-generated wiki page looks pretty neat.

    + + +
    + +
    +
    +

    + Building Django proxies and MUD libraries + +
    - January 19, 2015
    +

    +

    +

    2015 is here and there is a lot of activity going on in Evennia's repository, mailing list and IRC channel right now, with plenty of people asking questions and starting to use the system to build online games.

    +

    We get newcomers of all kinds, from experienced coders wanting to migrate from other code bases to newbies who are well versed in mudding but who aim to use Evennia for learning Python. At the moment the types of games planned or under development seems rather evenly distributed between RPI-style MUDs and MUSH games (maybe with a little dominance of MUSH) but there are also a couple of hack-and-slash concepts thrown into the mix. We also get some really wild concepts pitched to us now and then. What final games actually comes of it, who can tell, but people are certainly getting their MU*-creative urges scratched in greater numbers, which is a good sign.

    +

    Since Christmas our "devel" branch is visible online and is teeming with activity. So I thought I'd post an summary about it in this blog. The more detailed technical details for active developers can be found on Evennia's mailing list here (note that full docs are not yet written for devel-branch).

    +

    Django proxies for Typeclasses

    +

    I have written about Evennia's Typeclass system before on this blog. It is basically a way to "decorate" Django database models with a second set of classes to allow Evennia developers to create any type of game entity without having to modify the database schema. It does so by connecting one django model instance to one typeclass instance and overloading setattr and getattribute to transparently communicate between the two.

    +

    For the devel branch I have refactored our typeclass system to make use of Django's proxy models instead. Proxy models have existed for quite a while in Django, but they simply slipped under my radar until a user pointed them out to me late last year. A proxy model is basically a way to "replace the Python representation of a database table with a proxy class". Sounds like a Typeclass, doesn't it?

    +

    Now, proxy models doesn't work quite like typeclasses out of the box - for one thing if you query for them in the database you will get back the original model and not the proxy one. They also do not allow multiple inheritance. Finally I don't want Evennia users to have to set up django Meta info every time they use a proxy. So most work went into overloading the proxy multiclass inheritance check (there is a django issue about how to fix this). Along the way I also redefined the default managers and init methods to always load the proxy actually searched for and not the model. I finally created metaclasses to handle all the boilerplate. We choose to keep the name Typeclass also for this extended proxy. This is partly for legacy reasons, but typeclasses do have their own identity: they are not vanilla Django-proxies nor completely normal Python classes (although they are very close to the latter from the perspective of the end user).

    +

    Since typeclasses now are directly inheriting from the base class (due to meta-classing this looks like normal Python inheritance), it makes things a lot easier to visualize, explain and use. Performance-wise this system is en par with the old, or maybe a little faster, but it will also be a lot more straight forward to cache than the old. I have done preliminary testing with threading and it looks promising (but more on that in a future post).

    +

    Evennia as a Python library package

    +

    Evennia has until now been solely distributed as a version controlled source tree (first under SVN, then Mercurial and now via GIT and Github). In its current inception you clone the tree and find inside it a game/ directory where you create your game. A problem we have when helping newbies is that we can't easily put pre-filled templates in there - if people used them there might be merge conflicts when we update the templates upstream. So the way people configure Evennia is to make copies of template modules and then change the settings to point to that copy rather than the default module. This works well but it means a higher threshold of setup for new users and a lot of describing text. Also, while learning GIT is a useful skill, it's another hurdle to get past for those who just want to change something minor to see if Evennia is for them.

    +

    In the devel branch, Evennia is now a library. The game/ folder is no longer distributed as part of the repository but is created dynamically by using the new binary evennia launcher program, which is also responsible for creating (or migrating) the database as well as operating the server:

    +
    
+evennia --init mygame  
+
+cd mygame  
+
+evennia migrate  
+
+evennia start
+
+
    +

    Since this new folder is not under our source tree, we can set up and copy pre-made template modules to it that people can just immediately start filling in without worrying about merge conflicts. We can also dynamically create a setting file that fits the environment as well as set up a correct tree for overloading web functionality and so on. It also makes it a lot easier for people wanting to create multiple games and to put their work under separate version control.

    +

    Rather than traversing the repository structure as before you henceforth will just do import evennia in your code to have access to the entirety of the API. And finally this means it will (eventually) be possible to install Evennia from pypi with something like pip install evennia. This will greatly ease the first steps for those not keen on learning GIT.

    +

    For existing users

    +

    Both the typeclasses-as-proxies and the evennia library changes are now live in the devel branch. Some brave users have already started taking it through its paces (and is helping to flesh it out) but it will take some time before it merges into master.

    +

    The interesting thing is that despite all this sounding like a huge change to Evennia, the coding API doesn't change very much, the database schema almost not at all. With the exception of some properties specific to the old connection between the typeclass and model, code translate over pretty much without change from the developer's standpoint.

    +

    The main translation work for existing developers lies in copying over their code from the old game/ directory to the new dynamically created game folder. They need to do a search-and-replace so that they import from evennia rather than from src or ev. There may possibly be some other minor things. But so far testers have not found it too cumbersome or time consuming to do. And all agree that the new structure is worth it.

    +

    So, onward into 2015!

    + +
    +

    +

    Image: "Bibliothek St. Florian" by Original uploader was Stephan Brunker at de.wikipedia Later versions were uploaded by Luestling at de.wikipedia. - Originally from de.wikipedia; description page is/was here.. Licensed under CC BY-SA 3.0 via Wikimedia Commons - link

    +

    diff --git a/devblog/html/devblog2016.html b/devblog/html/devblog2016.html index fb4165edac..f816c7fffd 100644 --- a/devblog/html/devblog2016.html +++ b/devblog/html/devblog2016.html @@ -35,30 +35,6 @@
    -
  • 2020 (3) +
  • 2018 (4)
    • -
  • 2015 (1) +
  • 2017 (6) + + +
    • + +
  • 2016 (7) + + +
    • + +
  • 2015 (13) +
    • + +
  • 2014 (7) + + +
    • + +
  • 2013 (5) + + +
    • + +
  • 2012 (13) + +
    • @@ -260,7 +432,7 @@

    - +

    @@ -311,7 +483,7 @@

    - +

    @@ -361,8 +533,8 @@
    @@ -427,7 +599,7 @@
    @@ -457,7 +629,7 @@ @@ -504,7 +676,7 @@ @@ -523,7 +695,7 @@ diff --git a/devblog/html/devblog2017.html b/devblog/html/devblog2017.html index abb0936264..fdeb56e6f2 100644 --- a/devblog/html/devblog2017.html +++ b/devblog/html/devblog2017.html @@ -35,30 +35,6 @@
    -
  • 2020 (3) +
  • 2018 (4)
    • -
  • 2015 (1) +
  • 2017 (6) + + +
    • + +
  • 2016 (7) + + +
    • + +
  • 2015 (13) +
    • + +
  • 2014 (7) + + +
    • + +
  • 2013 (5) + + +
    • + +
  • 2012 (13) + +
    • @@ -320,7 +492,7 @@
    @@ -339,7 +511,7 @@ @@ -443,7 +615,7 @@ @@ -606,7 +778,7 @@ @@ -669,8 +841,8 @@ @@ -703,7 +875,7 @@ diff --git a/devblog/html/devblog2018.html b/devblog/html/devblog2018.html index 1404b2252c..d613445b85 100644 --- a/devblog/html/devblog2018.html +++ b/devblog/html/devblog2018.html @@ -35,30 +35,6 @@
    -
  • 2020 (3) +
  • 2018 (4) -
    • @@ -320,7 +492,7 @@ @@ -381,7 +553,7 @@ @@ -405,7 +577,7 @@ diff --git a/devblog/html/devblog2019.html b/devblog/html/devblog2019.html index 838772a70b..2e6cfe9361 100644 --- a/devblog/html/devblog2019.html +++ b/devblog/html/devblog2019.html @@ -35,30 +35,6 @@
    -
  • 2020 (3) +
  • 2018 (4)
    • -
  • 2015 (1) +
  • 2017 (6) + + +
    • + +
  • 2016 (7) + + +
    • + +
  • 2015 (13) +
    • + +
  • 2014 (7) + + +
    • + +
  • 2013 (5) + + +
    • + +
  • 2012 (13) + +
    • @@ -270,8 +442,8 @@
    @@ -298,7 +470,7 @@ @@ -514,7 +686,7 @@ class State(BaseState): @@ -635,7 +807,7 @@ class State(BaseState): @@ -654,7 +826,7 @@ class State(BaseState): @@ -686,8 +858,8 @@ class State(BaseState): @@ -715,7 +887,7 @@ class State(BaseState): diff --git a/devblog/html/devblog2020.html b/devblog/html/devblog2020.html index 19761fd282..19a0ebd8be 100644 --- a/devblog/html/devblog2020.html +++ b/devblog/html/devblog2020.html @@ -35,30 +35,6 @@
    @@ -417,7 +589,7 @@ @@ -478,8 +650,8 @@ diff --git a/devblog/html/devblog2021.html b/devblog/html/devblog2021.html index 8e849e54f5..73cd507537 100644 --- a/devblog/html/devblog2021.html +++ b/devblog/html/devblog2021.html @@ -35,30 +35,6 @@
    -
  • 2020 (3) +
  • 2018 (4)
    • -
  • 2015 (1) +
  • 2017 (6) + + +
    • + +
  • 2016 (7) + + +
    • + +
  • 2015 (13) +
    • + +
  • 2014 (7) + + +
    • + +
  • 2013 (5) + + +
    • + +
  • 2012 (13) + +
    • @@ -396,8 +568,8 @@
    @@ -448,7 +620,7 @@ diff --git a/devblog/source/2012-02-05-About-this-dev-blog.md b/devblog/source/2012-02-05-About-this-dev-blog.md new file mode 100644 index 0000000000..c4a99c3d91 --- /dev/null +++ b/devblog/source/2012-02-05-About-this-dev-blog.md @@ -0,0 +1,8 @@ +This is to be my [Evennia](http://www.evennia.com/) dev blog, but it will also cover various other musings concerning programming in general and mud designing in particular. Whereas the Evennia [mailing list](https://groups.google.com/forum/#%21forum/evennia) remains the main venue for discussion, I will probably use this blog for announcing features too. + +Some background: +Evennia is a Python MUD/MUX/MU* server. More correct is probably to call it a "MUD-building system". The developer codes their entire game using normal Python modules. All development is done with the full power of Python - gritty stuff like database operations and network communication are hidden away behind custom classes that you can treat and modify mostly like any Python primitives. + +Since the server is based on the [Twisted](http://twistedmatrix.com/trac/) and [Django](https://www.djangoproject.com/) technologies we can offer many modern features out of the box. Evennia is for example its own web server and comes with both its own website and an "comet"-style browser mud client. But before this turns into even more of a sales pitch, I'll just just direct you to the evennia website if you want to know more. :) + +I, Griatch, took over the development of Evennia from the original author, Greg Taylor, in 2010. \ No newline at end of file diff --git a/devblog/source/2012-02-05-Evennias-open-bottlenecks.md b/devblog/source/2012-02-05-Evennias-open-bottlenecks.md new file mode 100644 index 0000000000..5b6d9d31d7 --- /dev/null +++ b/devblog/source/2012-02-05-Evennias-open-bottlenecks.md @@ -0,0 +1,19 @@ +title: Evennia's open bottlenecks + +--- + +Since Evennia hit beta I have been mostly looking behind the scenes to see what can be cleaned up and not in the core server. One way to do that is to check where the bottlenecks are. Since a few commits, Evennia's runner has additions for launching the Python cProfiler at startup. This can be done for both the Portal and the Server separately. + +I have created a test runner (soon to hit trunk) that launches dummy clients. They log on and then goes about their way executing commands, creating objects and so on. The result of looking at the profiling data (using e.g. _runsnake)_ has been very interesting. + + + +Firstly, a comparably small part of execution time is actually spent in Evennia modules - most is spent using Python built-ins and in the Twisted/Django libraries. This is promising in a way - at least there are no expensive loops in the Evennia code itself that sucks up cycles. Of course it also means we depend heavily on django/twisted (no surprise there) and especially when it comes to database access, I know there could be more caching going on so as to cut down on db calls.  + + + +Of the Evennia modules, a lot of time is spent in getting properties off objects - this is hard to avoid, it a side effect of Evennia's typeclass system - and this is cached very aggressively already. What also stands out as taking a considerable time is the command system - the merging of command sets in particular does a lot of comparing operations. This happens every time a command is called, so there are things to be done there. The same goes for the help system; it needs to collect all the currently active command sets for the calling player. Maybe this could be cached somehow. + + + +More work on this is needed, but as usual optimization is a double-edged sword. Evennia is written to have a very readable code. Optimization is usually the opposite of readable, so one needs to thread carefully. \ No newline at end of file diff --git a/devblog/source/2012-02-15-Such-a-small-thing.md b/devblog/source/2012-02-15-Such-a-small-thing.md new file mode 100644 index 0000000000..9ac2273610 --- /dev/null +++ b/devblog/source/2012-02-15-Such-a-small-thing.md @@ -0,0 +1,65 @@ +title: Such a small thing ... + +--- + + +Lately I went back to clean up and optimize the workings of Evennia's Attributes. I had a nice idea for making the code easier to read and also faster by caching more aggressively. The end result was of course that I managed to break things. In the end it took me two weeks to get my new scheme to a state where it did what it already did before (although faster). + +Doing so, some of the trickier aspects of implementing easily accessible Attributes came back into view, and I thought I'd cover them here. Python intricacies and black magic to follow. You have been warned.  + +Attributes are, in Evennia lingo, arbitrary things a user may want to permanently store on an object, script or player. It could be numbers or strings like health, mana or short descriptions, but also more advanced stuff like lists, dictionaries or custom Python objects. + +Now, Evennia allows this syntax for defining an attribute on e.g. the object _myobj_: + +> myobj.db.test = [1,2,3,4] + +This very Pythonic-looking thing allows a user to transparently save that list (or whatever) to an attribute named, in this example, _test._ This will save to the database. + What happens is that _db_, which is a special object defined on all Evennia objects, takes all attributes on itself and saves them by overloading its __setattr__ default method (you can actually skip writing _db_ most of the time, and just use this like you would any Python attribute, but that's another story). + +Vice-versa, + +> value = myobj.db.test + +This makes use of the _db_ object's custom __get_attribute__ method behind the scenes. The _test_ attribute is transparently retrieved from the database (or cache) for you. + +Now, the (that is, my) headache comes when you try to do this: + +> myobj.db.test[3] = 5 + +Such a small, normal thing to do! Looks simple, right? It is actually trickier than it looks to allow for this basic functionality. +The problem is that Python do everything by reference. The list is a separate object and has no idea it is connected to _db._ _db_'s __get_attribute__ is called, and happily hands over the list _test_. And then _db_ is out of the picture!. My nifty save-to-database feature (which sits in _db_) knows nothing about how the 3rd index of the list _test_ now has a 5 instead of a 4. + +Now, of course, you could always do this: + +> temp = myobj.db.test + +> temp[3] = 5 + +> myobj.db.test = temp + +This will work fine. It is however also clumsy and hardly intuitive. The only solution I have been able to come up with is to have _db_ return something which is _almost_ a list but not quite. It's in fact returning an object I not-so-imaginatively named a _PackedList__._ This object works just like a list, except all modifying methods on it makes sure to save the result to the database. So for example, what is called when you do mylist[3] = 4 is a method on the list named __setitem__. I overload this, lets it do its usual thing, then call the save. + +> myobj.db.test[3] = 5 + +now works fine, since _test_ is in fact a _PackedList_ and knows that changes to it should be saved to the database. I do the same for dictionaries and for nested combinations of lists and dictionaries. So all is nice and dandy, right?  Things work just like Python now? + +No, unfortunately not. Consider this: + +> myobj.db.test = [1, 3, 4, [5, 6, 7]] + +A list with a list inside it. This is perfectly legal, and you can access all parts of this just fine: + +> val = myobj.db.test[3][2] # returns 7! + +But how about _assigning_ data to that internal nested list? + +> myobj.db.test[3][2] = 8 + +We would now expect _test_ to be [1, 3, 4, [5, 6, 8]]. It is not. It is infact only [5, 6, 8]. The inner list has replaced the entire attribute!  + +What actually happens here? _db_ returns a nested structure of two _PackedList_s. All nice and dandy.  But Python thinks they are two separate objects! The main list holds a reference to the internal list, but as far as I know _there is no way for the nested list to get the back-reference to the list holding it_! As far as the nested list knows, it is all alone in the world, and therefore there is no way to trigger a save in the "parent" list. + The result is that we update the nested list just fine - and that triggers the save operation to neatly overwrite the main list in the cache and the database. +  +This latter problem is not something I've been able to solve. The only way around it seems to use a temporary variable, assign properly, then save it back, as suggested earlier. I'm thinking this is a fundamental limitation in the way cPython is implemented, but maybe I'm missing some clever hack here (so anyone reading who has a better solution)? + +Either way, the _db_ functionality makes for easy coding when saving things to the database, so despite it not working _quite_ like normal Python, I think it's pretty useful. \ No newline at end of file diff --git a/devblog/source/2012-02-17-Commands-and-you.md b/devblog/source/2012-02-17-Commands-and-you.md new file mode 100644 index 0000000000..a23fce3410 --- /dev/null +++ b/devblog/source/2012-02-17-Commands-and-you.md @@ -0,0 +1,24 @@ +_Commands_ define how a Player interacts with a given game. In a text-based game it's not amiss to say that the available commands are paramount to the user experience. In principle commands could represent mouse clicks and other modernistic GUI sugar - but for this blog I'll stick with the traditional entered text. + +Like most things in Evennia, Commands are Python classes. If you read the [documentation](http://code.google.com/p/evennia/wiki/Commands) about them you'll find that the command classes are clumped together and tacked onto all objects in-game. Commands hanging onto a Character object will be available all the time, whereas commands tacked onto a grandfather's clock will only be available to you when you stand in front of said clock. + +The interesting thing with Commands being classes is that each Character gets a separate instance of each command. So when you do _look_ 100 times in a row, it's always the _same_ Look command instance that has its methods called by the engine. Apart from being an efficient way to handle things, this has a major neat side-effect: + +> _You can store things on the Command object and whatever you store can be retrieved next time you execute that command._ + + +I find this very cool mainly because I didn't really plan with this in mind when designing the command system - it was a happy side effect. A use I have thought of is to implement cooldowns. Say you have a powerful _bash_ command. It makes a lot of damage, but you need time to recover between bashes. So when you do the _bash_ command the Bash command object simply stores the current time on itself: + +> self.last_bash = time.time() + +Next time the Player tries to use _bash_, all the command object needs to is to check if self.last_bash is set, and compare the time stored herein with the current time. No twisted tasks needed, no overhead. Very neat and tidy. + +Another nice functionality (just added today in fact) is that Evennia can be set to _store a copy of the last command object executed_. What can one do with this? For starters, it allows for commands to check what a previous command was. This can be useful in itself, but since the next command actually have access to (a copy of) the previous command object itself, it will allow a lot more than that. + +Consider a _look_ command that remembers whatever object it is looking at. Since the Look command is a Python object, it simply stores the looked-at object on itself before returning the normal info to the Player. Next, let's assume we use a _get_ command. If no argument is given to this _get_ (no given object to pick up), the _get_ normally returns an error. But it can now instead peek at the _previous_ command (look) and see what _that_ command operated on. This allows for nice potential constructs like + +> >> look [at] box  + +> >> get [it] + +Evennia does not use this functionality in its default command set, but it offers some very powerful possibilities for MUD creators to design clever parsing schemes. \ No newline at end of file diff --git a/devblog/source/2012-02-22-Dummies-doing-dummy-things.md b/devblog/source/2012-02-22-Dummies-doing-dummy-things.md new file mode 100644 index 0000000000..4b240356b8 --- /dev/null +++ b/devblog/source/2012-02-22-Dummies-doing-dummy-things.md @@ -0,0 +1,47 @@ +[![](https://4.bp.blogspot.com/-fGC9QysIL-0/T0UwdxCwDiI/AAAAAAAABN4/cklfv7W28Iw/s1600/224108172066smileys.jpg)](https://4.bp.blogspot.com/-fGC9QysIL-0/T0UwdxCwDiI/AAAAAAAABN4/cklfv7W28Iw/s1600/224108172066smileys.jpg)It can often be interesting to test hypothetical situations. So the other day I did a little stress test to see how Evennia handles many players. This is a follow-up to the [open bottlenecks](http://evennia.blogspot.com/2012/02/evennias-open-bottlenecks.html) post. + +Evennia, being based on Twisted, is an asynchronous mud server. This means that the program is not multi-threaded but instead it tries to divide up the code it runs into smaller parts. It then quickly switches between executing these parts in turn, giving the impression of doing many things at the same time. There is nothing magical about this - each little piece of code blocks the queue as it runs. So if a particular part takes too long, everyone else will have to wait for their turn. Likewise, if Twisted cannot flip through the queue as fast or faster than new things get added to it, it will start to fall behind. + +The good thing is that all code in the queue _will_ run, although if the event loop cannot keep up there will be a noticeable delay before it does. In a mud, this results in "lagging" (in addition to whatever lag is introduced by your network connection). Running Evennia with a handful of users shows no effect of this, but what happens when we start to add more and more players? + +My "dummy runner" is a stand alone Twisted program that opens any number of Telnet connections to the Evennia server. Each such "dummy" client is meant to represent a player connecting. In order to mimic actual usage, a dummy client can perform a range of actions: + +- They can create a new account and log in +- They can look around and read random help files +- They can create objects, name and describe them (for testing, the server is set to grant build privileges to all new players) +- They can examine objects, rooms and themselves  +- They can dig new rooms, naming all exits and giving aliases +- They can move between rooms + +The clients tick every 5 seconds, at which time there is a 10% chance each will perform an action from the list above (login first, then one of the others at random).  This is meant to spread out the usage a bit, like it would be with real people. Some of these actions actually consist of multiple commands sent to the server (create + describe + set etc), possibly simulating people using shortcuts in their client to send several commands at the same time. + +Results + +Note that I didn't do a proper objective benchmark. Rather, I logged in and ran commands to see, very subjectively, how the game "felt" with a number of different players. The lag times are rough estimates by putting time() printouts in the server. + +First I tried with my development laptop, a Thinkpad X61s. It's about 5 years old by now and certainly not the fastest thing out there, mainly it's small and thin and has great battery life. I used a MySQL database. + +- **1-50** dummy players didn't show any real difference from playing alone. It's not so strange; 50 players meant on average 5 of them would do an action every 5 seconds. My laptop could handle that just fine. +- **50-75** dummy players introduced a brief lag (less than a second) when they logged in. 5-7 players logging in at exactly the same time will after all mean a lot of commands sent to the server (they not only log in, they create a new character at the same time). Throughout these tests, clients logging in showed the greatest effect on lag. I think this is mostly an artifact of how the clients operate by sending all relevant login commands at the same time. Once all were logged in, only brief lag occurred at random times as coincidence had all clients do something expensive at the same time. +- **75-100** dummy players introduced longer lags during login, as on average 7-10 clients were now logging in at exactly the same time. Most commands were unaffected, but occasionally there were some noticeable "hiccups" of about a second depending on what the clients were doing. +- **100-150** dummy players - here things started to go downhill rapidly. Dummy client login caused seriously lagging and at 150 logged in players, there were 15 exactly simultaneous actions coming in. This was clearly too much for my laptop; it lead to a very varying lag of 1-10 seconds also for simple commands. + +Next I tried my desktop machine. This is also not bleeding edge; it's a 4-year old machine with 3GHz processor and 4GB RAM. Since I don't have MySQL on this machine, I used SQLite3, which is interesting in its own right since it's Evennia's default database. + +- **1-75** dummy players didn't affect the feel of the game one bit. +- **75-100** showed some occasional effects starting to appear during dummy client logins.  +- **100-150** dummy players didn't introduce more than a few hiccups of less than a second when approximately 15 clients decided to do something expensive at the same time. +- **150-200** introduces 2-3 seconds lag during client mass-logins, but once all clients had connected, things worked nicely with only brief random hiccups. +- **200-250** showed the lag getting more uneven, varying from no lag at all to 2-3 seconds for normal times and up to 5 seconds when clients were logging in. +- **250-300** dummy players showed login lag getting worse. The general lag varied from 0-5 seconds depending on what the other clients were up to. +- **300-350** dummy players, about double of what the laptop could handle,  the CPU was not able to keep up anymore. The system remained stable and all commands got executed - eventually. But the lag times went up very rapidly. + +Conclusions + +So, based on this I would say 50-75 was a comfortable maximum number of (dummy) players to have on my laptop whereas the desktop could handle around 150 without much effort, maybe up to 250 on a good day. + +So what does these numbers mean? Well, the dummy clients are rather extreme, and 100+ builders walking around building stuff every 5 seconds is not something one will see in a game. Also so many logging on at the same time is not very likely (although it could happen after a crash or similar). If anything the usage pattern of human players will be more random and spread out, which helps the server to catch up on its event queue. + +On the other hand these tests were run with a vanilla Evennia install - a full game might introduce more expensive commands and subsystems. Human players may also introduce random spikes of usage. So take this for what it is - a most subjective, un-scientific and back-of-the-envelope measure. + +All in all though, I would say that few MUDs these days see 30 concurrent online users, even less 100 ... \ No newline at end of file diff --git a/devblog/source/2012-03-26-Shortcuts-to-goodness.md b/devblog/source/2012-03-26-Shortcuts-to-goodness.md new file mode 100644 index 0000000000..8041e9feed --- /dev/null +++ b/devblog/source/2012-03-26-Shortcuts-to-goodness.md @@ -0,0 +1,14 @@ +[![](https://lh4.googleusercontent.com/proxy/QkyxkcgmfR5RuTgXVrmufjqBSJDUNdFKVM50GLVV98Oj7kN0b7IJzWhCK8n1McUG119JfSMFKxCIRW57srOOW0vsPIcnC1D1aHCRJuwvLOydXOMQmkSWg1b9-h92QPchLVw8xYSoWaRB3I8oyHEpPcXEFEO86Vd9hI96fNdasSwU2B4OY5l5zKxmbd2sPIFSH8W6phmRB1lD7NW-sUJk_LQRgjRcl5iPRl9SuaOT3Dgm=s0-d)](http://upload.wikimedia.org/wikipedia/commons/thumb/d/d1/2011_attraction_repulsion_direction_arrows.png/120px-2011_attraction_repulsion_direction_arrows.png)Evennia, being a MUD-design system, needs to take some special considerations with its source code - its sole purpose is after all to be read, understood and extended. +Python is of course very readable by default and we have worked hard to give extensive comments and documentation. But for a new user looking into the code for the first time, it's still a lot of stuff to take in. Evennia consists of a set of Django-style "applications" interacting and in some cases inheriting from each other so as to avoid code duplication. For a new user to get an overview could therefore mean diving into more layers of code than one would like. + +I have now gone through the process of making Evennia's _API_ (Application Programming Interface) "flatter". This has meant exposing some of the most commonly used methods and classes at a higher level and fully documenting exactly what they inherit av every layer one looks at. But I have also added a new module ev.py to the root directory. It  implements "shortcuts" to all the most commonly used parts of the system, forming a very flat API. This means that what used to be + +      from src.objects.objects import Object + +can now be done as  + +      from ev import Object + +Not only should it be easier to find things (and less boilerplate code to write) but I like that one can also easier explore Evennia interactively this way.  Using a Python interpreter (I recommend ipython) you can just import ev and easily inspect all the important object classes, tab to their properties, helper functions and read their extensive doc strings. + +Creating this API, i.e. going through and identifying all the useful entry points a developer will need, was also interesting in that it shows how small the API really is. Most of the ev interface is really various search functions and convenient options to inspect the database in various ways. The MUD-specific parts of the API is really lean, as befits a barebones MUD server/creation system. \ No newline at end of file diff --git a/devblog/source/2012-05-30-Dummies-doing-even-more-dummy-things.md b/devblog/source/2012-05-30-Dummies-doing-even-more-dummy-things.md new file mode 100644 index 0000000000..a56f5d94e4 --- /dev/null +++ b/devblog/source/2012-05-30-Dummies-doing-even-more-dummy-things.md @@ -0,0 +1,29 @@ +title: Dummies doing (even more) dummy things + +--- + +[![](https://lh6.googleusercontent.com/proxy/7O-DqSxlUJDFdfWHuOcVzoSro_bzD9BSAipHXLKNG3gpwyTEptGPTOk5MAgX6yjrkAC2r7P1o9YtCZ1cjzTriLrE9I4kL3frKu4ZcvBpp3Uy8CvM4A=s0-d)](http://www.smiley-faces.org/backgrounds/smiley-background-012.jpg) + +This is a follow-up to the [Dummies doing dummy things](http://evennia.blogspot.se/2012/02/dummies-doing-dummy-things.html) post. I originally posted info about this update on the mailing list some time back, but it has been pointed out to me that it might be a nice thing to put on the dev blog too since it's well, related to development! + +I have been at it with further profiling in Evennia. Notably even more aggressive on-demand caching of objects as well as on-object attributes. I found from profiling that there was an issue with how object access checks were done - they caused the lock handler to hit the database every lock check as it retrieved the needed attributes. + +Whereas this was not much of a hit per call, access checks are done _all the time_, for commands, objects, scripts, well everything that might need restricted access. +After caching also attributes, there is no need to hit the database as often. Some commands, such as listing all command help entries do see this effect (although you still probably wouldn't notice it unless you checked before and after like I did). More importantly, under the hood I'm happy to see that the profile for normal Evennia usage is no longer dominated by Django db calls but by the functional python code in each command - that is, in code that the end user have full control over anyway. I'd say this is a good state of affairs for a mud creation system. + +  +In the previous "Dummies ..." post I ran tests with rather extreme conditions - I had dummy clients logging to basically act like heavy builders.  They dug rooms, created and defined objects randomly every five seconds (as well as walking around, reading help files, examining objects and other spurious things). In that post I found that my puny laptop could handle about **75** to **100** such builders at a time without me seeing a slowdown when playing. My old but more powerful desktop could handle some **200** or so. + +Now, I didn't re-run these build-heavy tests with the new caches in place. I imagine the numbers will improve a bit, but it's just a guess. By all means, if you expect regularly having more than 100 builders on your game continuously creating 250 new rooms/objects per minute, do get back to me ... + +... Instead I ran similar tests with more "normal" client usage. That is, I connected dummy clients that do what most players would do - they walk around, look at stuff, read help files and so on. I connected clients in batches of 100 at a time, letting them create accounts and logging in fully before connecting the next set of 100. + +All in all I added **1000** dummy clients this way before I saw a noticeable lag on my small laptop. I didn't find it necessary to try the desktop at this point. Whereas this of course was with a vanilla Evennia install, I'd say it should be reasonable room for most realistic mud concepts to grow in. + +With the rather extensive caching going on, it is interesting to know what the memory consumption is. + [![](https://4.bp.blogspot.com/-ZNiU4qTi8XE/T8aMHbBck7I/AAAAAAAABRc/vn6EUwkJjJQ/s400/2012-05-01-Evennia_1000_dummies.png)](https://4.bp.blogspot.com/-ZNiU4qTi8XE/T8aMHbBck7I/AAAAAAAABRc/vn6EUwkJjJQ/s1600/2012-05-01-Evennia_1000_dummies.png) +This graph shows memory info I noted down after adding each block of 100 players. The numbers fluctuated up and down a bit between readings (especially what the OS reported as total usage), which is why the lines are not perfectly straight. + +In the end the database holds 1000 players (which also means there are 1000 Character objects), about as many rooms and about twice as many attributes.  The "idmapper cache" is the mapper that makes sure all Django model instances retain their references between accesses (as opposed to normal Django were you can never be sure of this). "Attribute cache" is a cache storing the attribute objects themselves on the Objects, to avoid an extra database lookup. All in all we see that keeping the entire database in memory takes about 450MB. + +Evennia's caching is on-demand (so e.g. a room would not be loaded/cached until someone actually accessed it somehow). One could in principle run a script to clean all cached regularly if one was short on RAM - time will tell if this is something any user needs to worry about on modern hardware. \ No newline at end of file diff --git a/devblog/source/2012-06-11-Coding-from-the-inside.md b/devblog/source/2012-06-11-Coding-from-the-inside.md new file mode 100644 index 0000000000..22643f7b6b --- /dev/null +++ b/devblog/source/2012-06-11-Coding-from-the-inside.md @@ -0,0 +1,33 @@ +[![](https://3.bp.blogspot.com/-ICX3MmHP4t0/T9UobF85chI/AAAAAAAABRs/806SyA-AQzc/s1600/broken+egg.jpg)](https://3.bp.blogspot.com/-ICX3MmHP4t0/T9UobF85chI/AAAAAAAABRs/806SyA-AQzc/s1600/broken+egg.jpg) + +Some time ago, a message on the Evennia [mailing list](https://groups.google.com/forum/?fromgroups#%21topic/evennia/l5w_AcRdLds) asked about "softcode" support in Evennia. Softcode, a defacto standard in the MUX/MUCK/MUSH/MOO world, is conceptually a "safe" in-game scripting language that allows Players to extend the functionality of things without having access to the server source. + +Now, Evennia is meant to be extended by normal Python modules. For coding game systems and advanced stuff, there is really no reason (in my opinion) for a small development team to not use a modern version control system and proper text editors rather than entering things on a command line without formatting. + +But there _is_ a potential fun aspect of having an online scripting language - and that is player content creation. Crafters wanting to add some pizazz to their objects, builders getting an extra venue of creativity with their rooms - that kind of thing. I didn't plan to add softcode support to Evennia, but it "sounded like an interesting problem" and one thing led to another. + +Python is of course an excellent scripting language from the start. Problem is that it's notoriously tricky to make it run safely with _untrusted_ code - like that inserted by careless or even potentially malignant Players. Scanning the Internet on this topic is a pretty intimidating experience - everywhere you hear that it shouldn't be done, and that the various suggested solutions of a "sandbox" are all inherently unsafe. Python's awesome introspection utilities is its own enemy in this particular case. + +For Evennia we are however not looking for a full sandbox. We want a Python-like way for Players to influence a few determined systems. Moreover, we expect short, simple scripts that can do without most of Python's functionality (since our policy is that if it's too complex or large, it belongs in an external Python module). We could supply black-box "safe" functions to hide away complex functionality while still letting people change things we expect them to want to script. This willingness to accept heavy restrictions to the language should work to our advantage, I hope. + +Evennia actually already has a safe "mini-language" in the form its "lock system", and thus it was a natural way for me to start looking. A "lock string" has a very restricted syntax - it's basically function calls optionally separated by boolean operators, like this: + +> lockfunc1(*args) and lockfunc(*args, **kwargs) and not lockfunc2() + +The result of this evaluation will always be a boolean True/False (if the lock is passed or not). Only certain functions are available to use (controlled by the coder). The interesting thing is that this string can be supplied by the Player, but it is not _eval_uated - rather it's _manually_ parsed, from left to right. The function names and arguments are identified (as for the rest, only and/or/not are allowed). The functions are then called explicitly (in Python code, not evaluated as a string) and combined to get a result. This should be perfectly safe as long as your functions are well-defined. + + +For the potential softcode language, I first took this hands-on approach - manually parsing the string into its components. I got a pretty decent demo going, but the possibilities are much larger than in the simple lockstring case. Just parsing would be one thing, but then to also make sure that each part is okay to use too is another matter ... It would probably be doable, but then I got to supplying some sort of flow-control. The code starts to become littered with special cases which is never a good sign. + +So eventually I drifted off from the "lock-like" approach and looked into Python's _ast_ module. This allows you to view Python code as an "abstract syntax tree" (AST).  This solves the parsing issues but once you start dealing with the AST tree you are sort of looking at the problem from the other end - rather than parsing and building the script from scratch it more becomes a matter of removing what is already there (an AST tree can be compiled directly back into Python code after all). It nevertheless seemed like the best way forward. + +Testing a few different recipes from the web, I eventually settled on an [approach](http://code.activestate.com/recipes/496746/) which (with some modifications compared to the original) uses a whitelist (and a blacklist for some other things) to only allow a given set of ast nodes and items in the execution environment. It walks the AST tree before execution and kills dangerous Python structures in one large swath. I expanded on this a fair bit, cutting away a lot of Python functionality for our particular use case. Stuff like attribute acces and assignments, while loops and many other Pythonesque things went out the window. + +Around this highly stunted execution system I then built the Evennia in-game scripting system. This includes in-game commands as well as scriptable objects with suitable slots defining certain functionality the Player might want to change. Each Evennia developer can also supply any set of "safe" blackbox functions to offer more functionality to their Player-coders. + +A drawback is the lack of a functional timeout watchdog in case of a script running too long. I'm using Twisted's deferToThread to make sure the code blocks as little as possible, but alas, whereas I can check timeouts just fine,  the problem lies in reliably killing the errant sub-thread. Internet experts suggest this to be tricky to do safely at the best of times (with threads running arbitrary code at least), and not wanting to kill the Twisted server is not helping things. I pondered using Twisted's subprocess support, but haven't gotten further into that at this point. Fact is that most of the obvious DOS attack vectors (such as the while loop and huge powers etc) are completely disabled, so it should hopefully not be trivial to DOS the system (famous last words). + +I've tentatively dubbed the softcode system "Evlang" to differentiate it from our normal database-related "Scripts". +So is Evlang "safe" to use by untrusted evil Players? Well, suffice to say I'm putting it up with a huge EXPERIMENTAL flag, with plenty of warnings and mentions of "on your own risk". Running Evennia in a chroot jail and with minimum permissions is probably to recommend for the security paranoid. Hopefully Evennia coders will try all sorts of nasty stuff with it in the future and report their finding in our Issue tracker! + +But implementation details aside, I must admit it's cool to be able to add custom code like this - the creative possibilities really do open up. And Python - even a stunted version of it - is really very nice to work with, also from inside the game. \ No newline at end of file diff --git a/devblog/source/2012-06-26-Extending-time-and-details.md b/devblog/source/2012-06-26-Extending-time-and-details.md new file mode 100644 index 0000000000..148a1cd994 --- /dev/null +++ b/devblog/source/2012-06-26-Extending-time-and-details.md @@ -0,0 +1,13 @@ +[![](https://2.bp.blogspot.com/-D1dza7edP54/T-mEQhE_eEI/AAAAAAAABS0/omfBOfknDQc/s1600/TheVillasLivingRoom_Windows%25281%2529.JPG)](https://2.bp.blogspot.com/-D1dza7edP54/T-mEQhE_eEI/AAAAAAAABS0/omfBOfknDQc/s1600/TheVillasLivingRoom_Windows%25281%2529.JPG)For the fun of it I added an "Extended Room" contrib to Evennia the other night. + +("Contribs" are optional code snippets and systems that are not part of the actual codebase. They are intended for you to use or dissect as you like in your game development efforts). + +The ExtendedRoom is a room typeclass meant to showcase some more advanced features than the default one. Its functionality is by all means nothing revolutionary in MUD-world, but it was fun and very simple to do using only Evennia's basic building blocks - the whole thing took me some two hours to code, document and clean up for a contrib push. The "ExtendedRoom" contribution has the following features: + +- **Season-based descriptions**. The new Room typeclass will change its overall description based on the time of year (the contrib supports the four seasons, you can hack this as you please). It's interesting from an implementation point of view since it doesn't require any Script or ticker at all - it just checks on-demand, whenever it is being looked at, only updating if the season has actually changed. There is also a general description used as a fallback in case of a missing seasonal one. +- **Time-of-day-based descriptions**. Within each Season-based description you can embed time-of-day based ones with special tags. The contrib supports four time slots out of the box (morning, afternoon, evening, night). In the description, you just embed time-dependent text within tags, like _Morning sunlight is shining through the windows_. Only time-relevant tags will be shown. This is a simple regular expression substitution, should be easy to expand on if one wants more fine-grained time slots. +- **Details**. I took the inspiration of these from a MOO tutorial I read a long time ago. "Details" are "virtual" look-targets in the room. It allows you to add visual interest without having to add a bunch of actual objects for that purpose. Details are simply stored in a dictionary on the room. Details don't change with Season in this implementation, but they _are_ parsed for time-of-day based tags! +- **Custom commands**. The room is supported by extending two of the custom commands. The Details require a slightly modified version of the _look_ command. There is also a new _@desc_ for setting/listing details and seasonal descriptions. The new _time_ command, finally, simply shows the current game time and season in the room. + + +Installing and testing the snippet is simple - just add the new commands to the default cmdset (they will dynamically replace the same-named default ones), dig a few rooms of the new typeclass and play around! Especially the details do make building interesting rooms a lot more fun (I got hung up playing with them way too long last night). \ No newline at end of file diff --git a/devblog/source/2012-08-16-Taking-command.md b/devblog/source/2012-08-16-Taking-command.md new file mode 100644 index 0000000000..4b654bc8f5 --- /dev/null +++ b/devblog/source/2012-08-16-Taking-command.md @@ -0,0 +1,59 @@ +[![](https://3.bp.blogspot.com/-uODV_t0eTv8/UC0czBg9ctI/AAAAAAAABTk/NvEUkwrrZPY/s1600/200px-Venn0001.svg.png)](https://3.bp.blogspot.com/-uODV_t0eTv8/UC0czBg9ctI/AAAAAAAABTk/NvEUkwrrZPY/s1600/200px-Venn0001.svg.png)_Commands_ are the bread and butter of any game. Commands are the instructions coming in from the player telling the game (or their avatar in the game) to do stuff. This post will outline the reasoning leading up to Evennia's somewhat (I think) non-standard way of handling commands. + +In the case of MUDs and other text games commands usually come in the form of entered text. But clicking on a graphical button or using a joystick is also at some level issuing a command - one way or another the Player instructs the game in a way it understands. In this post I will stick to text commands though. So _open door with red key_ is a potential command. + +Evennia, being a MUD design system, needs to offer a stable and extensive way to handle new and old commands.  More than that, we need to allow developers pretty big freedom with developing their own command syntax if they so please (our default is not for everyone). A small hard-coded command set is not an option. + + +### Identifying the command + + + +First step is _identifying_ the command coming in. When looking at _open door with red key_ it's probably _open_ that is the unique command. The other words are "options" to the command, stuff the _open_ command supposedly knows what to do with. If you _know_ already at this stage exactly how the command syntax looks, you could hard-code the parsing already here. In Evennia's case that's not possible though - we aim to let people define their command syntax as freely as possible. Our identifier actually requires no more than that the uniquely identifying command word (or words) appear _first_ on the input line. It is hard to picture a command syntax where this isn't true ... but if so people may freely plug in their own identifyer routine. + +So the identifyer digs out the _open_ command and sends it its options ... but what kind of code object is _open_? + +###  The way to define the command + + +A common variant I've seen in various Python codebases is to implement commands as _functions_. A function maps intuitively to a command - it can take arguments and it does stuff in return. It is probably more than enough for some types of games. + +Evennia chooses to let the command be defined as a _class_ instead. There are a few reasons. Most predominantly, classes can inherit and require less boiler plate (there are a few more reasons that has to do with storing the results of a command between calls, but that's not as commonly useful). Each Evennia command class has two primary methods: + +- _parse()_ - this is responsible for parsing and splitting up the _options_ part of the command into easy-to use chunks. In the case of _open door with red key,_ it could be as simple as splitting the options into a list of strings. But this may potentially be more complex. A mux-like command, for exampe, takes _/switches_ to control its functionality. They also have a recurring syntax using the '=' character to set properties. These components could maybe be parsed into a list _switches_ and two parameters _lhs_ and _rhs_ holding the left- and right hand side of the equation sign.  +- _func()_ - this takes the chunks of pre-parsed input and actually does stuff with it.  + +One of of the good things with executing class instances is that neither of these methods need to have any arguments or returns. They just store the data on their object (_self.switches_) and the next method can just access them as it pleases. Same is true when the command system instantiates the command. It will set a few useful properties on the command for the programmer to make use of in their code (_self.caller_ always references the one executing the command, for example). This shortcut may sound like a minor thing, but for developers using Evennia to create countless custom commands for their game, it's really very nice to not have to have all the input/output boilerplate to remember.  + + + +... And of course, class objects support inheritance. In Evennia's default command set the _parse()_ function is  only implemented once, all handling all possible permutations of the syntax. Other commands just inherit from it and only needs to implement _func()._ Some advanced build commands just use a parent with an overloaded and slightly expanded _parse()_. + + +###  Commands in States + + +So we have individual commands. Just as important is how we now group and access them. The most common way to do this (also used in an older version of Evennia) is to use a simple _global list_. Whenever a player enters a command, the _identifier_ looks the command up in the list. Every player has access to this list (admin commands check permissions before running). It seems this is what is used in a large amount of code bases and thus obviously works well for many types of games. Where it starts to crack is when it comes to _game states._ + +- A first example is an in-game menu. Selecting a menu item means an instruction from the player - i.e. a command. A menu could have numbered options but it might also have named options that vary from menu node to menu node. Each of these are a command name that must be identified by the parser. Should you make _all_ those possible commands globally available to your players at all times? Or do you hide them somehow until the player actually is in a menu? Or do you bypass the command system entirely and write new code only for handling menus...? +- Second example: Picture this scenario: You are walking down a dark hallway, torch in hand. Suddenly your light goes out and you are thrown into darkness. You cannot see anything now, not even to look in your own backpack. How would you handle this in code? Trivially you can put _if_ statements in your _look_ and _inventory_ commands. They check for the "dark" flag. Fair enough. Next you knock your head and goes 'dizzy'. Suddenly your "navigation" skill is gone and your movement commands may randomly be turned around. Dizziness combined with darkness means your inventory command now returns a strange confused mess. Next you get into a fight ... the number of if statements starts piling up.   +- Last example: In the hypothetical FishingMUD,. you have lots of detailed skills for fishing. But different types of fishing rods makes different types of throws (commands) available. Also, they all work differently if you are on a shore as compared to being on a boat. Again, lots of if statements. It's all possible to do, but the problem is maintenance; your command body keep growing to handle edge cases. Especially in a MUD, where new features tend to be added gradually over the course of years, this gives lots of possibilities for regressions. + +All of these are examples of situation-dependent (or object-dependent) commands. Let's jointly call them _state-dependent commands._ You could picture handling the in-game menu by somehow dynamically changing the global list of commands available. But then the _global_ bit becomes problematic - not all players are in the same menu at the same time. So you'll then have to start to track _who_ has which list of commands available to them. And what happens when a state ends? How do you get back to the previous state - a state which may itself be different from the "default" state (like clearing your dizzy state while still being in darkness)? This means you have to track the previous few states and ... + +A few iterations of such thinking lead to what Evennia now uses: a _non-global_ _command set_ system. A command set (cmdset) is a structure that looks pretty much like a mathematical _set._ It can contain any number of (unique) command objects, and a particular command can occur in any number of command sets. + +- A cmdset stored on an object makes all commands in that cmdset available to the object. So all player characters in the game has a "default cmdset" stored on them with all the common commands like _look, get_ and so on. +- Optionally, an object can make its cmdset available to other objects in the same location instead. This allows for commands only applicable with a given object or location, such as _wind up grandfather clock._ Or the various commands of different types of fishing rods.  +- Cmdsets can be non-destructively combined and merged like mathematical sets, using operations like "Union", "Intersect" and a few other cmdset-special operations. Each cmdset can have priorities and exceptions to the various operations applied to them. Removing a set from the mix will dynamically rebuild the remaining sets into a new mixed set. + +The last point is the most interesting aspect of cmdsets. The ability to merge cmdsets allows you to develop your game states in isolation. You then just merge them in dynamically whenever the game state changes. So to implement the dark example above, you would define two types of "look" (the dark version probably being a child of the normal version). Normally you use your "default cmdset" containing the normal _look_. But once you end up in a dark room the system (or more likely the room) "merges" the _dark_ cmdset with the default one on the player, replacing same-named commands with new ones. The _dark_ cmdset contains the commands that are different (or new) to the dark condition - such as the _look_ command and the changed _inventory_ command.  Becoming dazed just means yet another merger - merging the _dazed_ set on top of the other two. Since all merges are non-destructive, you can later remove either of the sets to rebuild a new "combined" set only involving the remaining ones in any combination.  + + +Similarly, the menu becomes very simple to create in isolation (in Evennia it's actually an optional contrib). All it needs to do is define the required menu-commands in its own cmdset. Whenever someone triggers the menu, that cmdset is loaded onto the player. All relevant commands are then made available. Once the menu is exited, the menu-cmdset is simply removed and the player automatically returns to whichever state he or she was in before. + + +### Final words + + +The combination of _commands-as-classes_ and _command sets_ has proved to very flexible. It's not as easy to conceptualize as is the simple functions in a list, but so far it seems people are not having too much trouble. I also think it makes it pretty easy to both create and, importantly, expand a game with interesting new forms of gameplay _without_ drastically rewriting old systems. \ No newline at end of file diff --git a/devblog/source/2012-08-31-Combining-Twisted-and-Django.md b/devblog/source/2012-08-31-Combining-Twisted-and-Django.md new file mode 100644 index 0000000000..a1c830d34d --- /dev/null +++ b/devblog/source/2012-08-31-Combining-Twisted-and-Django.md @@ -0,0 +1,54 @@ +copyrights: _Image: Franco Nero as the twisted gunslinger Django_ + +--- + +### [Combining Twisted and Django](https://evennia.blogspot.com/2012/08/combining-twisted-and-django.html) + +[![](https://3.bp.blogspot.com/-Spum6fodLn0/TqPvuMp20mI/AAAAAAAAAIc/LteJztyYZXY/s320/DJango.jpg)](https://3.bp.blogspot.com/-Spum6fodLn0/TqPvuMp20mI/AAAAAAAAAIc/LteJztyYZXY/s1600/DJango.jpg) + +Newcomers to Evennia sometimes misunderstand it as being a "Django mud codebase somehow using Twisted". The correct description is rather that Evennia is a "Twisted-based mud server using Django". Allow me to elaborate. + +A mud/mux/moo/mu* is per definition a multi-user online game system. All these users need to co-exist on the server. If one player does something, other players shouldn't have to (noticeably) wait for that something to end before they can do anything. Furthermore it's important for the database schema to be easy to handle and upgrade. Finally, in a modern game, internet presence and web browser access is becoming a must. We combine two frameworks to achieve this. + + +### Two frameworks combined + + +_[Twisted](http://twistedmatrix.com/trac/)_ is a asynchronous Python framework. "Asynchronous" in this context means, very simplified, that Twisted chops up code execution into as small bits as the code lets it. It then flips through these snippets rapidly, executing each in turn. The result is the illusion of everything happening at the same time. The asynchronous operation is the basis for the framework, but it also helps that twisted makes it easy to support (and create) a massive range of different network protocols. + +_[Django](https://www.djangoproject.com/)_ implements a very nice abstract Python API for accessing a variety of SQL-like databases. It makes it very convenient to maintain the database schema (not to mention that _django-South_ gives us easy database migrations). The fact that Django is really a web framework also makes it easy to offer various web features. There is for example an "admin site" that comes with Django. It allows to modify the database graphically (in Evennia's case the admin site is not quite as polished as we would like yet, but it's coming). + +Here are some highlights of our architecture: + + +- _Portal_ - This is a stand-alone Twisted process talking to the outside world. It implements a range of communication protocols, such as telnet (traditional in MUD-world), ssh, ssl, a comet webclient and others. It is an auto-connecting client to _Server_ (below). +- _Server_ - This is the main MUD server. This twisted server handles everything related to the MUD world. It accesses and updates the database through Django models. It makes the world tick. Since all Players connect to the Server through the Portal's [AMP](http://twistedmatrix.com/documents/8.2.0/api/twisted.protocols.amp.html) connection, it means Server can be restarted without any players getting kicked off the game (they will re-sync from Portal as soon as Server is back up again). +- _Webserver -_ Evennia optionally starts its own Twisted webserver. This serves the game's website (using the same database as the game for showing game statistics, for example). The website is of course a full Django project, with all the possibilities that entails. The Django _admin site_ allows for modifying the database via a graphical interface.  +- _Webclient -_ There is a stand-alone MUD web client existing in a page on the default website. This uses Twisted to implement a long-polling ("comet") connection to a javascript client. As far as Evennia's concerned, this is just another outgoing protocol served by the Portal. +- _Other protocols_ - Since it's easy to add new connectivity, Evennia also offers a bunch of other connectivity options, such as relaying in-game channels to _IRC_ and _IMC2_ as well as _RSS_ feeds and some other goodies.  + + + +### On the joining of the two + + +An important thing to note about Twisted's asynchronous model is that there is no magic at work here: Each little snippet of code Twisted loops over _is_ blocking. It's just hopefully not blocking long enough for you to notice. So if you were to put _sleep(10)_ in one of those snippets, then congratulations, you just froze the entire server for ten seconds. + +Profiling becomes very important here. Evennia's main launcher takes command arguments to run either of its processes under Python's cProfile module. It also offers the ability to connect any number of dummy Players doing all sorts of automated random actions on the server. Such profile data is invaluable to know what is a bottleneck and what is not. + +I never found Twisted asynchronous paradigms much harder to understand than other code. But there are sure ways to write stupid blocking code that will come back and bite you. For example, much of  Evennia's workload is spent in the _Server,_ most notably in its _command handler_. This is not so strange; the command handler takes care of parsing and executing all input coming from Players, often modifying the game world in various ways (see my previous post for more info about the command handler). +The command handler used to be a monolithic, single method. This meant that Twisted had to let it run its full course before letting anyone else do their turn. Using Twisted's _[inlineCallbacks](http://twistedmatrix.com/documents/8.1.0/api/twisted.internet.defer.html#inlineCallbacks)_ instead allowed for yielding at many, many places in this method, giving Twisted ample possibilities to split execution. The effect on multi-user performance was quite impressive. Far from all code can  be rewritten like this though. + +Another important bottleneck on asynchronous operations is database operations. Django, as opposed to Twisted, is _not_ an asynchronous framework. Accessing the database _is_ a blocking operation and can be potentially expensive. It was never extremely bad in testing, to be honest. But for large database operations (e.g. many Players) database access was a noticeable effect. + +I have read of some people using Twisted's _deferToThread_ to do database writes. The idea sounds reasonable - just offload the operation to another thread and go on your merry way. It did not help us at all though - rather it made things slower. I don't know if this is some sort of overhead (or error) in my test implementation - or an effect of Python just not being ideal with using threading for concurrency (due to the GIL). Either way, certain databases like SQlite3 doesn't support multiple threads very well anyway, and we prefer to keep giving plenty of options with that. So no _deferToThread_ for database writes. I also did a little testing with parallel processes but found that even slower, at least once the number of writes started to pile up (we will offer easy process-pool offloading for other reasons though). + +As many have found out before us, caching is king here. There is not so much to do about writes, but at least in our case the database is more often read than written to. Caching data and accessing the cache instead of accessing a field is doing much for performance, sometimes a _lot._ Database access is always going to cost, but it does not dominate the profile. We are now at a point where one of the most expensive single operations a Player (even a Builder) performs during an entire gaming session is the hashing of their password during login. I'd say that's good enough for our use case anyway. + + + +### Django + MUD? + + + +It's interesting that whereas Twisted is a pretty natural fit for a Python MUD (I have learned that Twisted was in fact first intended for mudding, long ago), many tend to be intrigued and/or surprised about our use of Django. In the end these are only behind-the-scenes details though. The actual game designer using Evennia don't really see any of this. They don't really _need_ to know neither Django nor Twisted to code their own dream MUD. It's possible the combination fits less for some projects than for others. But at least in our case it has just helped us to offer more features faster and with less headaches. \ No newline at end of file diff --git a/devblog/source/2012-10-05-Community-interest.md b/devblog/source/2012-10-05-Community-interest.md new file mode 100644 index 0000000000..213b3790a0 --- /dev/null +++ b/devblog/source/2012-10-05-Community-interest.md @@ -0,0 +1,20 @@ +[![](https://2.bp.blogspot.com/-ETdBSHZJeKM/UG67GMElKvI/AAAAAAAABxI/J-PZhdCZDhU/s320/800px-Billard.JPG)](https://2.bp.blogspot.com/-ETdBSHZJeKM/UG67GMElKvI/AAAAAAAABxI/J-PZhdCZDhU/s1600/800px-Billard.JPG) + +It's fun to see a growing level of activity in the Evennia community. The last few months have seen an increase in the number of people showing up in our [IRC](http://webchat.freenode.net/?channels=evennia) channel and mailing list. With this has come a slew of interesting ideas, projects and MUD-related discussion (as well as a few inevitable excursions into interesting but very non-mud-related territory - sorry to those reading our chat logs). + +One sign of more people starting to actually use Evennia "for real" is how the number of bugs/feature requests have been increasing. These are sometimes issues related to new things being implemented but also features that have been there for some time and which people are now wanting to use in new and creative ways - or systems which noone has yet really used "seriously" before. This is very encouraging, especially since a lot of good alternative solutions, variations and edge cases can be ironed out this way. So keep submitting those Issues, people! + + +The budding Evennia community consists of people with a wide variety of interests, skillset and ambition. + +There are quite a few people who sees Evennia as a great stepping stone for learning Python, or for getting experience with creating a bigger programming project in general. Some are skilled programmers in other languages but we also have a few with only limited prior coding experience. From the experience in chat, it's really quite striking how fast members pick up the ropes. I'd like to think our documentation is at least partially helping here, but of course it helps that Python is inherently very easy a language to learn and use in the first place. +Not all are participating with the goal of building a specific game. The general flow of patches and clone repository merges have also picked up. We have some users which are primarily interested in a coding challenge, to help with fixing bugs and features, or which uses Evennia as a starting point for exploring various web- and technical solutions that may or may not be a part of Evennia in the future. + + +The proposed Evennia game projects are just as varied as its users - and none are yet open to the public. As is common with these things, it's of course hard to determine who actually has the time and discipline to bring their plans to fruition. But I should really start to keep some sort of record of who works on what, I'm terrible with remembering this stuff ... so below is just some sort of summary of my impressions, not a comprehensive listing. + +As can be expected, most proposed Evennia projects concern relatively standard MUD-style games. A few people are into building traditional hack-and-slash variety games, but most want to expand on the concept considerably. There was even one user exploring using Evennia for a RobotWars kind of experience (where you "program" robot programs in a custom language and battle them). Another project (_Avaloria,_ also blogging on the [MUD-dev](http://planet-muddev.disinterest.org/) rss feed) aims for a sort of base-building/strategy mechanic combined with more traditional MUD elements. There are at least two zombie-survival concepts floating around and a few large-scale procedural-content-driven science-fiction text games. One user has apparently a working Smaug->Evennia importer. + +It seems that most Evennia users want to offer some sort of roleplaying environment, or at least a "roleplay-friendly" one. Currently we have at least two MUCK admins who aim to convert their existing, running games to Evennia. Whereas the initial idea was to implement parsers for MUCK's MUF language, it seems the conclusion has now shifted to it being faster and easier to just rewrite the MUF-coded functionality in Python (and maybe use something like [Evlang](http://evennia.blogspot.se/2012/06/coding-from-inside.html) for player scripting instead). Several people have announced their interest in creating "RPI"-style games (_Armageddon_ seems to be a big inspiration here), but there was also a MOO admin and even a writer of Interactive Fiction who dropped into the mailing list to see if Evennia could be used for their style of game. + +How many of these projects actually reach a point of maturity remains to be seen. But that people are wanting to use the system and is really putting it through its paces is encouraging and very helpful for general Evennia development. \ No newline at end of file diff --git a/devblog/source/2012-10-28-Evennia-changes-to-BSD-license.md b/devblog/source/2012-10-28-Evennia-changes-to-BSD-license.md new file mode 100644 index 0000000000..b697750b6f --- /dev/null +++ b/devblog/source/2012-10-28-Evennia-changes-to-BSD-license.md @@ -0,0 +1,7 @@ +[![](https://4.bp.blogspot.com/-mFTXyssJj3Y/UI1NUnfYb-I/AAAAAAAABxs/2yJAilrx3j0/s200/open+neon+95n.jpg)](https://4.bp.blogspot.com/-mFTXyssJj3Y/UI1NUnfYb-I/AAAAAAAABxs/2yJAilrx3j0/s1600/open+neon+95n.jpg)As of today, [Evennia](http://www.evennia.com/) changes to use the very permissive [BSD license.](http://opensource.org/licenses/BSD-3-Clause) + +Now, our previous "Artistic License" was also very friendly. One main feature was that it made sure that changes people made to the core Evennia library (i.e. not the game-specific files) were also made available for possible inclusion upstream. A good notion perhaps, but the licensing text was also quite long and it was clear some newcomers parsed it as more restrictive than it actually was. + +... And let's be honest, it's not like I would have come hunting down anyone not complying fully with the Artistic license's terms. Changing to the much simpler and more well-known BSD license better clarifies the actual licensing situation. + +After all, far too many older MUD-code bases are weighted by a legacy of licensing issues. Anything we can do to avoid this is better in the long run. Indeed we hope this change in licensing will remove eventual licensing doubts for new adopters and have more people join and contribute to the project. \ No newline at end of file diff --git a/devblog/source/2013-01-29-Churning-behind-the-scenes.md b/devblog/source/2013-01-29-Churning-behind-the-scenes.md new file mode 100644 index 0000000000..9877a321fc --- /dev/null +++ b/devblog/source/2013-01-29-Churning-behind-the-scenes.md @@ -0,0 +1,37 @@ +[![](https://1.bp.blogspot.com/-kMcLTjgmpa0/UQfBh7FsD7I/AAAAAAAABys/uGhAhtwG22s/s200/red_curtain_hand3crop_category.jpg)](https://1.bp.blogspot.com/-kMcLTjgmpa0/UQfBh7FsD7I/AAAAAAAABys/uGhAhtwG22s/s1600/red_curtain_hand3crop_category.jpg) + +At the moment there are several Evennia projects churning along behind the scenes, none of which I've yet gotten to the point of pushing into a finished state.  Apart from bug fixes and other minor things happening, these are the main updates in the pipeline at the moment. + +### Multiple Characters per Player/Session + +Evennia has for a long time enforced a clean separation between the _Player_ and the _Character._ It's a much appreciated feature among our users. The _Player_ is "you", the human playing the game. It knows your password, eventual user profile etc. The _Character_ is your avatar in-game. This setup makes it easy for a Player to have many characters, and to "puppet" characters - all you need to do is "disconnect" the Player object from the Character object, then connect to another Character object (assuming you are allowed to puppet that object, obviously). So far so good.  + + + +What Evennia currently _doesn't_ support is being logged in with _different_ client sessions to the _same_ Player/account while puppeting _multiple_ characters _at the same time._ Currently multiple client sessions may log into the same Player account, but they will then all just act as separate views of the same action (all will see the same output, you can send commands from each but they will end up with the same Character).  + + + +Allowing each session to control a separate Character suggests changing the way the session is tracked by the player and Character. This turns out to be more work than I originally envisioned when seeing the feature request in the issue tracker. But if my plan works out it will indeed become quite easy to use Evennia to both allow multi-play or not as you please, without having to remember separate passwords for each Character/account. + + + +### Webserver change to Server level + +Evennia consists of two main processes, the _Portal_ and the _Server._ The details of those were covered in an earlier blog post [here](http://evennia.blogspot.se/2012/08/combining-twisted-and-django.html). Evennia comes with a Twisted-based webserver which is currently operating on the _Portal_ level. This has the advantage of not being affected by Server-reboots. The drawback is however that being in a different process from the main Server, accessing the database and notably its server-side caches becomes a problem - changing the database from the Portal side does not automatically update the caches on the Server side, telling them that the database has changed. Also writing to the database from two processes may introduce race conditions.  + + + +For our simple default setup (like a website just listing some database statistics) this is not a terrible problem, but as more users start to use Evennia, there is a growing interest in more advanced uses of the webserver. Several developers want to use the webserver to build game-related rich website experiences for their games - online character generation, tie-in forums and things like that. Out-of-sync caches then becomes a real concern.  + + + +One way around this could be to implement a framework (such as memcached) for homogenizing caches across all Evennia processes. After lots of IRC discussions I'm going with what seems to be the more elegant and clean solution though - moving the webserver into the _Server_ process altogether. The _Portal_ side will thus only hold a web proxy and the webclient protocol. This way all database access will happen from the same process simplifying things a lot. It will make it much easier for users to use django to create rich web experiences without having to worry about pesky behind the scenes things like caches and the like.  + + + +### Out-of-band communication  + +This has been "brewing" for quite some time, I've been strangely unmotivated to finalize it. Out of band communication means the MUD client can send and receive data to/from the server directly, without the player having to necessesarily enter an active command or see any immediate effect. This could be things like updating a health bar in a client-side GUI, redirect text to a specific client window but also potentially more advanced stuff. I created the Evennia-side oob-handler over Christmas; it allows for client sessions to "sign up" for "listening" to attribute updates, do scheduled checks and so on. It's already in the codebase but is not activated nor tested yet. + +On the protocol side (for serializing data to the client) I have a MSDP implementation ready for telnet subnegotiation, it should be simple to add also GMCP once everything is tested. A JSON-based side channel for the webclient is already in place since a long time if I remember correctly, it just need to be connected to the server-side oob-handler once that's finished. \ No newline at end of file diff --git a/devblog/source/2013-05-13-One-to-Many.md b/devblog/source/2013-05-13-One-to-Many.md new file mode 100644 index 0000000000..74d4c9ecb9 --- /dev/null +++ b/devblog/source/2013-05-13-One-to-Many.md @@ -0,0 +1,16 @@ +[![](https://4.bp.blogspot.com/-M_YNUYvWuiw/UZCNa-U24lI/AAAAAAAAB2o/6wZzFjpCSvk/s320/one-in-many.jpg)](https://4.bp.blogspot.com/-M_YNUYvWuiw/UZCNa-U24lI/AAAAAAAAB2o/6wZzFjpCSvk/s1600/one-in-many.jpg) + +As of yesterday, I completed and merged the first of the three upcoming Evennia features I mentioned in my [Churning Behind the Scenes](http://evennia.blogspot.se/2013/01/churning-behind-scenes.html) blog post: the "Multiple Characters per Player" feature. + +Evennia makes a strict division between _Player_ (this is an object storing login-info and represents the person connecting to the game) and their _Character_ (their representation in-game; Characters are just Objects with some nice defaults). When you log into the game with a client, a _Session_ tracks that particular connection. + +Previously the Player class would normally only handle one Session at a time. This made for an easy implementation and this behavior is quite familiar to users of many other mud code bases. There was an option to allow more than one Session, but each were then treated equally: all Sessions would see the same returns and the same in-game entities were controlled by all (and giving the quit command from one would kick all out). + +What changed now is that the Player class will manage each Session separately, without interfering with other Sessions connected to the same Player. Each Session can be connected, through the Player, to an individual Character. So multiple Characters could in principle be controlled simultaneously by the same real-world player using different open mud clients. This gives a lot of flexibility for games supporting multi-play but also as a nice way to transparently puppet temporary extras in heavy roleplaying games. + +It is still possible to force Evennia to accept only one Session per Player just like before, but this is now an option, not a limitation. And even in hardcore one-character-at-a-time roleplaying games it is nice for builders and admins to be able to have separate staff or npc characters without needing a separate account for each. + +This feature took a lot more work than I anticipated - it consitutes a lot of under-the-hood changes. But it also gave me ample opportunity to fix and clean up older systems and fix bugs. The outcome is more consistency and standardization in several places. There are plenty of other noteworthy changes that were made along the way in the dev branch along with some API changes users should be aware of. +So if you are an Evennia game developer you should peek at the more detailed mailing list  [announcement](https://groups.google.com/forum/#!topic/evennia/EjAW8S2N86I) on what has changed. The wiki is not updated yet, that will come soon. + +Now onward to the next feature! \ No newline at end of file diff --git a/devblog/source/2013-10-22-A-list-of-Evennia-topics.md b/devblog/source/2013-10-22-A-list-of-Evennia-topics.md new file mode 100644 index 0000000000..ba76eb2042 --- /dev/null +++ b/devblog/source/2013-10-22-A-list-of-Evennia-topics.md @@ -0,0 +1,22 @@ +[![](https://4.bp.blogspot.com/-WXVxxb06XBA/UmZ_5TSzmrI/AAAAAAAACE4/NbBAjohOi6E/s1600/building-blocks.jpg)](https://4.bp.blogspot.com/-WXVxxb06XBA/UmZ_5TSzmrI/AAAAAAAACE4/NbBAjohOi6E/s1600/building-blocks.jpg) + +Some Evennia updates. + +## Development +Lots of work has been happening in the [dev-clone](http://code.google.com/r/griatch-evennia-dev/source/list) of Evennia over the last few months. +As alluded to in the last blog, the main work has been to move Evennia's webserver component into the _Server_-half of Evennia for various reasons, the most obvious one to make sure that all database writes happen in the same process, avoiding race conditions. But this move lead to a rework of the cache system, which in turn lead to me having to finalize the plans for how Out-of-Band protocols should be implemented server-side. And once that was finalized, OOB was pretty much implemented anyway. As part of making sure OOB trackers were updated correctly at all times meant reworking some of the ways data is stored ... So one thing led to another making this a bigger update than originally planned. + +I plan to make a more detailed post to the [mailing list](https://groups.google.com/forum/#!forum/evennia) soon with more technical details of the (relatively minor) API changes existing users should expect. The merging of the clone into the main repo is still a little way off, but adventurous users have already started testing things. + +## Google Code +I like Google Code. It's easy to manage and maintain, it has a good wiki and Issue system, not to mention that it allows the use of Mercurial. But in the beginning of September, suddenly all links to our user's clone repositories were _gone_ from the front of the project page_._ Not only that, creating new clones just didn't work anymore. +Now any site can have bugs, and we made an [issue](http://code.google.com/p/support/issues/detail?id=30989) for it (other projects were similarly affected). But nothing happened for the longest time - at least two months given that we didn't report it right away. Just recently the functionality came back but there is no confirmation or comments from Google (our issue is not even closed). +That such a fundamental feature can go unheeded for so long is disturbing to me, driving home the fact that Google is certainly not putting much priority in their code hosting. + +## Community +Some furious activity in the [IRC chat](http://webchat.freenode.net/?channels=evennia&uio=MT1mYWxzZSY5PXRydWUmMTE9MTk1JjEyPXRydWUbb) lately, with new people dropping in to chat and ask about Evennia. For example, an enthusiastic new user learned not only about Evennia but also Python for the first time. It was a lot of fun to see him go from having _no programming experience except_ _mush softcode_ to doing advanced Evennia system implementations in the course of a week and offering good feedback on new features in two. Good show! The freedom you get upgrading from something like softcode to Evennia's use of a full modern programming language was seemingly quite eye-opening. + +Other discussions have concerned the policies around using clones/branches for development as well as the benefits of some other hosting solution. Nothing has been decided on this. There is however now also an official GitHub mirror of Evennia's main repo to be found [here](https://github.com/Evennia/evennia). + +## Imaginary Realities +The deadline for entering articles for the _Imaginary Realities_ web zine [reboot](http://posted-stuff.blogspot.se/2013/10/imaginary-realities-update.html) has passed. It's a good initiative to bring this back - the [original (archived) webzine](http://en.wikipedia.org/wiki/Imaginary_Realities) remains a useful mud-creation resource to this day. I entered two articles, one about Evennia and another about general mud-roleplaying. It will be fun to see how it comes out, apparently the first issue will appear Nov 13 \ No newline at end of file diff --git a/devblog/source/2013-11-28-Out-of-band-mergings.md b/devblog/source/2013-11-28-Out-of-band-mergings.md new file mode 100644 index 0000000000..59ce3811f1 --- /dev/null +++ b/devblog/source/2013-11-28-Out-of-band-mergings.md @@ -0,0 +1,16 @@ +[![](https://4.bp.blogspot.com/-RcZi6McnLtc/Upd90AEpnTI/AAAAAAAACHA/ALGE2Ompi6k/s200/FS-0609-Merger-1.jpg)](https://4.bp.blogspot.com/-RcZi6McnLtc/Upd90AEpnTI/AAAAAAAACHA/ALGE2Ompi6k/s1600/FS-0609-Merger-1.jpg) + +As of today the development repository of Evennia, which has been brewing for a few months now, merged into the main repository. This update grew from one experimental feature to a relatively big update in the end. Together with the "many-character-per-player" feature released earlier, this update covers all the stuff I talked about in my [Behind the Scenes](http://evennia.blogspot.se/2013/01/churning-behind-scenes.html) blog post. + + +-  Evennia's webserver was moved from _Portal_ to _Server_. This moves all database-modifying operations into the same process and neatly avoids race conditions when modifying a game world from various interfaces.  +- The OOB (Out Of Band) handler was implemented. This goes together with a protocol for telnet sub-negotiations according to the [MSDP](http://tintin.sourceforge.net/msdp/) specification. The handler allows on-demand reporting whenever database fields update. It also offers regular polling of properties if needed. A user can customize which oob commands are available to the client and write whatever handlers are needed for their particular game. In the future we'll also add support for GMCP, but the lack of a central, official specification is off-putting (if there is a central document besides accounts of how individual games chose to implement GMCP, please let me know). For our own included web client, we'll likely just use JSON straight off. +- Our channel system is now _typeclassed_. If you are not familiar with Evennia this won't mean much to you - In short it means developers will be able to customize their channel system much easier than in the past since a channel can be treated pretty much like any Python class (thanks go to user Kelketek who actually did the implementation). +- We added the concept of _Tagging_, as a more generalized version of our old _Alias_ system. Tagging is just what it sounds like - it allows you to tag all your objects in order to group them and easily (and efficiently) find them later. Tagging offers a powerful way to create what other code bases refer to as "zones". There are many other possible uses though, such as having effects appear only in certainly tagged rooms, indicate which Characters have joined a particular guild and so on.  +- Behind the scenes there were a lot of cleanups, along with minor API changes mentioned on the mailing list. A slew of older Issues were also fixed with this merge. + +Now, in the immediate future we'll be working on updating the documentation. Evennia prides itself with having a large body of documentation and this update comes with small changes to various existing systems. There are also a few bigger new features (such as OOB). No doubt the near future will also see other issues popping up as more people try things out. + +#### Imaginary Realities update + +Apparently the reboot of _Imaginary Realities_ (to which I contribute two articles) has been [pushed forward a week or two](http://posted-stuff.blogspot.se/2013/11/imaginary-realities-update_15.html). Reason being, apparently, to finalize the actual presentation of the content. I already signed off on the last editorial fixes way before deadline, so I guess it's just to wait and see what comes of it! \ No newline at end of file diff --git a/devblog/source/2013-12-16-Imaginary-Realities-is-back.md b/devblog/source/2013-12-16-Imaginary-Realities-is-back.md new file mode 100644 index 0000000000..bba0a203b3 --- /dev/null +++ b/devblog/source/2013-12-16-Imaginary-Realities-is-back.md @@ -0,0 +1,25 @@ +[![](https://lh4.googleusercontent.com/proxy/CEw97RBnw7FiR_z7fqlj9ELtwgDU-knwkyiW8Ybjg9tjTUAerwxQ0-omUEwpZAz4jKP1ozmnCLHPBh1A4QxrGZzYD3DEFfIdTReu0TTPJwo=s0-d)](http://imaginary-realities.disinterest.org/pics/logo_new.gif)The _[Imaginariy Realities](http://imaginary-realities.disinterest.org/)_ webzine was the place to go to for MUD game design articles in the late 90's. Last released in 2001, its articles are still worth the read for any game designers today.  + +But guess what - this venerable ezine has now returned! You can find the new issue [here](http://journal.imaginary-realities.com/volume-05/issue-01/index.html). + +I think this is a good community initiative worthy of support. I contibuted two articles myself (one of which is about Evennia) and would like to thank the staff/editors who took their work very seriously and did sterling work on getting everything in shape. + +Thanks also to the other authors who penned some very interesting articles. Great job guys! + +My impressions:  + + +- KaVir elaborates in _A modern interface for a modern MUD_ on the advantages of not sticking with an outdated UI just for the sake of it. Adding a more accessible presentation is not that hard and won't ruin your game but rather help it. Whereas I have read his argument about this before, this is a good summary to take to heart. Evennia's javascript web client is currently mainly a telnet clone; there's a lot of things we could offer to make it easier for users to offer a more graphical presentation. +- Molly O’Hara, in her _A well built zone is a work of art,_ outlines a list of useful things to keep in mind when designing a zone. While some of these vary with game type, others do not. I like the suggestion that scripting bugs need not be the most important aspect - syntactic errors can be handled by automated means as long as the design aspect of the zone is solid. +- _A journey through Paradice_ [sic] is Matthew Chaplain's entry on designing a dice-roller using the telnet protocol. Some interesting things here, including making creative use of the telnet character-mode and VT100 control sequences. This ties a bit into KaVir's article, in that the interface used for "modern" MUDs are often voefully missing out on a lot of possibilities. +- _Blind accessibility: challenges and opportunities_ by Matthew “Chaos” Sheahan, is based on interviews with a blind mudder and a game admin having implemented lots of support for seeing-impaired players. This was a really interesting article since I've been pondering myself what could be done from Evennia's core side to help players support players using screen readers. Most seem to be down to configuration options though, and avoiding making colour or ascii art the only sources of information. These are all things Evennia devs implement depend on their game. We may offer some good contribs to build from though. +- _Evennia: an introduction -_ this is mine. It not-so-briefly sums up stuff about Evennia and the more important systems it relies on. +- _Getting a roleplaying scene going_ - another article of mine. This is a light-hearted list of tropes for getting a RP scene going on an RP-mud. It's based on things I've tried or seen myself in play.  +- Darcie “Natilena” Laur laments on the often opaque newbie guides in _Introducing new players and redesigning MUD School._ It describes how she tested (and improved) her own MUD's starter area while testing it on her kids. It made me think more on having Evennia offering easier ways to dump text logs in all sorts of situations. And we find out that kids have the attention span of zombie squirrels - something new learned every day! +- Finally, _The Hunger Game, or how I learned to break the ship from the bottle_ is Michael “Drakkos” Heron's epic about his journey developing and releasing his zombie-survival MUD _Epitaph._ Drakkos is a frequent blogger on the MUD-planet feed, so I knew some of this already, but it's a good read and contains some useful warnings and things-to-think-of for those thinking of starting their own MUD project. We already give some of the same advice (albeit with fewer words) in our wiki but I'm seriously considering linking to Drakkos post from there as well - it gives a more comprehensive treatment and offers a real-world example of the long road to a released game. + +The issue also contains a call to help recover old MUDing resources that has gone offline since their heyday. Having browsed through the classic MUD-dev mailing list archives I can agree they are not quite as easily searchable as could be hoped. It would also be great to see the old comment posts tied to the articles of the old _Imaginary Realities_ articles, if those could be found somehow. + +This is actually one thing which I do miss with this first Imaginary Realities issue - a way for readers to comment on the articles. This would likely mean a much higher level of complexity and work though, so I can certainly see why it's not there - using the existing MUD forums is probably enough for now. + +Anyway, I'm happy to see this thing getting off on a good start. I'm already looking forward to the next issue! \ No newline at end of file diff --git a/devblog/source/2014-01-24-Looking-forwards-and-backwards.md b/devblog/source/2014-01-24-Looking-forwards-and-backwards.md new file mode 100644 index 0000000000..011c46c844 --- /dev/null +++ b/devblog/source/2014-01-24-Looking-forwards-and-backwards.md @@ -0,0 +1,13 @@ +[![](https://2.bp.blogspot.com/-wjnGsBNyUqg/UuLtTZAo_II/AAAAAAAACJE/vYvxUE0zIpw/s1600/1390625062_binoculars.png)](https://2.bp.blogspot.com/-wjnGsBNyUqg/UuLtTZAo_II/AAAAAAAACJE/vYvxUE0zIpw/s1600/1390625062_binoculars.png) + +We are almost a month into the new year, time to look forward. + +But first a look backwards. The year of 2013 was a year of big development projects and lots of quiet in the main repository in between. Two major updates were released during the year. + +The first update, the "many sessions per player" update, originated in a feature request that I thought would be easy to implement but which led to far-ranging changes (and honestly, improvements) to how Players and Sessions interconnect. It was a _lot_ more more work than I anticipated. + +The second update was about moving Evennia's web server from the Portal level into the Server-level. The actual moving of the server was actually considerably easier than I thought it would be. But it turned out that a truckload of other things came along with it. Not only did the cache system have to change in order to accommodate the new webs erver, I had to also finalize the Out-of-band structure, since this made use of the cache system. And while I were at it, other fixes were done and ... the update grew and grew. When it finally merged late last year it closed plenty of issues, but it would probably have been better to structure it into more, small updates instead. + +Anyway, 2014 promises continued (and hopefully more continuous and gradual) development of Evennia. The closest upcoming upheaval is our move from Google Code to GitHub in a few days (I'll probably do a blog about that once it's done). Apart from that we are currently in a fixing state, cleaning up and fixing remnant issues from the big mergers. + +Another Issue of the MUD e-zine [Imaginary Realities](http://journal.imaginary-realities.com/volume-05/issue-01/index.html) is coming too. I just contributed with an Evennia-related article. If anyone reading this blog has anything MUD-related to write about, do consider [contributing](http://posted-stuff.blogspot.se/2013/12/write-article-for-imaginary-realities.html) before January 31, they need more articles! I don't think you need to be too advanced, anything from a mud-development anecdote to retells of good MUD gaming memories might be interesting I would think. \ No newline at end of file diff --git a/devblog/source/2014-02-08-Moving-from-Google-Code-to-Github.md b/devblog/source/2014-02-08-Moving-from-Google-Code-to-Github.md new file mode 100644 index 0000000000..f04695b524 --- /dev/null +++ b/devblog/source/2014-02-08-Moving-from-Google-Code-to-Github.md @@ -0,0 +1,152 @@ +[![](https://1.bp.blogspot.com/-O_mHSjm4u90/UvVmXrY3kkI/AAAAAAAACJc/5bREd9YEbLU/s1600/ar12943602961303.jpg)](https://1.bp.blogspot.com/-O_mHSjm4u90/UvVmXrY3kkI/AAAAAAAACJc/5bREd9YEbLU/s1600/ar12943602961303.jpg) + +A few weeks back, the Evennia project made the leap from Google Code to GitHub ([here](https://github.com/evennia/evennia)). Things have been calming down so it's time to give a summary of how the process went. + +Firstly I want to say that I liked Google Code. It did everything expected of it with little hassle. It had a very good Issue system (better than GitHub in my opinion) and it allowed us to use Mercurial instead of Git for version control (I just happen to like Mercurial better than Git, so sue me). Now, GitHub is getting to be something of a standard these days. But whereas our users have occationaly inquired about us making the move, I've been reluctant to do so.  + +The problem I _did_ have with Google Code was that I got the increasing feeling that Google didn't care all that much about it. It worked decently, but it was not really going anywhere either. What finally made me change my mind though was an event just after summer last year. There was a bug in Google Code that made the links to online clones disappear. It was worse than that - creating new online clones of the main repo didn't work - people wanting to contribute using a clone just couldn't.  + +This is extremely critical functionality for a code-sharing website to have! I made a bug report and many other projects chimed in seeing the same issues. Eventually the links returned and everything worked the way it had. But it took _several months_ before this critical bug was fixed. Even then Google didn't even bother to close my issue. This suggested quite strongly to me that Google Code is not really a priority even for its parent company. It was time to consider a move. + +I was never personally a fan of Git. It is undoubtedly powerful, but I always felt its syntax way too archaic and the number of ways to shoot yourself in the foot way too many. But I do like GitHub better than BitBucket (I've used both in other projects), so that's where we nevertheless were heading. + +Already last year I created an Evennia "organization" on GitHub and one of our  users first helped to set up a Git Mirror of our Mercurial repo. The idea was a good one - have a mirror on GitHub, allowing the transition to be more gradual. In the end this didn't work out though - there were some issue with the hg-git conversion and the mirror never didn't actually update. When I checked back and it was three months behind we just removed that first ill-fated version.  + +In the end I decided to not fiddle about with it, but to move everything over in one go. + + +###  Converting the repository + +I set aside a new folder on my hard drive and cloned the original mercurial repo into a new sub folder. A good idea is to set up a quick Python [virtual environment](https://pypi.python.org/pypi/virtualenv) for easily getting updated dependencies of build scripts.  + +I initialized an empty Git repository and used a program called [hg-fast-export](https://github.com/frej/fast-export) to convert. As it turned out there were some finer details to consider when doing that: + + +- The most obvious one was that the conversion initially failed, complaining about the Mercurial original containing "unnamed branches". These came from a contributor who did _something_ to spawn off all sorts of weird branches with little purpose. I should not have merged those into main in the first place, but in those days I didn't know mercurial well enough to be  concerned. In the end I simply used mercurial's MQ extension to remove the unnamed (and unused) branches so the conversion could complete. +- The second issue was that Mercurial is less stringent about its author strings than Git is. Git's author string is "name ". Over the years we have gotten contributions from people with all sorts of combinations of names, with or without an email address. So for this we had to supply a mapping file to the converter. It's basically a list of old_author_string = new_author_string and allows for grouping the various used names as needed (some of them were the same person using slightly different author strings).  + + +Once this was in place, the repo conversion worked fine. It was just a matter of changing the .hgignore file to a .gitignore file and change some code that made use of mercurial to get and display the current revision id. + +###  Converting the Wiki, part one + + +Evennia's wiki consitutes our documentation, it's some 80+ pages or so by now. Definitely not something we want to loose. Google Code use a dialect of MediaWiki whereas GitHub's wiki supports a few other formats, like markdown or reST. I needed to convert between them. + +Digging around a bit I found [googlecode2github](https://github.com/trentm/googlecode2github.git). This download contains python scripts for converting the wiki as well as Issues. I didn't really get the issues-converter to work, so I had to find another solution for that (see next section). + +All in all, the initial wiki conversion worked decently - all the pages were converted over and were readable. I was even to the point of declaring success when finding the damn thing messed up the links. Googe Code writes links like this: [MyLink Text to see on page]. The script converted this to [[MyLink|Text to see on page]]. Which may look fine except it isn't. GitHub actually wants the syntax in the inverse order: [[Text to see on page|MyLink]]. + +Furthermore, in Google Code's wiki, code blocks were marked with + +``` +{{{ + +}}} +``` + +In markdown, code blocks are created just by indenting the block by four spaces. The converter dutifully did this - but it didn't add empty lines above and below the block, which is another thing markdown requires. The result was that all code ended up mixed into the running text output. + +I could have gone back and fixed the converter script, but I suspected there would be enough small things to fix anyway. So in the end I went through 80+ pages of fixing link syntax and adding empty lines by hand. After that I could finally push the first converted wiki version up to the GitHub wiki repository. + +Some time later I also found that there is a way to let GitHub wiki pages use syntax highlighting for the language of your choice. The way to do this is to enclose your code blocks like this: + +```` +```python + + + +``` +```` + +This is apparently "GitHub-flavoured" markdown. So another stint into all the pages followed, to update everything for prettiness. + + +###  Converting Google Code Issues + + +I didn't want to loose our Issues from Google Code. I looked around a bit and tested some conversions for this (it helps to be able to create and delete repos on GitHub with abandon when things fail). I eventually settled on [google-code-issues-migrator](https://github.com/arthur-debert/google-code-issues-migrator). + +This is a Python script that gathers all the Issues from a given Google Code project. It then uses GitHub's API to re-post the issues. It retains the issue numbers and re-maps the Google Code Issue tags to GitHub's equivalent. It didn't retain most other formatting and whereas I ended up as the creator of all issues, the converter included the name of the original author as well as a link back to the original Google Code one. I found that to be quite sufficient for our needs.  + + +###  Converting the IRC announcer + + +A lot of development discussion goes on in our IRC channel #evennia on Freenode. There is an announcer bot in there that I've written, that collates information from various sources and reports it in the IRC channel: + +- Repository updates +- Wiki updates +- Issue creation and updates +- Mailing list/forum posts +- Dev-blog updates (this blog) + + +Say what you will about Google, but they are great at offering RSS feeds to all their stuff. So my IRC bot was basically a glorified threaded RSS reader that echoed changes to the channel as they came in. This had been working nicely for years. + +GitHub does offer RSS feeds to -some- of their offerings, but it's a lot more patchy. I eventually had to do quite a bit of hacking to get everything reporting the way we were used to. + + +- GitHub has its own IRC announcer bot that reports to IRC. The problem is that this will connect, send message and then disconnect. This causes a lot of spam in the channel. We neither can nor want to set +n on our channel to allow external messages either. The way I solved this was to expand my own custom IRC bot to sit in _two_ irc channels. The GitHub announcer connects to only one of them (so this gets all the spammy connect messages). My IRC bot picks up the announcement and echoes it cleanly to our main #evennia channel.  It works really well. +- Issues are handled by the GitHub announcer in the same way.  +- GitHub has no automatic way to report wiki updates. It doesn't even have a proper RSS feed. However, a user clued me in on using the [pipes](http://pipes.yahoo.com/pipes/pipe.info?_id=02bfefe73ba14054570ce82f1929e433) website to relay an RSS feed from github. I then configured my IRC bot to check that RSS and report it (I also changed the IRC colours to match the GitHub-announcer ones). +- Mailing list and blog haven't changed, so those are still handled via RSS as before. + + +All this done, the modified IRC announcement works well. + + +###  Closing the book on Google Code + + +At this point all the critical things were moved over. So after some heads-up warnings on the mailing list (and users helping to rewrite our documentation to use Git instead of mercurial) we eventually made the official move. + +One thing I really dislike is when a project switches hosts and don't let users know about it in their revision history. So I made a mercurial-only last commit announcing that the repo is closed and giving the link to the new one. + +The Google Code page doesn't go anywhere, but I changed the front page to point to GitHub instead. I even made an issue in the Issue tracker with a title telling people not to use that tracker anymore. Finally I re-pointed all the links on [http://www.evennia.com](http://www.evennia.com/) to GitHub and made a mailing list posting. Move was officially complete. + + +###  Converting the Wiki, part 2 + + +At this point were were officially moved over and I started to look into getting fancy with our documentation. We have for the longest time made automated translations of our wiki for compiling by [ReadTheDocs](https://readthedocs.org/projects/evennia/). + +Getting Google Code's special wikimedia syntax into reST (that ReadTheDocs uses) used to mean jumping through a few hoops. My hackish solution worked in two steps. First a custom python script (whose originating url I can no longer find, sorry) converted the Google Code wiki to HTML. Once this was done, [pandoc](http://johnmacfarlane.net/pandoc/) converted the files from HTML to reST. The result was ... acceptable. There were some minor issues here and there but mostly the result was readable. + +I figured that converting from the more standard Markdown of the GitHub wiki to reST should be a breeze by comparison. Not so. + +The first hurdle was that the version of pandoc coming with my Linux distribution was too old to support Github-flavoured markdown syntax. I knew from before that Pandoc worked so I didn't want to start with something else. I had to download the several hundred MBs needed by the Haskell build environment and their package manager in order to get and compile all the dependencies and finally the latest version of pandoc. To their credit it was all a very streamlined experience, it just took quite some time. + +The second hurdle came when finally looping pandoc to convert all wiki files. It turns out to be that the [[Text on page|address]] syntax I had manually corrected earlier is a special syntax offered by _Gollum,_ the engine powering GitHub's wiki behind the scenes. None of the markdown-to-reSt converters I looked at (pandoc or otherwise) even recognized this syntax as a link at all. As it turns out, normal markdown actually expects its links in the format [Text on page](address). + +I was not going to go through and edit all those pages _again._ So my next step was to write a script to scan and replace all the [[...|...]] syntax in our wiki and replace it with the standard markdown one. After this the markdown files converted to reST quite nicely -- formatting-wise they look much better than the old wiki to HTML to reST chain I had to use from Google Code. + +Problem was that when compiling these reST pages into HTML with Sphinx, no links worked. + +Each individual page looked okay, just that the links were not pointing to anything reasonable. In retrospect this was not so strange. Pandoc knows nothing about the relationships between files, and clearly the simple naming scheme used for addresses is something the wiki softwares knows and Sphinx does not. + +Some thinking lead to a custom Python script for renaming the link targets in the converted pages to their html page name. This needed to handle the fact that wiki links also allows whitespace. So the [Start](Getting Started) link would be converted to [Start](GettingStarted.html), which seems to be the format with which Sphinx will generate its pages. + +One also needs to have a "toc" (Table of Contents) to tie all those pages together for the benefit of Sphinx. I just used a "hidden" toc, letting my converter script add this to the bottom of my normal index file. As long as it's included _somewhere,_ Sphinx will be happy. + +Originally I put the reST files in a subfolder of the GitHub wiki repo, I thought I could just point ReadTheDocs to that repo later. The GitHub wiki has a strange "feature" though. It seems to pick its wiki pages from _wherever_ they are in the repo, no matter if they are in the root or in subfolders. Suddenly I was starting to see reST-style pages appear in the online wiki, and sometimes I would get the markdown version (the two would go out of sync). Very strange and confusing. + +Since the files clearly "polluted" our wiki, I had to move the converted reST files to a separate branch of the wiki repository. This has the advantage of keeping all the support scripts and converter mechanisms separate from the normal wiki content. ReadTheDocs can luckily be set to read its information from another branch than master, so finally the latest converted wiki can again be read there! + +That concludes what I think was the last main conversion effort. Phew! + + +###  Impressions so far + + +GitHub is nice. The merge requests and easy way to comment on them are really good. Since people are more familiar with using GitHub overall, it does seem to be a shorter step for people to make a fork and contribute small things. Doing the same in Google Code was probably not harder per se, just something less people were previously familiar with. + +Due to the modular way Evennia is structured, people are recommended to make a fresh clone of the new Git repo and simply copy their plugin source files and database back into it. So far this seems to have gone smoothly. + +The GitHub issue tracker is worse than the Google Code one. It has no good way to order Issues or list them in a more compact form (nor in a matrix). Not having good issue templates is really limiting; having to reply to issues only to ask for basic info they should include in their issue is an unnecessary time sink. + +I also find that there is no clear way to announce an issue change (like "Needing more information"). Tags work partly for this, but setting them is not announced anywhere as far as I can tell - they are just there. + +Most things also takes so much spaaace. Overall GitHub seems designed for people with big monitors. I have those, but most of the time I prefer working on my laptop. I'm sure it's a matter of habit, but Google Code is very compact by comparison. It gave a lot better overview of things. On GitHub I have to scroll everywhere and this is true both in the repo view, wiki and issues. + +These small quips nonwithstanding, I think this move will serve us well. There is a good wibe of development and continuing improvement going on at GitHub. There's plenty of help and tutorials all over. Since so many people are using GitHub, problems are more likely to have been answered before. And of course we hope this will in effect help more people find Evennia and join the fun. \ No newline at end of file diff --git a/devblog/source/2014-05-16-Imaginary-realities-volume6-issue-1.md b/devblog/source/2014-05-16-Imaginary-realities-volume6-issue-1.md new file mode 100644 index 0000000000..2d0cdec346 --- /dev/null +++ b/devblog/source/2014-05-16-Imaginary-realities-volume6-issue-1.md @@ -0,0 +1,21 @@ +title: Imaginary Realities volume 6, issue 1 + +--- + +[![](https://lh4.googleusercontent.com/proxy/CEw97RBnw7FiR_z7fqlj9ELtwgDU-knwkyiW8Ybjg9tjTUAerwxQ0-omUEwpZAz4jKP1ozmnCLHPBh1A4QxrGZzYD3DEFfIdTReu0TTPJwo=s0-d)](http://imaginary-realities.disinterest.org/pics/logo_new.gif) + +I'm a bit late with writing about it, but the latest issue of _Imaginary Realities_ has been out for a month or so now. You can find it [](https://www.blogger.com/)[here](http://journal.imaginary-realities.com/volume-06/issue-01/index.html). + +Here is a brief summary of the articles in the latest issue.  + + +- In _**A Journey Through Paradice, Part II**,_ Matthew Chaplan continues his description of the C++ codebase _Paradice9_, notably focusing on its input handling, which uses character-mode telnet to produce plenty of interesting effects in a custom terminal client. There are plenty of interesting features (or potential features) discussed. An example is the client knowing to store the receiver of a "reply" command the moment the command is entered rather than waiting for the player the press return (at which point someone else might have written to you and the reply-to target would have changed in a traditional setup). There is no denying the power of having a custom client for your game. And whereas I think some more secure protocol than telnet would maybe be better to use when you control both server and client anyway, it's really interesting to see the power and features you can achieve with it.  +- _**Building a Giant Mech in Evennia** -_ this is my entry for this issue; a short little tutorial on designing a machine of mirth and mayhem in Evennia.  +- Richard “KaVir” Woolcock's _**Describing a Virtual World**_ covers the different uses of dynamically created descriptions in text games. He summarizes the types, from the most common, fully static room description up unto the character-dependent procedurally generated descriptions in his own _GodWarsII_ mud. It's a fascinating read since it not only goes into rooms but also how to build piecemeal and situation-aware character and object descriptions as well as procedural help and quest info. The techniques are not for every type of game of course. But a good and informative read for anyone interested in game design. +- _**Dynamic room descriptions**,_ by Jana, are also covering room descriptions although takes a more conservative conclusion than the full procedural contruction of KaVir's article. This covers the hybrid hand-written description - that is hand-written text that uses a markup language for optional or situation-dependent text. It makes for a good text on implementing such a system (an Evennia equivalent is _extended_room.py_ found in our _contrib_ folder.  +- **_Saddle Up - A Personal Story about Riding Your Demon to Success_** is a summary and inspirational story by Michael "Drakkos" Heron. It ties back to his work with _Epitath_ and how it has affected and improved (and continues to improve) his personal and professional life. I like that he manages to include his game development work into his teaching and reasearch, a cool way to make use of your hobby. He has a point on the usability of a coding hobby like this: I myself have had lots of use and even landed project work based on my work with Evennia. One of our users landed his current job based on knowledge he learned working with Evennia. So there is definitely an advantage to mud-development outside the hobby realm. +- _**The Successful Quest Builder**_ by John "TheDude" Robinette and Joanna "Lorana" Liberty covers the construction of a Quest from the designer's perspective. Rather than focusing on gameplay considerations the authors here focus on the technical aspects; learning the codebase's tools and things to think about debugging and developing something that is enjoyable for the players. +- The article **_Your MUD Should Have an Account System_** finally, is Matthew “Chaos” Sheahan's argument as to why a game should use a single login account system rather than the old way of creating a new account per player. Much of the argument is around converting an old-school code base into this configuration and how it's not as hard as one may think. I fully agree on his assessment (although I wonder just how "easy" it is to patch on such a system on an old-running mud). He even mentions Evennia as an example of a modern codebase having this functionality out of the box (yay!). + + +Deadline for the next issue is [announced to be May 31 2014](http://posted-stuff.blogspot.se/2014/04/write-article-for-imaginary-realities.html) so don't be shy to contribute your own article. Richard Tew hints at in his introduction, finding people to write articles is the tricky part still. \ No newline at end of file diff --git a/devblog/source/2014-06-15-Bringing-back-Python-memory.md b/devblog/source/2014-06-15-Bringing-back-Python-memory.md new file mode 100644 index 0000000000..80f99e16ec --- /dev/null +++ b/devblog/source/2014-06-15-Bringing-back-Python-memory.md @@ -0,0 +1,26 @@ +[![](https://2.bp.blogspot.com/-AoyW71rwdmw/U52RCQqYoDI/AAAAAAAADnU/KH7QGKf_GmM/s1600/RAM.jpg)](https://2.bp.blogspot.com/-AoyW71rwdmw/U52RCQqYoDI/AAAAAAAADnU/KH7QGKf_GmM/s1600/RAM.jpg) + +Lately I've done work on the memory management of Evennia. Analyzing the memory footprint of a python program is a rather educational thing in general. + +Python keeps tracks of all objects (from variables to classes and everything in between) via a memory reference. When other objects reference that object it tracks this too. +Once nothing references an object, it does not need to be in memory any more - in a more low-level languages this might lead to a memory leak. Python's _garbage collector_ handles this for us though - it goes through all abandoned objects and frees the memory for usage by other things. The garbage collector will however _not_ do its thing as long as some other object (which will not be garbage-collected) _still_ holds a reference to the object. This is what you want - you don't want existing objects to stop working because an object they rely on is suddenly not there. + +Normally in Django, whenever you retrieve an database model instance, that exists only in memory then and there. If you later retrieve the same object from the database, the model instance you have to work with is most likely a new one. This is okay for most usage, but Evennia's typeclass system (described in an earlier blog entry) as well our wish to store temporary properties on models (existing until next server restart) does not work if the model instance we get is always different. It would also help if we didn't have to load models from the database more than necessary. + +For this reason, Evennia uses something called the _idmapper_. This is a cache mechanism (heavily modified for Evennia) that allows objects to be loaded from the database only once and then be reused when later accessed. The speedup achieved from this is important, but as said it also makes critical systems work properly. + +The tradeoff of speed and utility is memory usage. Since the idmapper never drops those references it means that objects will never be garbage collected. The result was that the memory usage of Evennia could rise rapidly with an increasing number of objects. Whereas some objects (like those with temporary attributes) should indeed not be garbage collected, in a working game there is likely to be objects without such volatile data. An example might be objects that are not used some of the time - simply because players or the game don't need them for the moment. For such objects it may be okay to re-load them on demand rather than keep them in memory indefinitely. + +When looking into this I found that simply force-flushing the idmapper did _not_ clean up all objects from memory. The reason for this has to do with how Evennia references objects via a range of other means. The reference count never went to zero and so the garbage collector never got around to it.  + +With the excellent [objgraph](https://pypi.python.org/pypi/objgraph) library it is actually pretty easy to track just what is referencing what, and to figure out what to remove. Using this I went through a rather prolonged spree of cleanups where I gradually (and carefully) cleaned up Evennia's object referencing to a point where the only external reference to most objects were the idmapper cache reference. So removing that (like when deliberately flushing the cache) will now make the object possible to garbage-collect. + +[This](http://postimg.org/image/p00v4oinl/) is how the reference map used to look for one type of Evennia object (ObjectDB) before the cleanup. Note the several references into the ObjectDB and the cyclic references for all handlers (the cyclic reference is in itself not a problem for reference-counting but they are slow and unnecessary; I now made all handlers use lazy-loading with weak referencing instead). + +[This](http://postimg.org/image/90i7l4mlt/) is how the reference map looks for the same object now. The __instance__ cache is the idmapper reference. There are also no more cyclic references for handlers (the display don't even pick up on them for this depth of recursion). Just removing that single link will now garbage-collect ObjectDB and its typeclass (ignore the _g_ reference, that is just the variable holding the object in ipython). +We also see that the dbobj.typeclass <-> typeclass.dbobj references keep each other alive and when one goes the other one goes too - just as expected. + +An curious aspect of Python memory handling is that (C-)Python does _not_ actually release the memory back to operating system when flushing the idmapper cache. Rather Python makes it internally available so that it does not need to request any more. The result is that if you look at Evennia with the _top_ command, its memory requirement (for example while continuously creating new objects) will not actually _drop_ on a idmapper flush, it will just _stop rising_.  This is discussed at length in [this blog](http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object.htm), it was good to learn it was not something I did at least. + + +Apart from the memory stuff, there is work ongoing with fixing the latest batch of user issue reports. Another dev is working on cleaning up the web-related code, it should make it a lot cleaner to overload web functionality with custom code. One of those days I'll also try to sit down and finally convert our web client from long-polling to use web sockets now that Evennia suppports web sockets natively. Time, time ... \ No newline at end of file diff --git a/devblog/source/2014-06-30-Webby-stuff.md b/devblog/source/2014-06-30-Webby-stuff.md new file mode 100644 index 0000000000..bd4aa64029 --- /dev/null +++ b/devblog/source/2014-06-30-Webby-stuff.md @@ -0,0 +1,18 @@ +[![](https://3.bp.blogspot.com/-82MT42ksmuU/U7FpMy-LgGI/AAAAAAAADqA/ok6MfvNNTvY/s1600/spiderweb.jpg)](https://3.bp.blogspot.com/-82MT42ksmuU/U7FpMy-LgGI/AAAAAAAADqA/ok6MfvNNTvY/s1600/spiderweb.jpg) + +Latest Evennia come with a range of improvements, mainly related to its integration with the web. + + +### New and improved ways to expand the website/webclient + + +Thanks to the work of contributor Kelketek, Evennia's Django-based web system (website and webclient) has been restructured to be much easier to expand. Previously you had to basically copy the entire web/ folder into your game and modify things in-place. This was not ideal since it made it inherently harder to update when things changed upstream. Now Evennia makes use of Django's _collectstatic_ functionality to allow people to plugin and overload only the media files and templates that they need. Kelketek wrote a new and shiny [web tutorial](https://github.com/evennia/evennia/wiki/Web%20Tutorial) explaining just how things work. + + + +### Websocket-based webclient with OOB + + +Evennia's webclient was an ajax-based one using a long polling ("comet") paradigm to work. These days all modern browsers support [websockets](http://en.wikipedia.org/wiki/WebSocket) though, a protocol that allows asynchronous server-client communication without the cludgery of long polling. So Evennia's new webclient will now use websockets if the browser supports it and fall back to the old comet client if it does not. + +The new client also has full support for OOB (Out-of-band) communication. The client uses JSON for straight forward OOB messaging with the server. As part of this, I had an excuse to go back to clean up and make the OOB backbone of Evennia more complete. The server-side oob commands are borrowed from [MSDP](http://tintin.sourceforge.net/msdp/) but the server side is of course independent of communication protocol (so webclient and telnet extensions can call the same server-side callbacks). I've not yet finalized the documentation for how to use the OOB yet, that will come soon-ish. \ No newline at end of file diff --git a/devblog/source/2014-08-04-Dance-my-puppets.md b/devblog/source/2014-08-04-Dance-my-puppets.md new file mode 100644 index 0000000000..c40a151156 --- /dev/null +++ b/devblog/source/2014-08-04-Dance-my-puppets.md @@ -0,0 +1,44 @@ +[![](https://lh3.googleusercontent.com/proxy/hraqnaf9pLN_5rYSN5vnqysZwOaipiA32vImwp1-TWic6HtbYxqlwtRLjJgl9WQd6IY0TsCGPiPaEw8HEEgcxx_yy9S3E3KRKRk0Ksdm50RDLgAtNaftkVdS2EGU0nuOqdIeUHIWmijKMWNtknEj7891LEFUgjaTDCcsvenRq7f3032soPG_mfUz5bXw7PMGiyQd42PSnvbBh4ufwvzvlQZp-93GVKsTH40MN5vgQm0SfEDVp4fNY49Yyo16MLMQM5MVEfWA9EKmy_HxCPwX07NmWLwIRbRD1_q821tQXDaYeJzzj40If2hJrB1EKPTzo41A7HKfsPbRnla48S9wtqcny_kTO88AdcF1=s0-d)](http://upload.wikimedia.org/wikipedia/commons/thumb/7/70/5016_-_Archaeological_Museum,_Athens_-_Dolls_-_Photo_by_Giovanni_Dall%27Orto,_Nov_13_2009.jpg/374px-5016_-_Archaeological_Museum,_Athens_-_Dolls_-_Photo_by_Giovanni_Dall%27Orto,_Nov_13_2009.jpg) + +In many traditional multiplayer text engines for MUD/MUSH/MU*, the player connects to the game with an account name that also becomes their character's in-game name. When they log into the game they immediately "become" that character. If they want to play with another character, they need to create a new account. + +A single-login system is easy to implement but many code bases try to expand with some sort of "account system" where a single login "account" will allow you to manage one or more game characters. Matthew “Chaos” Sheahan  beautifully argues for the benefits of an account system in the April issue of [Imaginary Realities](http://journal.imaginary-realities.com/volume-06/issue-01/index.html); you can read his article [here](http://journal.imaginary-realities.com/volume-06/issue-01/your-mud-should-have-an-account-system/index.html). + + + +### Evennia and account systems + +First a brief show of how Evennia handles this. We use the following separation: + + +**Session(s) <-> Player <-> Objects/Characters(s)** + + +The _Session_ object represents individual client connections to Evennia. The _Player_ is our "account" object. It holds the password hash and your login name but has no in-game existence. Finally we have _Objects_, the most common being a subclass of Object we call _Character._ Objects exist in the game. They are "puppeted" by Sessions via the Player account. + +From this separation an account system follows naturally. Evennia also offers fully flexible puppeting out of the box: Changing characters (or for staff to puppet an NPC) is simply a matter of "disconnecting" from one Character and connecting to another (presuming you have permission to do so). + + +### The Multisession modes of Evennia + +This is the main gist of this entry since we just added another of these (mode 3). Evennia now offers four different _multisession modes_ for the game designer to choose between. They affect how you gamers may control their characters and can be changed with just a server reload.  + +#### Mode 0 + +This is emulates the "traditional" mud codebase style. In mode 0 a Session controls one Character and one character only. Only one Session per account is allowed - that is, if a user try to connect to their Player account with a different client the old connection will be disconnected. In the default command set a new Character is created with the same name as the Player account and the two are automatically connected whenever they log in. To the user this makes Player and Character seem to be virtually the same thing. + +#### Mode 1 + +In this mode, multiple Sessions are allowed per Player account. You still only have one Character per account but you can control that Character from any number of simultaneously connected clients. This is a requirement from MUSHes and some slower-moving games where there are communities of gamers who want to conveniently track the progress of the game continuously on multiple clients and computers.  + +#### Mode 2 + +In multisession mode 2, multiple Characters are allowed per Player account. No Characters are created by default in this mode, rather the default command set will drop you to a simplified OOC management screen where you can create new characters, list the ones you already have and puppet them. This mode offers true multiplaying, where you can connect via several clients simultaneously, each Session controlling a different Character. + +#### Mode 3 + +This mode allows gamers not only to play multiple Characters on the same Player account (as in mode 2) but to also connect _multiple Sessions to each Character._ This is a multi-character version of Mode 1, where players can control the same Character via Player logins from several different clients on different machines in any combination. + + + +It's interesting that some of these modes may seem silly or superfluous to people used to a certain type of MU* yet are killer features for other communities. It goes to show how different the needs are for users of different game styles. \ No newline at end of file diff --git a/devblog/source/2014-10-02-Slowly-moving-through-town.md b/devblog/source/2014-10-02-Slowly-moving-through-town.md new file mode 100644 index 0000000000..67f5f0f087 --- /dev/null +++ b/devblog/source/2014-10-02-Slowly-moving-through-town.md @@ -0,0 +1,43 @@ +[![](https://4.bp.blogspot.com/-LEAcNfC4EmQ/VC1ySqlz7cI/AAAAAAAAEO4/yVEgs7T6jlo/s1600/Grapevinesnail_01.jpg)](https://4.bp.blogspot.com/-LEAcNfC4EmQ/VC1ySqlz7cI/AAAAAAAAEO4/yVEgs7T6jlo/s1600/Grapevinesnail_01.jpg) + +After getting questions about it I recently added the [Slow Exit contribution](https://github.com/evennia/evennia/blob/master/contrib/slow_exit.py) to the main repository as an example.  + +Delayed movement is something often seen in various text games, it simply means that the time to move from room to room is artificially extended. + +Evennia's default model uses traditional MU* rooms. These are simple nodes with exits linking them together. Such Rooms have no internal size and no inherent spatial relationship to each other. Moving from any Room to any other is happening as fast as the system can process the movement. + +Introducing a delay on exit traversal can have a surprisingly big effect on a game: + +- It dramatically changes the "feel" of the game. It often makes the game feel less "twitch" and slows things down in a very real way. It lets Players consider movement as a "cost". +- It simulates movement speed. A "quick" (or maybe well-rested) character might perceive an actual difference in traversal. The traversal speed can vary depending on if the Character is "running" or "walking". +- It can emulate travel distance. An Exit leading to "the top of the mountain" may take longer to traverse than going "inside the tent". +- It makes movement a "cost" to take into consideration in the game. Moving back and forth over and over across a distance of multiple rooms becomes a much more daunting prospect with a time delay than if you could just zip along as quickly as you could press the button. This also has effects on map and quest design. + +Introducing delayed movement in Evennia is simple. But to explain the idea, let's first briefly explain how Evennia implements Exits. + + +#### A brief sideline: About Exits + + +An Exit in Evennia is a persistent Object sitting in a room. The Exit class is just like any Object except for two things - it stores a "destination" property and it houses a CommandSet on itself. This particular CommandSet holds a single command with the same name as the Exit object. + +Commands and CommandSets are things [I've covered in earlier blog posts](http://evennia.blogspot.se/2012/08/taking-command.html). Suffice to say is that any number of command sets can be merged together dynamically to at any moment represent the commands available to the Character at any given time or situation. + +What happens when an Exit bject is in the same room as a Character is that the Exit's command set is dynamically merged with that of the Character. This means a new command - which always has the same name as the Exit - becomes available. The result is that if the Exit object is called "south", the Character can use the command "south". By default all the command does is to call a hook method on the Exit object. This hook hooks simply moves the calling Character to the "destination" stored by the Exit. Done! + +The nice thing with this is that the whole system is implemented without any special cases or custom hard-wired code. It also means that the entire Exit system can be changed and modified without ever touching Evennia's core. +  + +#### Delaying Exits + +To delay the traversal, the principle is simple - after the Exit command has triggered, wait for a little while before continuing. + +Technically we define a new class of Exit, let's call it SlowExit, inheriting from the default Exit. We locate the spot where the Exit normally sends traversing objects on their way (this is a method called move_to()). + +Since Evennia is based on Twisted, we use Twisted's intrinsic CallLater() function to delay the move for as many seconds we desire (in the contrib I use a thin wrapper around CallLater called delay()). The result is that the command is called, you get a little text saying that you have started moving ... and a few seconds later you actually move. + +Once one understands how Exits work it's really quite straight forward - see the [code on github](https://github.com/evennia/evennia/blob/master/contrib/slow_exit.py) for more details (it's got plenty of comments). + +In the contrib are also some example utility commands for setting one's movement speed and to abort movement if you change your mind before the timeout has passed. + +This simple start can easily be expanded as befits each individual game. One can imagine introducing anything from stamina costs to make travel time be dynamically calculated based on terrain or other factors. \ No newline at end of file diff --git a/devblog/source/2015-01-19-Building-Django-proxies-and-MUD-libraries.md b/devblog/source/2015-01-19-Building-Django-proxies-and-MUD-libraries.md new file mode 100644 index 0000000000..41db296b31 --- /dev/null +++ b/devblog/source/2015-01-19-Building-Django-proxies-and-MUD-libraries.md @@ -0,0 +1,49 @@ + +copyrights: Image: "Bibliothek St. Florian" by Original uploader was Stephan Brunker at de.wikipedia Later versions were uploaded by Luestling at de.wikipedia. - Originally from de.wikipedia; description page is/was here.. Licensed under CC BY-SA 3.0 via Wikimedia Commons - [link](http://commons.wikimedia.org/wiki/File:Bibliothek_St._Florian.jpg#mediaviewer/File:Bibliothek_St._Florian.jpg) + +--- + +[![](https://lh5.googleusercontent.com/proxy/06dVvLrRNe-EqZJBbCF12T6O69WuNMoJpK1Vo_aIRTed86sKe49GhKvHrA_y02GuSOxzZQGojbU0YCyHQzqWXVJME8YUJd89A3IDqqCag5b_xnBXxNwwPlq4Mpd7a7JdNQ=s0-d)](http://upload.wikimedia.org/wikipedia/commons/5/5e/Bibliothek_St._Florian.jpg) + +2015 is here and there is a lot of activity going on in Evennia's repository, mailing list and IRC channel right now, with plenty of people asking questions and starting to use the system to build online games. + +We get newcomers of all kinds, from experienced coders wanting to migrate from other code bases to newbies who are well versed in mudding but who aim to use Evennia for learning Python. At the moment the types of games planned or under development seems rather evenly distributed between RPI-style MUDs and MUSH games (maybe with a little dominance of MUSH) but there are also a couple of hack-and-slash concepts thrown into the mix. We also get some really wild concepts pitched to us now and then. What final games actually comes of it, who can tell, but people are certainly getting their MU*-creative urges scratched in greater numbers, which is a good sign. + +Since Christmas our _"devel"_ branch is visible online and is teeming with activity. So I thought I'd post an summary about it in this blog. The more detailed technical details for active developers can be found on Evennia's mailing list [here](https://groups.google.com/forum/#%21topic/evennia/6ug7m872GIk) (note that full docs are not yet written for devel-branch). + +## Django proxies for Typeclasses + +I have written about Evennia's Typeclass system before on this blog. It is basically a way to "decorate" Django database models with a second set of classes to allow Evennia developers to create any type of game entity without having to modify the database schema. It does so by connecting one django model instance to one typeclass instance and overloading __setattr__ and __getattribute__ to transparently communicate between the two. + +For the devel branch I have refactored our typeclass system to make use of Django's _[proxy models](https://docs.djangoproject.com/en/dev/topics/db/models/#proxy-models)_ instead. Proxy models have existed for quite a while in Django, but they simply slipped under my radar until a user pointed them out to me late last year. A proxy model is basically a way to "replace the Python representation of a database table with a proxy class". Sounds like a Typeclass, doesn't it? +Now, proxy models doesn't work _quite_ like typeclasses out of the box - for one thing if you query for them in the database you will get back the original model and not the proxy one. They also do not allow multiple inheritance. Finally I don't want Evennia users to have to set up django Meta info every time they use a proxy. So most work went into overloading the proxy multiclass inheritance check (there is a django issue about how to fix this). Along the way I also redefined the default managers and __init__ methods to always load the proxy actually searched for and not the model. I finally created metaclasses to handle all the boilerplate. We choose to keep the name _Typeclass_ also for this extended proxy. This is partly for legacy reasons, but typeclasses do have their own identity: they are not vanilla Django-proxies nor completely normal Python classes (although they are very close to the latter from the perspective of the end user). +Since typeclasses now are directly inheriting from the base class (due to meta-classing this looks like normal Python inheritance), it makes things a lot easier to visualize, explain and use. Performance-wise this system is en par with the old, or maybe a little faster, but it will also be a lot more straight forward to cache than the old. I have done preliminary testing with threading and it looks promising (but more on that in a future post).  + + +## Evennia as a Python library package +Evennia has until now been solely distributed as a version controlled source tree (first under SVN, then Mercurial and now via GIT and Github). In its current inception you clone the tree and find inside it a game/ directory where you create your game. A problem we have when helping newbies is that we can't easily put pre-filled templates in there - if people used them there might be merge conflicts when we update the templates upstream. So the way people configure Evennia is to make copies of template modules and then change the settings to point to that copy rather than the default module. This works well but it means a higher threshold of setup for new users and a lot of describing text. Also, while learning GIT is a useful skill, it's another hurdle to get past for those who just want to change something minor to see if Evennia is for them. + +In the devel branch, Evennia is now a library. The game/ folder is no longer distributed as part of the repository but is created dynamically by using the new binary evennia launcher program, which is also responsible for creating (or migrating) the database as well as operating the server: + +``` +evennia --init mygame +cd mygame +evennia migrate +evennia start +``` + + +Since this new folder is _not_ under our source tree, we can set up and copy pre-made template modules to it that people can just immediately start filling in without worrying about merge conflicts. We can also dynamically create a setting file that fits the environment as well as set up a correct tree for overloading web functionality and so on. It also makes it a lot easier for people wanting to create multiple games and to put their work under separate version control. + +Rather than traversing the repository structure as before you henceforth will just do import evennia in your code to have access to the entirety of the API. And finally this means it will (eventually) be possible to install Evennia from [pypi](https://pypi.python.org/pypi/Evennia-MUD-Server/Beta) with something like pip install evennia. This will greatly ease the first steps for those not keen on learning GIT. + +## For existing users + +Both the typeclasses-as-proxies and the evennia library changes are now live in the devel branch. Some brave users have already started taking it through its paces (and is helping to flesh it out) but it will take some time before it merges into master. + +The interesting thing is that despite all this sounding like a huge change to Evennia, the coding API doesn't change very much, the database schema almost not at all. With the exception of some properties specific to the old connection between the typeclass and model, code translate over pretty much without change from the developer's standpoint. + +The main translation work for existing developers lies in copying over their code from the old game/ directory to the new dynamically created game folder. They need to do a search-and-replace so that they import from evennia rather than from src or ev. There may possibly be some other minor things. But so far testers have not found it too cumbersome or time consuming to do. And all agree that the new structure is worth it. + +So, onward into 2015! + diff --git a/devblog/source/2015-05-09-Documenting-Python-without-Sphinx.md b/devblog/source/2015-05-09-Documenting-Python-without-Sphinx.md new file mode 100644 index 0000000000..9ec0b22d01 --- /dev/null +++ b/devblog/source/2015-05-09-Documenting-Python-without-Sphinx.md @@ -0,0 +1,36 @@ +copyrights: (Image from [Wikimedia commons](http://en.wikipedia.org/wiki/File:La_Granja_de_San_Ildefonso_Sfinx01.jpg)) + +--- + +[![](https://lh4.googleusercontent.com/proxy/KbIwdEDoKI3YPVgVArY87c8tHYHbRamzP5RsqyRKu62rVXabv9fFLBgcMorVujBaKdsfxtKnFpUUNS1z3OkGBo-en1f9pwaNyYeTu08PxNSR3EB_Zjvs9LtQfcSzXx_h7dU4RdnC81jTtAkfoGEewKp4KOP3dnml_Z6yAMdPFWHjU_r8T5_jjC7Oi0pKp3w1PV9_ep2u9zCF1dLEgvoHDIs=s0-d)](http://upload.wikimedia.org/wikipedia/commons/thumb/f/fd/La_Granja_de_San_Ildefonso_Sfinx01.jpg/800px-La_Granja_de_San_Ildefonso_Sfinx01.jpg) + +Last week Evennia merged its development branch [with all the features mentioned in the last post](http://evennia.blogspot.se/2015/01/building-django-proxies-and-mud.html). Post-merger we have since gone through and fixed remaining bugs and shortened the list at a good clip. + +One thing I have been considering is how to make Evennia's API auto-documenting - we are after all a MUD creation library and whereas our code has always been well-documented the docs were always only accessible from the source files themselves. + +Now, when you hear "Python" and "documentation" in the same sentence, the first thought usually involves Sphinx or [Sphinx autodoc](http://sphinx-doc.org/ext/autodoc.html) in some form. Sphinx produces [very nice looking documentation](https://docs.djangoproject.com/en/1.7/) indeed. My problem is however as follows: + +- I don't want our API documentation to be written in a different format from the rest of our documentation, which is in Github's wiki using Markdown.  Our users should be able to help document Evennia without remembering which formatting language is to be used. +- I don't like reStructuredText syntax. This is a personal thing. I get that it is powerful but it is also really, really ugly to read in its raw form in the source code. I feel the sources must be easy to read on their own. +- Sphinx plugins like [napoleon](http://sphinx-doc.org/latest/ext/napoleon.html) understands this ugliness and allows you to document your functions and classes in a saner form, such as the "Google style". One still needs reST for in-place formatting though. +- Organizing sphinx document trees is fiddly and having had a few runs with sphinx autodoc it's just a mess trying to get it to section the Evennia sources in a way that makes sense. It could probably be done if I worked a lot more with it, but it's a generic page generator and I feel that I will eventually have to get down to make those toctrees myself before I'm happy. +- I want to host the API docs as normal Wiki files on Github (this might be possible with reST too I suppose). + + +Long story short, rather than messing with getting Sphinx to do what I want, I ended up writing my own api-to-github-Markdown parser for the Evennia sources: api2md. Using Python's inspect module and aiming for a subset of the [Google formatted docstrings](http://sphinx-doc.org/latest/ext/example_google.html), this was maybe a day's work in total - the rest was/is fine-tuning for prettiness. +  +Now whenever the source is updated, I follow the following procedure to fully update the API docs: + + +1. I pull the latest version of Evennia's wiki git repository from github alongside the latest version of the main Evennia repository. +2. I run api2md on the changed Evennia sources. This crawls the main repo for top-level package imports (which is a small list currently hard-coded in the crawler - this is to know which modules should create "submodule" pages rather than try to list class contents etc). Under each package I specify it then recursively gets all modules. For each module in that package, it creates a new Markdown formatted wiki page which it drops in a folder in the wiki repository. The files are named after the model's path in the library, meaning you get files like evennia.objects.models.md and can easily cross-link to subsections (aka classes and functions) on a page using page anchors. +3. I add eventual new files and commit the changes, then push the result to the Github wiki online. Done! + +(I could probably automate this with a git hook. Maybe as a future project.) + +The api2md program currently has some Evennia-custom elements in it (notably in which packages it imports) but it's otherwise a very generic parser of Python code into Markdown. It could maybe be broken out into its own package at some point if there's interest.    + +The interesting thing is that since I already have code for [converting our wiki to reST](http://evennia.blogspot.se/2014/02/moving-from-google-code-to-github.html) and ReadTheDocs, I should be able to get the best of both worlds and convert our API wiki pages the same way later. The result will probably not be quite as custom-stunning as a Sphinx generated autodoc (markdown is a lot simpler in what formatting options it can offer) but that is a lesser concern. +So far very few of Evennia's docstrings are actually updated for the Google style syntax (or any type of formatting, really) so the result is often not too useful. We hope that many people will help us with the doc strings in the future - it's a great and easy way to get to know Evennia while helping out. + +But where the sources _are_ updated, [the auto-generated wiki page looks pretty neat](https://github.com/evennia/evennia/wiki/evennia.utils.evtable). diff --git a/devblog/source/2015-05-11-Things-goin-on.md b/devblog/source/2015-05-11-Things-goin-on.md new file mode 100644 index 0000000000..6ada506dc5 --- /dev/null +++ b/devblog/source/2015-05-11-Things-goin-on.md @@ -0,0 +1,19 @@ +copyrights: _Image: Truss from Gems NFX_ + +--- + +[![](https://4.bp.blogspot.com/-2Y8na9nvtog/VU_WU7zs8uI/AAAAAAAAEYc/IyRyP69b99w/s200/Truss-Stage-Light-Truss-.jpg)](https://4.bp.blogspot.com/-2Y8na9nvtog/VU_WU7zs8uI/AAAAAAAAEYc/IyRyP69b99w/s1600/Truss-Stage-Light-Truss-.jpg) + +We are currently in a maintenance and cleanup phase of Evennia, where bugs are found and reported and things are getting more and more stable as people learn and use the new features we merged a few months back. + +Overall though I must say the relatively big changes we did to the infrastructure (making Evennia into a full library and making a complete overhaul of the typeclass system behind the scenes) went over with surprising smoothness. There were a flurry of things to fix immediately after the devel-branch merger but no more than expected. For the big changes it really worked very well I think, with no big disaster stories. We have a few bugs lingering in the issue tracker that need to be addressed but nothing show-stopping. + +I have been a bit busy with various projects off-MUD so to speak. I was contracted for making the cover and illustration for a book (this is not hobby art for once, but a professional commission which I was quite excited to be asked to do). I also author and draw [a fantasy comic as part of another project.](http://griatch-art.deviantart.com/art/Crossing-borders-page-1-527982536) + +I've not been slacking off on on the MUD side though: I have written and submitted an article for the revived [Imaginary Realities e-zine](http://journal.imaginary-realities.com/) (next issue should be out end of May/early June?) and another article (on Evennia) for the new [Optional Realities](http://optionalrealities.com/) MUD community website. I also contributed a game-design blurb for the latter's [Dreaming Big](http://optionalrealities.com/forums/index.php?topic=13.0) contest, where you compete (for cash prizes, actually!) by submitting a game sale's pitch under 600 words. + +The above mentioned Optional Realities website is so far picking up pace with quite good discussion in its forums (the similarity of name with Imaginary Realities is unfortunate, apparently they were not aware of the former when choosing it).  While targeted at discussions of RPI-style games (a sort of sub-genre of roleplay-enforced MUDs with perma-death), it already hosts several invited articles on game design and general game development that can be interesting for any MU* dev. + +People should know by now that I like to support MUD community efforts when possible, and Evennia is thus  listed as an official "affiliate" to Optional Realities (which admittedly means little more than us linking to each other but still). The team behind OR is however also using Evennia for their own "Project Redshift" Sci-fi mud, so we hope to get plenty of good feedback as their project matures. + + \ No newline at end of file diff --git a/devblog/source/2015-05-30-Dreaming-big.md b/devblog/source/2015-05-30-Dreaming-big.md new file mode 100644 index 0000000000..1d3281bed5 --- /dev/null +++ b/devblog/source/2015-05-30-Dreaming-big.md @@ -0,0 +1,11 @@ +title: Dreaming big? + +--- + +[![](https://1.bp.blogspot.com/-o3D_2InpMD0/VWnYBCUe2QI/AAAAAAAAEaQ/3JCuwjCrzjA/s320/Battle_of_the_throne_small.jpg)](https://1.bp.blogspot.com/-o3D_2InpMD0/VWnYBCUe2QI/AAAAAAAAEaQ/3JCuwjCrzjA/s1600/Battle_of_the_throne_small.jpg) + +Optional Reality's [Dreaming Big Challenge](http://optionalrealities.com/forums/index.php?topic=13.0) has come to an end. This was a challenge about designing a "sales pitch" in 600 words or less for a text-based online roleplaying game. This is the kind of pitch you'd tease knowledgeable investors with -  if investors cared about multiplayer text games and you ended up in an elevator with one, that is. + +There were 10 entries to the competition and as the [results are now in](http://optionalrealities.com/forums/index.php?topic=101.0) (including very elaborate judge feedback!), I encourage you to go read all the entries. The focus was on originality and fresh ideas and maybe these short pitches represent a cross-section of the current trends or a whiff of where the wind is blowing for future games. Or maybe it will help inspire you to make a sales pitch of your own. + +You can find all the entries [linked from the second post of this thread.](http://optionalrealities.com/forums/index.php?topic=100.0) Enjoy! \ No newline at end of file diff --git a/devblog/source/2015-06-15-Need-your-help.md b/devblog/source/2015-06-15-Need-your-help.md new file mode 100644 index 0000000000..84630e9d0e --- /dev/null +++ b/devblog/source/2015-06-15-Need-your-help.md @@ -0,0 +1,45 @@ +title: Need your help! + +--- + +[![](https://sites.google.com/site/evenniaserver/_/rsrc/1429261373725/old_evennia/demo_color.png)](https://sites.google.com/site/evenniaserver/_/rsrc/1429261373725/old_evennia/demo_color.png)_This for all you developers out there who want to make a game with Evennia but are not sure about what game to make or where to start off._ + + +### We need an example game + +One of the main critiques Evennia get from newbies is the lack of an (optional) full game implementation to use as an example and base to build from. So, Evennia needs a full, BSD-licensed example game. I'm talking "diku-like", something you could in principle hook up and allow players into within minutes of installing Evennia. The Tutorial world we already have is a start but it is more of a solo quest, it's not designed to be a full multiplayer game. Whereas Evennia supports other forms of MU* too, the idea is that the systems from a more "code-heavy" MUD could easily be extracted and adopted to a more freeform-style game whereas the reverse is not generally true. + +The exact structure of such a game would be up to the person or team taking this on, but it should be making use of Evennia's api and come distributed as a custom game folder (the folder you get with evennia --init). We will set this up as a separate repository under the [Evennia github organisation](https://github.com/evennia) - a spin-off from the main evennia project, and maintained separately. + + +### We need you! + + +Thing is, while I am (and, I'm sure other Evennia core devs) certainly willing to give considerable help and input on such a project, it's _not_ something I have time to take the _lead_ on myself. So I'm looking for enthusiastic coders who would be willing to step up to both help and _take the lead_ on this; both designing and (especially) coding such an example game. Even if you have your own game in mind for the future, you _still_ need to build most of these systems, so starting with a generic system will still help you towards that final goal - plus you get to be immortalized in the code credits, of course. + +### +Suggestion for game + +Being an example game, it should be well-documented and following good code practices (this is something we can always fix and adjust as we go though). The systems should be designed as stand-alone/modular as possible to make them easy to rip out and re-purpose (you know people will do so anyway). These are the general features I would imagine are needed (they are open to discussion): + +- Generic fantasy theme (lore is not the focus here, but it can still be made interesting) +- Character creation module +- Races (say, 2-3) +- Classes (say 2-3) +- Attributes and Skills (based on D&D? Limit number of skills to the minimal set) +- Rule module for making skill checks, rolls etc (D&D rules?) +- Combat system (twitch? Turn-based?) +- Mobs, both friendly and aggressive, with AI +- Trade with NPC / other players (money system) +- Quest system +- Eventual new GM/admin tools as needed +- Small game world (batch-built) to demonstrate all features (of good quality to show off) +- More? Less? + +### I'm interested! + +Great! We are as a first step looking for a driven **lead dev** for this project, a person who has the enthusiasm, coding experience and _drive_ to see the project through and manage it. You will (hopefully) get plenty of collaborators willing to help out but It is my experience that a successful hobby project really needs at least one person taking responsibility to "lead the charge" and having the final say on features: Collaborative development can otherwise easily mean that everyone does their own thing or cannot agree on a common course. This would be a spin-off from the main Evennia project and maintained separately as mentioned above. + +Reply to [this thread](https://groups.google.com/forum/#!msg/evennia/48PMDirb7go/w_hZ1mWhH8AJ) if you are willing to participate **_at any level_** to the project, including chipping in with code from your already ongoing development. I don't know if there'd be any "competition" over the lead-dev position but if multiple really enthusiastic and willing devs step forward we'll handle that then. + +So get in touch! \ No newline at end of file diff --git a/devblog/source/2015-06-22-Announcing-the-Evennia-example-game-Ainneve.md b/devblog/source/2015-06-22-Announcing-the-Evennia-example-game-Ainneve.md new file mode 100644 index 0000000000..c66f238f1d --- /dev/null +++ b/devblog/source/2015-06-22-Announcing-the-Evennia-example-game-Ainneve.md @@ -0,0 +1,22 @@ +title: Announcing the Evennia example-game project "Ainneve" +copyrights: _Image from [loveintoblender](http://loveintoblender.blogspot.se/p/blog-page_21.html)._ + +--- + +[![](https://1.bp.blogspot.com/-QZScgcWodAg/UZu0R2DaJOI/AAAAAAAAAIw/57rwaNQNyx8/s1600/Sunrise+landscape+render+retouches.png)](https://1.bp.blogspot.com/-QZScgcWodAg/UZu0R2DaJOI/AAAAAAAAAIw/57rwaNQNyx8/s1600/Sunrise+landscape+render+retouches.png) + +The Evennia example-game project is underway! + +I was quite impressed with the response I got on the mailing list to my call for developing an Evennia example game (see my [Need your Help](http://evennia.blogspot.se/2015/06/need-your-help.html) blog post). + +The nature of the responses varied, many were from interested people with little to no experience in Evennia or Python whereas others had the experience but not the time to lead it. It was however clear that the interest to work on an "official" Evennia game is quite big. + +I'm happy to announce, however, that after only a week we now have a solid lead developer/manager, [George Oliver](https://github.com/ergodicbreak). Helping him on the technical/architecture side is [Whitenoise](https://github.com/whitenoiseoss) (who, despite a barren github profile, is a professional developer). + +George put together a [game proposal](https://docs.google.com/document/d/1fu-zYvFytKHGxFHD6_bLQ4FrotAQ0izXuWE_gn8q2Oc/edit) based on the [OpenAdventure](http://geekguild.com/openadventure/) rpg, an open-source (CC-SA) ruleset that is also [found on github](https://github.com/openadventure/Open-Adventure). The example game is to be named "Ainneve" and its development is found in a [in a separate repository under the github Evennia organisation](https://github.com/evennia/ainneve). + +All the relevant links and future discussion can be found [on the mailing list](https://groups.google.com/forum/#!msg/evennia/rOMo5DwhXgE/0jmCpj4gITYJ). + +George and whitenoise have already made it clear that they aim to not only make Ainneve a good example Evennia game for others to learn from and build on, but to make the development itself a possibility for people of all skill levels to get involved. So get in touch with them if you are at all interested in Python, Evennia and mud development! + +So thanks to George and whitenoise for taking this on, looking forward to see where it leads! \ No newline at end of file diff --git a/devblog/source/2015-08-27-A-wagon-load-of-post-summer-updates.md b/devblog/source/2015-08-27-A-wagon-load-of-post-summer-updates.md new file mode 100644 index 0000000000..15606cb9ad --- /dev/null +++ b/devblog/source/2015-08-27-A-wagon-load-of-post-summer-updates.md @@ -0,0 +1,34 @@ +copyright: [_The troll here a-cometh_](http://fav.me/d5kp988) by Griatch + +--- + +[![](https://lh3.googleusercontent.com/proxy/5aVkih3LRcrhVNGudLivJ7KJPkDL7bYpusuEQuygCeBQSlnWXawPMkI4a4WCeYkwfnp5LewB3XhGUdNYzcVGd40ZLAz6Oy_-PdVhyZrJtGZqYvBsBOdC8y6cYmiAJ4Vz00l2hv3gQ70yYKbYQyT8khFQFS6grg=s0-d)](http://img10.deviantart.net/adb4/i/2012/322/f/8/the_troll_here_a_cometh_by_griatch_art-d5kp988.jpg) + +Summer vacations are over and work resumes in Evennia land! Here's a wagon-load of small updates on what's going on. +  + +### Ainneve + +The [Ainneve project](https://github.com/evennia/ainneve), the creation of an official, open-source Evennia demo game, has gotten going. The lead devs of the project are keen to make this a collaborative effort and there is a lot of good discussion and code being written. After some slowdown at the end of summer it's bound to pick up again.  + +Ainneve's a rare chance to see a full MUD getting developed from scratch out in the open. [The current issue list](https://github.com/evennia/ainneve/issues) includes tags for difficulty and allows also newbie Python coders to jump in. Not to mention you have a chance to get valuable feedback on your work by seasoned coders! + +So if you are at all interested in making a MUD, try out Python/Evennia or just get involved in a semi-big programming project, this is a great chance to do so. + + +### Imaginary Realities + +Since a few weeks, there is a new issue of [Imaginary realities](http://journal.imaginary-realities.com/index.html) (vol 7, issue 3) is out. As usual I have an article in it. This venerable e-zine was revitalized to include articles on both MU* as well as roguelikes, Interactive fiction and others. Not only is this issue the most content-rich since the reboot, with this issue they have also spruced up their interface to make past issues easier to navigate. + + +- "[A text MUD with a working ecology system](http://journal.imaginary-realities.com/volume-07/issue-03/a-text-mud-with-a-working-ecology-system/index.html)" - in this article Molly O'Hara  details the concepts behind simulating a functioning ecologic network in a game. Interesting stuff and some parts of this is certainly worth considering for any open-world game design. I wonder at which level of detail the system become more complex than the players can appreciate though. +- "[Dispelling the gloom](http://journal.imaginary-realities.com/volume-07/issue-03/dispelling-the-gloom/index.html)" by Tomasz Gruca is an interesting essay on the author's history growing up in the former Soviet Union and eventually finding text adventure games and interactive fiction, a passion he has apparently lately re-kindled. He makes the observation that the current "retro" trend of games have not really reached back to the text-based game world when it comes to mainstream acceptance. +- "[How integral are letters and text to ASCII gaming?](http://journal.imaginary-realities.com/volume-07/issue-03/how-integral-are-letters-and-text-to-ascii-gaming/index.html)"by Mark R. Johnson goes into the practical use of ASCII in traditional rogue-like games (beyond nostalgia). This is a meaty article that goes into both text-as-graphics as well as the use of text for aiding imagination and suggest subtle puzzles in some classic rogue-likes.  +- ["Legend and the lore"](http://journal.imaginary-realities.com/volume-07/issue-03/legend-and-the-lore/index.html) (Hugo Zombiestalker) proclaims the death of the traditional point-and-click adventure game and but then moves on to try to distill just why those games nevertheless was so appealing to him and how it can be applied in modern game designs like zombie-survival MUD _Epitath_ which he is a senior developer for. Plenty of good observations here! +- "[The bonds of mudding"](http://journal.imaginary-realities.com/volume-07/issue-03/the-bonds-of-mudding/index.html) by Clint Itan Kuranes Knapp writes about the community that can spring up on a long-running MUD, the interactions the friends and the relationships that could persist already long before "social media" became a buzz word. A warm text with plenty of anecdotes and examples and things to ponder for both designers and staff when wanting to cater for this type of player bonding.  +- "[The mercurial temperament at the end of the world"](http://journal.imaginary-realities.com/volume-07/issue-03/the-mercurial-temperament-at-the-end-of-the-world/index.html) (Drakkos) discusses NPCs and how they rarely are as interactive as one would want (the term "vend a chat" is a good one I think). He then goes on to how they have implemented their "Mercurial" system for NPCs in _Epitath_. This seems to be a state-AI system where NPCs have moods that affects what they say based on their circumstance and relation to other actors in the world. Sounds really cool and since he goes into some details on the implementation there is a lot to ponder here.  +- "[Where do I begin?](http://journal.imaginary-realities.com/volume-07/issue-03/where-do-i-begin/index.html)" by me, Griatch, discusses one of the more common questions we get in the Evennia chat - 'I want to make a MUD, but how do I begin?' This article starts before Evennia's Game planning wiki page - it discusses assessing your capabilities and resources in the form of programming skills, code bases and motivations to help you figure out what you can realistically accomplish.  + +### Evennia Web client + +In the pipeline I have some updates to Evennia's websocket/JSON MUD-web client component. These are changes that are intended to make the webclient easier to customize and hook into Evennia output using only HTML/CSS. More details on this will be forthcoming when I have more solid stuff to show. \ No newline at end of file diff --git a/devblog/source/2015-09-24-Pushing-through-a-straw.md b/devblog/source/2015-09-24-Pushing-through-a-straw.md new file mode 100644 index 0000000000..38d09eb476 --- /dev/null +++ b/devblog/source/2015-09-24-Pushing-through-a-straw.md @@ -0,0 +1,40 @@ +[![](https://upload.wikimedia.org/wikipedia/commons/thumb/1/1d/DrinkingStraws.jpg/401px-DrinkingStraws.jpg)](https://upload.wikimedia.org/wikipedia/commons/thumb/1/1d/DrinkingStraws.jpg/401px-DrinkingStraws.jpg) + +Recently, a user reported a noticeable delay between sending a command in-game to multiple users and the result of that command appearing to everyone. You didn't notice this when testing alone but I could confirm there was almost a second delay _sometimes_ between entering the command and some users seeing the result. A second is very long for stuff like this. Processing time for a single command is usually in the milliseconds. What was going on? + + +### Some background  + +Evennia has two components, the _Portal_ and the _Server,_ running as two separate processes_._ The basic principle is that players connecting to an Evennia instance connects to the Portal side - this is the outward facing part of Evennia. The connection data and any other input and output will be piped from the Portal to the Server and back again. + +The main reason for this setup is that it allows us to completely reset the Server component (reloading module data in memory is otherwise error-prone or at least very tricky to make generically work in Python) without anyone getting disconnected from the Portal. On the whole it works very well. + + +### Debugging + + +Tracing of timings throughout the entire Server<->Portal pipeline quickly led me to rule out the command handler or any of the core systems being responsible - including both sides of the process divide was a few milliseconds. But in the _transfer_ between Portal and Server, an additional _900_ miliseconds were suddenly added! This was clearly the cause for the delay. + +Turns out that it all came down to faulty optimization. At some point I built a batch-send algorithm between the Server and Portal. The idea was to group command data together if they arrived too fast - bunch them together and send them as a single batch. In theory this would be more efficient once the rate of command sending increased. It was partly a DoS protection, partly a way to optimize transfer handling. + +The (faulty) idea was to drop incoming data into a queue and if the rate was too high, wait to empty that queue until a certain "command/per second" rate was fullfilled. There was also a timed routine that every second force-emptied the queue to make sure it would be cleaned up also if noone made any further commands. + +In retrospect it sounds silly but the "rate of command" was based on a simple two-time-points derivative; + + +> rate = 1 / (now - last_command_time) + + +If this rate exceeded a set value, the batch queuing mechanism would kick in. The issue with this (which is easy to see now) is that if you set your limit at (say) 100 commands / second, two commands can happen to enter so close to each other time that their rate far exceed that limit just based on the time passed between them. But _there are not actually 100 commands coming in every second_ which is really what the mechanism was meant to react to. + +So basically using a moment-to-moment rate like this is just too noisy to be useful; the value will jump all over the place. The slowdown seen was basically the DoS protection kicking in because when you are relaying data to other users, each of them will receive "commands" in quick succession - fast enough to trigger the limiter. These would be routed to the queue and the sometimes-delay simply depended on  when the queue cleanup mechanism happened to kick in. + + +### Resolution + + +Once having identfied the rate measuring bug, the obvious solution to this would be to gather command rates over a longer time period and take an average - you will then filter out the moment-to-moment noise and get an actually useful rate. + +Instead I ended up going with an even simpler solution: Every command that comes in ups a counter. If I want a command rate limit of 100 commands/second, I wait until that counter reaches 100. At that point I check when the time difference between now and when the counter was last reset. If this value is below 1, well then our command rate is higher than 100/second and I can kick in whatever queuing or limiter is needed. The drawback is that until you have 100 commands you won't know the rate. In practice though, once the rate is high enough to be of interest, this simple solution leads to an automatic check with minimal overhead. + +In the end I actually removed the batch-sending component completely and instead added command DoS protection higher up on the Portal side. The Command-input is now rate limited using the same count-until-limit mechanism. Seems to work fine. People have no artificial slowdowns anymore and the DoS limiter will only kick in at loads that are actually relevant. And so all was again well in Evennia world. \ No newline at end of file diff --git a/devblog/source/2015-09-29-Evennia-on-Podcast-init.md b/devblog/source/2015-09-29-Evennia-on-Podcast-init.md new file mode 100644 index 0000000000..05dda540cd --- /dev/null +++ b/devblog/source/2015-09-29-Evennia-on-Podcast-init.md @@ -0,0 +1,17 @@ +title: Evennia on `podcast.__init__` +copyrights: Logo ©`Podcast.__init__` + +--- + +[![](https://lh6.googleusercontent.com/proxy/vHgHjWEhvYzINt2GwdHGR_s1jI4YncKvh3lIVRCM93oiRAcG4yBpMSIBidwGJ2-kNKprClzeyJMDHM84-QeMJxOyYTHcP_U8GA=s0-d)](http://pythonpodcast.com/images/podcast_init_logo.png) + +So a few weeks back I was contacted by the general Python podcast _Podcast.__init___. They had stumbled on Evennia and were intrigued about MUD development still going on - and in Python even!  + +As it turned out at least one of the two hosts were an old-school MU* gamer and between the two of them they had plenty of good questions both on multiplayer text games in general and about Evennia technology in particular.  + +You can listen to my accent via one of the links below: + +- Google+: [https://plus.google.com/+Podcastinit-the-python-podcast/posts/Vu9dPVjsR9h](https://plus.google.com/+Podcastinit-the-python-podcast/posts/Vu9dPVjsR9h) +- Show Notes: [http://pythonpodcast.com/griatch_evennia.html](http://pythonpodcast.com/griatch_evennia.html) +- Patreon Post: [https://www.patreon.com/posts/3448526](https://www.patreon.com/posts/3448526) +- Twitter Announcement: [https://twitter.com/Podcast__init__/status/648703365019529216](https://twitter.com/Podcast__init__/status/648703365019529216) \ No newline at end of file diff --git a/devblog/source/2015-10-02-Emoting-System.md b/devblog/source/2015-10-02-Emoting-System.md new file mode 100644 index 0000000000..a79448833f --- /dev/null +++ b/devblog/source/2015-10-02-Emoting-System.md @@ -0,0 +1,29 @@ +title: Emoting Systems, or how to chat up a girl +copyrights: Image: ©Griatch [deviantart](https://deviantart.com/griatch-art) + +[![](https://lh6.googleusercontent.com/proxy/l_2MjemzGRtxOqicx4EXXRT63jmryGr7Ml_wvtSSDGg8Rc-oVzKL-_Jxy9f4gvkzROS3eE8jR3R9I2iguw0Cki4oc4reqwi607HGigS6iSO8BxHuZ53k70NUI0ZfWTpXoCqBm-Wr0LakUjfQQxIJIehaRGbvrvU=s0-d)](http://pre03.deviantart.net/54d0/th/pre/i/2012/114/c/8/what__s_yer_name__by_griatch_art-d4xfokb.jpg) + +A few days ago I pushed an emoting contribution to Evennia. A "contrib" is an optional plugin system that is not part of core Evennia but is meant to be easily picked up and used in people's own designs. + +If you are not familiar with what an emoting system does, it is a way to decribe the actions of a character in the game. The simplest form of emote is a single command (like the command **dance** leading to some canned response, or in the case of a graphical game, a dance animation). This contribution offers a more sophisticated system though, allowing input like the following: + +emote /me smiles at /cheerful as he sits at her table. "Hello!" he says. + +Now, this includes /keywords that relate to the objects in the room. So assuming there is **_a very cheerful girl_** in the room, this string will come out as  + +Griatch smiles at _**a very cheerful girl**_ as he sits at her table. **"Hello!"** he says. + +But she will actually see only my outward appearance (the short description) since she doesn't know me. So the cheerful girl (let's say her name is Sandra) would for example see + +_**A man in flowing robes**_ smiles at _**Sandra**_ as he sits at her table. **"Hello!"** he says. + +The emoting system has the following features:  + + +- Short description replacement in emotes and in searches, as seen above. This means that you can do **look cute** and the system will know what you want to look at (in vanilla Evennia you'd need to use **look Sandra**). +- Multi-word searching and disambiguation. If there is **a cute girl** and **a cute puppy** both in the same room, your referencing of /cute will  give an error listing the alternatives. You can then either include more words to make your reference unique or use an index (1-cute, 2-cute) to make it clear who you mean. This mimics normal object-key disambiguation in Evennia. +- Recognition. You can assign your own aliases to people. If Sandra introduces herself you could assign her the name **Sandra** and henceforth be able to reference her as such and see that name appear. But you could also name her **The girl calling herself Sandra** if you didn't believe that's her actual name. +- Languages. Everything within double-quotes is parsed as spoken language (like the Hello! above). By using writing this as **(elvish)"Hello!"**, this could be spoken in another language and those who don't speak elvish would receive an obfuscated string. +- Masking. A person wearing a mask can force people's recognition replacement to deactivate so that they are not recognized anymore. + +The emoting contrib comes as two files in evennia/contrib/: rpsystem.py and rplanguage.py. To use them fully, make your Characters and Rooms inherit from the supplied classes and/or add the new commands to the Character command set. Enjoy! \ No newline at end of file diff --git a/devblog/source/2015-10-11-Illustrations-and-soaps.md b/devblog/source/2015-10-11-Illustrations-and-soaps.md new file mode 100644 index 0000000000..1b8b1b31b3 --- /dev/null +++ b/devblog/source/2015-10-11-Illustrations-and-soaps.md @@ -0,0 +1,13 @@ +[![](https://lh6.googleusercontent.com/proxy/akVd5ALnso-4giJqE1OAHXgxuIZeAx2IYQC-o_zAarsIu1ge2XQkPSQO_nKFz_dX6P5EreO39-HileXV7UmobyAsfE-MlDBUy2J5c1dx4vb5hUjgR6WXzAvOcmrZ20NXVGR2ivvcqKA=s0-d)](http://optionalrealities.com/wp-content/uploads/2015/10/Evennia_illustrated_fig2.png) + +I recently made an article presenting the infrastructure and main coding features of Evennia using a series of nifty diagrams. + +It can hopefully help newcomers get a feeling for the grand structure of an Evennia server/game a little easier. It's published as a feature on [Optional Realities](http://optionalrealities.com/) and you can find the article here: [Evennia Illustrated](http://optionalrealities.com/evennia-illustrated/). + +I also recently joined [MU Soapbox](http://musoapbox.net/topic/620/evennia-a-python-based-mu-server), a forum predominantly discussing MUSH games, to answer some technical questions on Evennia. Unsurprisingly, some (although so far surprisingly few) MUSHers express concern about Evennia's explicit lack of softcode (that is, the ability for players to use a safe in-game language to code their game rather than to use external Python modules). Their argument is sound:  They are used to in-game coding as a way to easily express creatitivy and easily garner help from players. + +Evennia's stance here clash a bit with those opinions: Our philosophy is that our command system is powerful enough to offer players any complexity of build commands they want. The design/coding of the game itself should be done using proper coding IDEs and modern version control tools. + +There is no denying that compared to a softcode-coded game, a player-level contributor to an Evennia game needs some extra tools to create and contribute code over version control. The admin also needs to check such contributions for malicious code before merging it into their running game. Now, these are differences I actually consider _advantages_ from a code-quality perspective. And for finding help, people on average are more likely to know Python than a custom softcode language. But here opinions differ and in a given game community those language adoption numbers can be severely skewed. + +So far, the MUSHers that have adopted Evennia seems to have done so very much to _get away_ from softcode. It will be interesting to see if things like Kelketek's in-development [Group building Evennia contrib](https://github.com/evennia/evennia/pull/711) will be stirring some interest from those on the fence, or if coding their entire game in softcode is indeed an irreplaceable source of their gaming fun. \ No newline at end of file diff --git a/devblog/source/2015-11-12-MIT-uses-Evennia.md b/devblog/source/2015-11-12-MIT-uses-Evennia.md new file mode 100644 index 0000000000..d8a9034fd6 --- /dev/null +++ b/devblog/source/2015-11-12-MIT-uses-Evennia.md @@ -0,0 +1,50 @@ +title: MIT uses Evennia! +copyrights: Image: MIT news + +--- + +[![](https://2.bp.blogspot.com/-FPg1PS8At3k/VkR7vqSz1LI/AAAAAAAAEco/bKSpvhDxLp0/s320/MIT-Language-Games_0.jpg)](https://2.bp.blogspot.com/-FPg1PS8At3k/VkR7vqSz1LI/AAAAAAAAEco/bKSpvhDxLp0/s1600/MIT-Language-Games_0.jpg) + +[](https://www.blogger.com/)Evennia was recently used as a test bed to train an AI system to attempt to play a MUD as a human would - by only reading and understanding the text on the screen. + +Researchers at MIT (Massachusetts Institute of Technology) recently presented the paper _Language understanding for Text-based games using Deep reinforcement learning_ [(PDF)](https://arxiv.org/pdf/1506.08941.pdf) at a conference on natural language processing. A summary is in the [MIT press release](http://news.mit.edu/2015/learning-language-playing-computer-games-0924#_msocom_1). + +I was contacted by these fine folks some time ago so I knew they had plans to use Evennia for their research. It's great to see they now have an article out on it! Evennia devs are also mentioned in the acknowledgements - so something for the Evennia dev community to be proud of!  + + +### MUDs are tricky + +The main complication for an AI playing a MUD is that the computer has no access to the actual game state but must try to surmise how well it's doing only from the text given (same as a human would). The researchers compare the results from a range of deep-learning neural network algorithm that they train to play. + +To test their AI, the researchers first used Evennia to build a simple training "Home World": a 4-room "house" where the simple goal is to find and eat an apple while refraining to go to sleep. The room descriptions used here were pretty formulaic although not trivial to give a challenge. This they used to train their AI system. + +They then took this trained neural network and applied it to the real challenge, playing the Evennia [Tutorial World](https://github.com/evennia/evennia/wiki/Tutorial%20World%20Introduction). You can yourself try this out in our demo install or by just running a single command when starting Evennia. They call it "Fantasy World" in the article. + +The tutorial world has hand-written descriptions and often describes the exits as part of the room text. The article actually makes a comprehensive analysis of the tutorial world, including the available game states and transitions as well as the number of words and number of commands per state. Interesting stuff in itself. I presume the scientists have modified their copy of the tutorial world to provide better metrics for their analysis. + + +### A bridge too far + +As far as I understand from the article, the AI does understand to use commands with one or two arguments (like _eat apple_ or the _move red-root right_), but they note that actually finding the tomb of the fallen hero (the main quest of the tutorial) is too hard for the AI: + + +> [...]However, this is a complex quest that requires the player to memorize game events and perform high-level planning which are beyond the scope of this current work. + +So instead they evaluate the AI's performance on a more mundane task: Getting across the bridge to the castle. It's not clear to me if the AI actually plays more of the game too or if their test just exposes the AI to the bridge itself. I suspect it _does_ play more due to the examples they use from other rooms; evaluating the bridge-crossing is just a clear-cut metric to use for "success". + +The MIT press release claims that the AI is also scored on how much health/magic it has, but I don't see that mentioned in the article itself (and the tutorial world only has magic if you find the hero's tomb which they claim they cannot do). + +The bridge in Evennia's tutorial world is actually a single "room" that takes multiple steps to cross. At every step the room description changes to describe the progress. Random texts will appear as the bridge sways in the wind and various environmental cues are heard and seen. There is also a small chance of falling off the bridge if one lingers too long on it. + +So although all you really need to do is to walk east repeatedly, I can see why this can be a challenge to a neural network having no mental image of what a bridge is. It can only work off the text it's given at any given time. + +In the paper, the algorithms are evaluated both on their ability to actually cross the bridge and on how optimal their solution was, for example by not issuing invalid commands to the situation. + + +### Beyond the bridge + +The results are that after being trained on the training house setup, the AI _will_ eventually be able to cross the bridge. The particular algorithm proposed also perform slightly better than the comparison ones (and _a lot_ better than simple randomness). + +So from the perspective of the researchers this seems to be a success. Even so, this reinforces the fact that quite some way to go before an AI can *actually* play a real MUD successfully. Using MUDs for this type of research is a good idea though, and I do hope they expand and continue this line work in the future. + +Who knows, maybe the AI will even find that ancient tomb eventually! \ No newline at end of file diff --git a/devblog/templates/post.html b/devblog/templates/post.html index bfc42d796b..8d5e4162a1 100644 --- a/devblog/templates/post.html +++ b/devblog/templates/post.html @@ -6,6 +6,6 @@ {{ post.html }}