Andreas/evennia
1
0
Fork 0
mirror of https://github.com/evennia/evennia.git synced 2026-03-16 21:06:30 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update dummyrunner docs

Browse source
This commit is contained in:
Griatch 2021-06-02 10:17:04 +02:00
parent 677a34d06e
commit 514cb1b9af
4 changed files with 207 additions and 78 deletions
Download patch file Download diff file Expand all files Collapse all files

1
CHANGELOG.md
View file

@ -65,6 +65,7 @@ Up requirements to Django 3.2+
- Remove `mygame/web/static_overrides` and -`template_overrides`, reorganize website/admin/client/api
into a more consistent structure for overriding. Expanded webpage documentation considerably.
- REST API list-view was shortened (#2401). New CSS/HTML. Add ReDoc for API autodoc page.
- Update and fix dummyrunner with cleaner code and setup.
### Evennia 0.9.5 (2019-2020)

280
docs/source/Coding/Profiling.md
View file

@ -4,24 +4,27 @@
## Introduction
Sometimes it can be useful to try to determine just how efficient a particular piece of code is, or
to figure out if one could speed up things more than they are. There are many ways to test the
performance of Python and the running server.
Sometimes it can be useful to try to determine just how efficient a particular
piece of code is, or to figure out if one could speed up things more than they
are. There are many ways to test the performance of Python and the running
server.
Before digging into this section, remember Donald Knuth's [words of
wisdom](https://en.wikipedia.org/wiki/Program_optimization#When_to_optimize):
Before digging into this section, remember Donald Knuth's
[words of wisdom](https://en.wikipedia.org/wiki/Program_optimization#When_to_optimize):
> *[...]about 97% of the time: Premature optimization is the root of all evil*.
That is, don't start to try to optimize your code until you have actually identified a need to do
so. This means your code must actually be working before you start to consider optimization.
Optimization will also often make your code more complex and harder to read. Consider readability
and maintainability and you may find that a small gain in speed is just not worth it.
That is, don't start to try to optimize your code until you have actually
identified a need to do so. This means your code must actually be working before
you start to consider optimization. Optimization will also often make your code
more complex and harder to read. Consider readability and maintainability and
you may find that a small gain in speed is just not worth it.
## Simple timer tests
Python's `timeit` module is very good for testing small things. For example, in order to test if it
is faster to use a `for` loop or a list comprehension you could use the following code:
Python's `timeit` module is very good for testing small things. For example, in
order to test if it is faster to use a `for` loop or a list comprehension you
could use the following code:
```python
import timeit
@ -33,93 +36,218 @@ is faster to use a `for` loop or a list comprehension you could use the followin
<<< 5.358283996582031
```
The `setup` keyword is used to set up things that should not be included in the time measurement,
like `a = []` in the first call.
The `setup` keyword is used to set up things that should not be included in the
time measurement, like `a = []` in the first call.
By default the `timeit` function will re-run the given test 1000000 times and returns the *total
time* to do so (so *not* the average per test). A hint is to not use this default for testing
something that includes database writes - for that you may want to use a lower number of repeats
(say 100 or 1000) using the `number=100` keyword.
By default the `timeit` function will re-run the given test 1000000 times and
returns the *total time* to do so (so *not* the average per test). A hint is to
not use this default for testing something that includes database writes - for
that you may want to use a lower number of repeats (say 100 or 1000) using the
`number=100` keyword.
## Using cProfile
Python comes with its own profiler, named cProfile (this is for cPython, no tests have been done
with `pypy` at this point). Due to the way Evennia's processes are handled, there is no point in
using the normal way to start the profiler (`python -m cProfile evennia.py`). Instead you start the
profiler through the launcher:
Python comes with its own profiler, named cProfile (this is for cPython, no
tests have been done with `pypy` at this point). Due to the way Evennia's
processes are handled, there is no point in using the normal way to start the
profiler (`python -m cProfile evennia.py`). Instead you start the profiler
through the launcher:
evennia --profiler start
This will start Evennia with the Server component running (in daemon mode) under cProfile. You could
instead try `--profile` with the `portal` argument to profile the Portal (you would then need to
This will start Evennia with the Server component running (in daemon mode) under
cProfile. You could instead try `--profile` with the `portal` argument to
profile the Portal (you would then need to
[start the Server separately](../Setup/Start-Stop-Reload)).
Please note that while the profiler is running, your process will use a lot more memory than usual.
Memory usage is even likely to climb over time. So don't leave it running perpetually but monitor it
carefully (for example using the `top` command on Linux or the Task Manager's memory display on
Windows).
Please note that while the profiler is running, your process will use a lot more
memory than usual. Memory usage is even likely to climb over time. So don't
leave it running perpetually but monitor it carefully (for example using the
`top` command on Linux or the Task Manager's memory display on Windows).
Once you have run the server for a while, you need to stop it so the profiler can give its report.
Do *not* kill the program from your task manager or by sending it a kill signal - this will most
likely also mess with the profiler. Instead either use `evennia.py stop` or (which may be even
better), use `@shutdown` from inside the game.
Once you have run the server for a while, you need to stop it so the profiler
can give its report. Do *not* kill the program from your task manager or by
sending it a kill signal - this will most likely also mess with the profiler.
Instead either use `evennia.py stop` or (which may be even better), use
`@shutdown` from inside the game.
Once the server has fully shut down (this may be a lot slower than usual) you will find that
profiler has created a new file `mygame/server/logs/server.prof`.
Once the server has fully shut down (this may be a lot slower than usual) you
will find that profiler has created a new file `mygame/server/logs/server.prof`.
## Analyzing the profile
### Analyzing the profile
The `server.prof` file is a binary file. There are many ways to analyze and display its contents,
all of which has only been tested in Linux (If you are a Windows/Mac user, let us know what works).
The `server.prof` file is a binary file. There are many ways to analyze and
display its contents, all of which has only been tested in Linux (If you are a
Windows/Mac user, let us know what works).
We recommend the
[Runsnake](http://www.vrplumber.com/programming/runsnakerun/) visualizer to see the processor usage
of different processes in a graphical form. For more detailed listing of usage time, you can use
[KCachegrind](http://kcachegrind.sourceforge.net/html/Home.html). To make KCachegrind work with
Python profiles you also need the wrapper script
[pyprof2calltree](https://pypi.python.org/pypi/pyprof2calltree/). You can get pyprof2calltree via
`pip` whereas KCacheGrind is something you need to get via your package manager or their homepage.
You can look at the contents of the profile file with Python's in-built `pstats`
module in the evennia shell (it's recommended you install `ipython` with `pip
install ipython` in your virtualenv first, for prettier output):
How to analyze and interpret profiling data is not a trivial issue and depends on what you are
profiling for. Evennia being an asynchronous server can also confuse profiling. Ask on the mailing
list if you need help and be ready to be able to supply your `server.prof` file for comparison,
along with the exact conditions under which it was obtained.
evennia shell
Then in the shell
```python
import pstats
from pstats import SortKey
p = pstats.Stats('server/log/server.prof')
p.strip_dirs().sort_stats(-1).print_stats()
```
See the
[Python profiling documentation](https://docs.python.org/3/library/profile.html#instant-user-s-manual)
for more information.
You can also visualize the data in various ways.
- [Runsnake](https://pypi.org/project/RunSnakeRun/) visualizes the profile to
give a good overview. Install with `pip install runsnakerun`. Note that this
may require a C compiler and be quite slow to install.
- For more detailed listing of usage time, you can use
[KCachegrind](http://kcachegrind.sourceforge.net/html/Home.html). To make
KCachegrind work with Python profiles you also need the wrapper script
[pyprof2calltree](https://pypi.python.org/pypi/pyprof2calltree/). You can get
`pyprof2calltree` via `pip` whereas KCacheGrind is something you need to get
via your package manager or their homepage.
How to analyze and interpret profiling data is not a trivial issue and depends
on what you are profiling for. Evennia being an asynchronous server can also
confuse profiling. Ask on the mailing list if you need help and be ready to be
able to supply your `server.prof` file for comparison, along with the exact
conditions under which it was obtained.
## The Dummyrunner
It is difficult to test "actual" game performance without having players in your game. For this
reason Evennia comes with the *Dummyrunner* system. The Dummyrunner is a stress-testing system: a
separate program that logs into your game with simulated players (aka "bots" or "dummies"). Once
connected these dummies will semi-randomly perform various tasks from a list of possible actions.
Use `Ctrl-C` to stop the Dummyrunner.
It is difficult to test "actual" game performance without having players in your
game. For this reason Evennia comes with the *Dummyrunner* system. The
Dummyrunner is a stress-testing system: a separate program that logs into your
game with simulated players (aka "bots" or "dummies"). Once connected, these
dummies will semi-randomly perform various tasks from a list of possible
actions. Use `Ctrl-C` to stop the Dummyrunner.
> Warning: You should not run the Dummyrunner on a production database. It will spawn many objects
and also needs to run with general permissions.
```warning::
To launch the Dummyrunner, first start your server normally (with or without profiling, as above).
Then start a new terminal/console window and active your virtualenv there too. In the new terminal,
try to connect 10 dummy players:
You should not run the Dummyrunner on a production database. It
will spawn many objects and also needs to run with general permissions.
evennia --dummyrunner 10
The first time you do this you will most likely get a warning from Dummyrunner. It will tell you to
copy an import string to the end of your settings file. Quit the Dummyrunner (`Ctrl-C`) and follow
the instructions. Restart Evennia and try `evennia --dummyrunner 10` again. Make sure to remove that
extra settings line when running a public server.
The actions perform by the dummies is controlled by a settings file. The default Dummyrunner
settings file is `evennia/server/server/profiling/dummyrunner_settings.py` but you shouldn't modify
this directly. Rather create/copy the default file to `mygame/server/conf/` and modify it there. To
make sure to use your file over the default, add the following line to your settings file:
```python
DUMMYRUNNER_SETTINGS_MODULE = "server/conf/dummyrunner_settings.py"
This is the recommended process for using the dummy runner:
```
> Hint: Don't start with too many dummies. The Dummyrunner defaults to taxing the server much more
intensely than an equal number of human players. A good dummy number to start with is 10-100.
1. Stop your server completely with `evennia stop`.
1. At _the end_ of your `mygame/server/conf.settings.py` file, add the line
Once you have the dummyrunner running, stop it with `Ctrl-C`.
from evennia.server.profiling.settings_mixin import *
This will override your settings and disable Evennia's rate limiters and
DoS-protections, which would otherwise block mass-connecting clients from
one IP. Notably, it will also change to a different (faster) password hasher.
1. (recommended): Build a new database. If you use default Sqlite3 and want to
keep your existing database, just rename `mygame/server/evennia.db3` to
`mygame/server/evennia.db3_backup` and run `evennia migrate` and `evennia
start` to create a new superuser as usual.
1. (recommended) Log into the game as your superuser. This is just so you
can manually check response. If you kept an old database, you will _not_
be able to connect with an _existing_ user since the password hasher changed!
1. Start the dummyrunner with 10 dummy users from the terminal with
evennia --dummyrunner 10
Use `Ctrl-C` (or `Cmd-C`) to stop it.
If you want to see what the dummies are actually doing you can run with a single
dummy:
evennia --dummyrunner 1
The inputs/outputs from the dummy will then be printed. By default the runner
uses the 'looker' profile, which just logs in and sends the 'look' command
over and over. To change the settings, copy the file
`evennia/server/profiling/dummyrunner_settings.py` to your `mygame/server/conf/`
directory, then add this line to your settings file to use it in the new
location:
DUMMYRUNNER_SETTINGS_MODULE = "server/conf/dummyrunner_settings.py"
The dummyrunner settings file is a python code module in its own right - it
defines the actions available to the dummies. These are just tuples of command
strings (like "look here") for the dummy to send to the server along with a
probability of them happening. The dummyrunner looks for a global variable
`ACTIONS`, a list of tuples, where the first two elements define the
commands for logging in/out of the server.
Below is a simplified minimal setup (the default settings file adds a lot more
functionality and info):
```python
# minimal dummyrunner setup file
# Time between each dummyrunner "tick", in seconds. Each dummy will be called
# with this frequency.
TIMESTEP = 1
# Chance of a dummy actually performing an action on a given tick. This
# spreads out usage randomly, like it would be in reality.
CHANCE_OF_ACTION = 0.5
# Chance of a currently unlogged-in dummy performing its login action every
# tick. This emulates not all accounts logging in at exactly the same time.
CHANCE_OF_LOGIN = 0.01
# Which telnet port to connect to. If set to None, uses the first default
# telnet port of the running server.
TELNET_PORT = None
# actions
def c_login(client):
name = f"Character-{client.gid}"
pwd = f"23fwsf23sdfw23wef23"
return (
f"create {name} {pwd}"
f"connect {name} {pwd}"
)
def c_logout(client):
return ("quit", )
def c_look(client):
return ("look here", "look me")
# this is read by dummyrunner.
ACTIONS = (
c_login,
c_logout,
(1.0, c_look) # (probability, command-generator)
)
```
At the bottom of the default file are a few default profiles you can test out
by just setting the `PROFILE` variable to one of the options.
### Dummyrunner hints
- Don't start with too many dummies. The Dummyrunner taxes the server much more
than 'real' users tend to do. Start with 10-100 to begin with.
- Stress-testing can be fun, but also consider what a 'realistic' number of
users would be for your game.
- Note in the dummyrunner output how many commands/s are being sent to the
server by all dummies. This is usually a lot higher than what you'd
realistically expect to see from the same number of users.
- The default settings sets up a 'lag' measure to measaure the round-about
message time. It updates with an average every 30 seconds. It can be worth to
have this running for a small number of dummies in one terminal before adding
more by starting another dummyrunner in another terminal - the first one will
act as a measure of how lag changes with different loads. Also verify the
lag-times by entering commands manually in-game.
- Check the CPU usage of your server using `top/htop` (linux). In-game, use the
`server` command.
- You can run the server with `--profiler start` to test it with dummies. Note
that the profiler will itself affect server performance, especially memory
consumption.
- Generally, the dummyrunner system makes for a decent test of general
performance; but it is of course hard to actually mimic human user behavior.
For this, actual real-game testing is required.
Generally, the dummyrunner system makes for a decent test of general performance; but it is of
course hard to actually mimic human user behavior. For this, actual real-game testing is required.

2
docs/source/toc.md
View file

@ -1,5 +1,5 @@
# Toc
- [API root](api/evennia-api.rst)
- [Coding/Coding Introduction](Coding/Coding-Introduction)
- [Coding/Coding Overview](Coding/Coding-Overview)
- [Coding/Continuous Integration](Coding/Continuous-Integration)

2
evennia/server/profiling/dummyrunner_settings.py
View file

@ -258,7 +258,7 @@ def c_measure_lag(client):
# socializing_builder - builds a lot, creates help entries, moves, chat (spammy)
# only_digger - extreme builder that only digs room after room
PROFILE = "normal_player"
PROFILE = "looker"
if PROFILE == 'idler':