Andreas/evennia
1
0
Fork 0
mirror of https://github.com/evennia/evennia.git synced 2026-03-18 13:56:30 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
evennia/docs/1.0-dev/Coding/Profiling.html
Evennia docbuilder action 3165f49b4c Updated HTML docs
2022-02-06 18:34:09 +00:00

389 lines
No EOL
26 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<title>Profiling &#8212; Evennia 1.0-dev documentation</title>
<link rel="stylesheet" href="../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<script src="../_static/language_data.js"></script>
<link rel="shortcut icon" href="../_static/favicon.ico"/>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Quirks" href="Quirks.html" />
<link rel="prev" title="Unit Testing" href="Unit-Testing.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="Quirks.html" title="Quirks"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="Unit-Testing.html" title="Unit Testing"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Evennia 1.0-dev</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="Coding-Overview.html" accesskey="U">Coding and development help</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Profiling</a></li>
</ul>
<div class="develop">develop branch</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section class="tex2jax_ignore mathjax_ignore" id="profiling">
<h1>Profiling<a class="headerlink" href="#profiling" title="Permalink to this headline"></a></h1>
<p><em>This is considered an advanced topic mainly of interest to server developers.</em></p>
<section id="introduction">
<h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline"></a></h2>
<p>Sometimes it can be useful to try to determine just how efficient a particular
piece of code is, or to figure out if one could speed up things more than they
are. There are many ways to test the performance of Python and the running
server.</p>
<p>Before digging into this section, remember Donald Knuth’s
<a class="reference external" href="https://en.wikipedia.org/wiki/Program_optimization#When_to_optimize">words of wisdom</a>:</p>
<blockquote>
<div><p><em>[…]about 97% of the time: Premature optimization is the root of all evil</em>.</p>
</div></blockquote>
<p>That is, don’t start to try to optimize your code until you have actually
identified a need to do so. This means your code must actually be working before
you start to consider optimization. Optimization will also often make your code
more complex and harder to read. Consider readability and maintainability and
you may find that a small gain in speed is just not worth it.</p>
</section>
<section id="simple-timer-tests">
<h2>Simple timer tests<a class="headerlink" href="#simple-timer-tests" title="Permalink to this headline"></a></h2>
<p>Python’s <code class="docutils literal notranslate"><span class="pre">timeit</span></code> module is very good for testing small things. For example, in
order to test if it is faster to use a <code class="docutils literal notranslate"><span class="pre">for</span></code> loop or a list comprehension you
could use the following code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span> <span class="kn">import</span> <span class="nn">timeit</span>
<span class="c1"># Time to do 1000000 for loops</span>
<span class="n">timeit</span><span class="o">.</span><span class="n">timeit</span><span class="p">(</span><span class="s2">&quot;for i in range(100):</span><span class="se">\n</span><span class="s2"> a.append(i)&quot;</span><span class="p">,</span> <span class="n">setup</span><span class="o">=</span><span class="s2">&quot;a = []&quot;</span><span class="p">)</span>
<span class="o">&lt;&lt;&lt;</span> <span class="mf">10.70982813835144</span>
<span class="c1"># Time to do 1000000 list comprehensions</span>
<span class="n">timeit</span><span class="o">.</span><span class="n">timeit</span><span class="p">(</span><span class="s2">&quot;a = [i for i in range(100)]&quot;</span><span class="p">)</span>
<span class="o">&lt;&lt;&lt;</span> <span class="mf">5.358283996582031</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">setup</span></code> keyword is used to set up things that should not be included in the
time measurement, like <code class="docutils literal notranslate"><span class="pre">a</span> <span class="pre">=</span> <span class="pre">[]</span></code> in the first call.</p>
<p>By default the <code class="docutils literal notranslate"><span class="pre">timeit</span></code> function will re-run the given test 1000000 times and
returns the <em>total time</em> to do so (so <em>not</em> the average per test). A hint is to
not use this default for testing something that includes database writes - for
that you may want to use a lower number of repeats (say 100 or 1000) using the
<code class="docutils literal notranslate"><span class="pre">number=100</span></code> keyword.</p>
</section>
<section id="using-cprofile">
<h2>Using cProfile<a class="headerlink" href="#using-cprofile" title="Permalink to this headline"></a></h2>
<p>Python comes with its own profiler, named cProfile (this is for cPython, no
tests have been done with <code class="docutils literal notranslate"><span class="pre">pypy</span></code> at this point). Due to the way Evennia’s
processes are handled, there is no point in using the normal way to start the
profiler (<code class="docutils literal notranslate"><span class="pre">python</span> <span class="pre">-m</span> <span class="pre">cProfile</span> <span class="pre">evennia.py</span></code>). Instead you start the profiler
through the launcher:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>evennia --profiler start
</pre></div>
</div>
<p>This will start Evennia with the Server component running (in daemon mode) under
cProfile. You could instead try <code class="docutils literal notranslate"><span class="pre">--profile</span></code> with the <code class="docutils literal notranslate"><span class="pre">portal</span></code> argument to
profile the Portal (you would then need to
<a class="reference internal" href="../Setup/Start-Stop-Reload.html"><span class="doc std std-doc">start the Server separately</span></a>).</p>
<p>Please note that while the profiler is running, your process will use a lot more
memory than usual. Memory usage is even likely to climb over time. So don’t
leave it running perpetually but monitor it carefully (for example using the
<code class="docutils literal notranslate"><span class="pre">top</span></code> command on Linux or the Task Manager’s memory display on Windows).</p>
<p>Once you have run the server for a while, you need to stop it so the profiler
can give its report. Do <em>not</em> kill the program from your task manager or by
sending it a kill signal - this will most likely also mess with the profiler.
Instead either use <code class="docutils literal notranslate"><span class="pre">evennia.py</span> <span class="pre">stop</span></code> or (which may be even better), use
<code class="docutils literal notranslate"><span class="pre">&#64;shutdown</span></code> from inside the game.</p>
<p>Once the server has fully shut down (this may be a lot slower than usual) you
will find that profiler has created a new file <code class="docutils literal notranslate"><span class="pre">mygame/server/logs/server.prof</span></code>.</p>
<section id="analyzing-the-profile">
<h3>Analyzing the profile<a class="headerlink" href="#analyzing-the-profile" title="Permalink to this headline"></a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">server.prof</span></code> file is a binary file. There are many ways to analyze and
display its contents, all of which has only been tested in Linux (If you are a
Windows/Mac user, let us know what works).</p>
<p>You can look at the contents of the profile file with Python’s in-built <code class="docutils literal notranslate"><span class="pre">pstats</span></code>
module in the evennia shell (it’s recommended you install <code class="docutils literal notranslate"><span class="pre">ipython</span></code> with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">ipython</span></code> in your virtualenv first, for prettier output):</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>evennia shell
</pre></div>
</div>
<p>Then in the shell</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">pstats</span>
<span class="kn">from</span> <span class="nn">pstats</span> <span class="kn">import</span> <span class="n">SortKey</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">pstats</span><span class="o">.</span><span class="n">Stats</span><span class="p">(</span><span class="s1">&#39;server/log/server.prof&#39;</span><span class="p">)</span>
<span class="n">p</span><span class="o">.</span><span class="n">strip_dirs</span><span class="p">()</span><span class="o">.</span><span class="n">sort_stats</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">print_stats</span><span class="p">()</span>
</pre></div>
</div>
<p>See the
<a class="reference external" href="https://docs.python.org/3/library/profile.html#instant-user-s-manual">Python profiling documentation</a>
for more information.</p>
<p>You can also visualize the data in various ways.</p>
<ul class="simple">
<li><p><a class="reference external" href="https://pypi.org/project/RunSnakeRun/">Runsnake</a> visualizes the profile to
give a good overview. Install with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">runsnakerun</span></code>. Note that this
may require a C compiler and be quite slow to install.</p></li>
<li><p>For more detailed listing of usage time, you can use
<a class="reference external" href="http://kcachegrind.sourceforge.net/html/Home.html">KCachegrind</a>. To make
KCachegrind work with Python profiles you also need the wrapper script
<a class="reference external" href="https://pypi.python.org/pypi/pyprof2calltree/">pyprof2calltree</a>. You can get
<code class="docutils literal notranslate"><span class="pre">pyprof2calltree</span></code> via <code class="docutils literal notranslate"><span class="pre">pip</span></code> whereas KCacheGrind is something you need to get
via your package manager or their homepage.</p></li>
</ul>
<p>How to analyze and interpret profiling data is not a trivial issue and depends
on what you are profiling for. Evennia being an asynchronous server can also
confuse profiling. Ask on the mailing list if you need help and be ready to be
able to supply your <code class="docutils literal notranslate"><span class="pre">server.prof</span></code> file for comparison, along with the exact
conditions under which it was obtained.</p>
</section>
</section>
<section id="the-dummyrunner">
<h2>The Dummyrunner<a class="headerlink" href="#the-dummyrunner" title="Permalink to this headline"></a></h2>
<p>It is difficult to test “actual” game performance without having players in your
game. For this reason Evennia comes with the <em>Dummyrunner</em> system. The
Dummyrunner is a stress-testing system: a separate program that logs into your
game with simulated players (aka “bots” or “dummies”). Once connected, these
dummies will semi-randomly perform various tasks from a list of possible
actions. Use <code class="docutils literal notranslate"><span class="pre">Ctrl-C</span></code> to stop the Dummyrunner.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>You should not run the Dummyrunner on a production database. It
will spawn many objects and also needs to run with general permissions.
</pre></div>
</div>
<p>This is the recommended process for using the dummy runner:</p>
</div>
<ol>
<li><p>Stop your server completely with <code class="docutils literal notranslate"><span class="pre">evennia</span> <span class="pre">stop</span></code>.</p></li>
<li><p>At <em>the end</em> of your <code class="docutils literal notranslate"><span class="pre">mygame/server/conf.settings.py</span></code> file, add the line</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span> from evennia.server.profiling.settings_mixin import *
</pre></div>
</div>
<p>This will override your settings and disable Evennia’s rate limiters and
DoS-protections, which would otherwise block mass-connecting clients from
one IP. Notably, it will also change to a different (faster) password hasher.</p>
</li>
<li><p>(recommended): Build a new database. If you use default Sqlite3 and want to
keep your existing database, just rename <code class="docutils literal notranslate"><span class="pre">mygame/server/evennia.db3</span></code> to
<code class="docutils literal notranslate"><span class="pre">mygame/server/evennia.db3_backup</span></code> and run <code class="docutils literal notranslate"><span class="pre">evennia</span> <span class="pre">migrate</span></code> and <code class="docutils literal notranslate"><span class="pre">evennia</span> <span class="pre">start</span></code> to create a new superuser as usual.</p></li>
<li><p>(recommended) Log into the game as your superuser. This is just so you
can manually check response. If you kept an old database, you will <em>not</em>
be able to connect with an <em>existing</em> user since the password hasher changed!</p></li>
<li><p>Start the dummyrunner with 10 dummy users from the terminal with</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span> evennia --dummyrunner 10
</pre></div>
</div>
<p>Use <code class="docutils literal notranslate"><span class="pre">Ctrl-C</span></code> (or <code class="docutils literal notranslate"><span class="pre">Cmd-C</span></code>) to stop it.</p>
</li>
</ol>
<p>If you want to see what the dummies are actually doing you can run with a single
dummy:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>evennia --dummyrunner 1
</pre></div>
</div>
<p>The inputs/outputs from the dummy will then be printed. By default the runner
uses the ‘looker’ profile, which just logs in and sends the ‘look’ command
over and over. To change the settings, copy the file
<code class="docutils literal notranslate"><span class="pre">evennia/server/profiling/dummyrunner_settings.py</span></code> to your <code class="docutils literal notranslate"><span class="pre">mygame/server/conf/</span></code>
directory, then add this line to your settings file to use it in the new
location:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>DUMMYRUNNER_SETTINGS_MODULE = &quot;server/conf/dummyrunner_settings.py&quot;
</pre></div>
</div>
<p>The dummyrunner settings file is a python code module in its own right - it
defines the actions available to the dummies. These are just tuples of command
strings (like “look here”) for the dummy to send to the server along with a
probability of them happening. The dummyrunner looks for a global variable
<code class="docutils literal notranslate"><span class="pre">ACTIONS</span></code>, a list of tuples, where the first two elements define the
commands for logging in/out of the server.</p>
<p>Below is a simplified minimal setup (the default settings file adds a lot more
functionality and info):</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># minimal dummyrunner setup file</span>
<span class="c1"># Time between each dummyrunner &quot;tick&quot;, in seconds. Each dummy will be called</span>
<span class="c1"># with this frequency.</span>
<span class="n">TIMESTEP</span> <span class="o">=</span> <span class="mi">1</span>
<span class="c1"># Chance of a dummy actually performing an action on a given tick. This</span>
<span class="c1"># spreads out usage randomly, like it would be in reality.</span>
<span class="n">CHANCE_OF_ACTION</span> <span class="o">=</span> <span class="mf">0.5</span>
<span class="c1"># Chance of a currently unlogged-in dummy performing its login action every</span>
<span class="c1"># tick. This emulates not all accounts logging in at exactly the same time.</span>
<span class="n">CHANCE_OF_LOGIN</span> <span class="o">=</span> <span class="mf">0.01</span>
<span class="c1"># Which telnet port to connect to. If set to None, uses the first default</span>
<span class="c1"># telnet port of the running server.</span>
<span class="n">TELNET_PORT</span> <span class="o">=</span> <span class="kc">None</span>
<span class="c1"># actions</span>
<span class="k">def</span> <span class="nf">c_login</span><span class="p">(</span><span class="n">client</span><span class="p">):</span>
<span class="n">name</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;Character-</span><span class="si">{</span><span class="n">client</span><span class="o">.</span><span class="n">gid</span><span class="si">}</span><span class="s2">&quot;</span>
<span class="n">pwd</span> <span class="o">=</span> <span class="sa">f</span><span class="s2">&quot;23fwsf23sdfw23wef23&quot;</span>
<span class="k">return</span> <span class="p">(</span>
<span class="sa">f</span><span class="s2">&quot;create </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">pwd</span><span class="si">}</span><span class="s2">&quot;</span>
<span class="sa">f</span><span class="s2">&quot;connect </span><span class="si">{</span><span class="n">name</span><span class="si">}</span><span class="s2"> </span><span class="si">{</span><span class="n">pwd</span><span class="si">}</span><span class="s2">&quot;</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">c_logout</span><span class="p">(</span><span class="n">client</span><span class="p">):</span>
<span class="k">return</span> <span class="p">(</span><span class="s2">&quot;quit&quot;</span><span class="p">,</span> <span class="p">)</span>
<span class="k">def</span> <span class="nf">c_look</span><span class="p">(</span><span class="n">client</span><span class="p">):</span>
<span class="k">return</span> <span class="p">(</span><span class="s2">&quot;look here&quot;</span><span class="p">,</span> <span class="s2">&quot;look me&quot;</span><span class="p">)</span>
<span class="c1"># this is read by dummyrunner.</span>
<span class="n">ACTIONS</span> <span class="o">=</span> <span class="p">(</span>
<span class="n">c_login</span><span class="p">,</span>
<span class="n">c_logout</span><span class="p">,</span>
<span class="p">(</span><span class="mf">1.0</span><span class="p">,</span> <span class="n">c_look</span><span class="p">)</span> <span class="c1"># (probability, command-generator)</span>
<span class="p">)</span>
</pre></div>
</div>
<p>At the bottom of the default file are a few default profiles you can test out
by just setting the <code class="docutils literal notranslate"><span class="pre">PROFILE</span></code> variable to one of the options.</p>
<section id="dummyrunner-hints">
<h3>Dummyrunner hints<a class="headerlink" href="#dummyrunner-hints" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><p>Don’t start with too many dummies. The Dummyrunner taxes the server much more
than ‘real’ users tend to do. Start with 10-100 to begin with.</p></li>
<li><p>Stress-testing can be fun, but also consider what a ‘realistic’ number of
users would be for your game.</p></li>
<li><p>Note in the dummyrunner output how many commands/s are being sent to the
server by all dummies. This is usually a lot higher than what you’d
realistically expect to see from the same number of users.</p></li>
<li><p>The default settings sets up a ‘lag’ measure to measaure the round-about
message time. It updates with an average every 30 seconds. It can be worth to
have this running for a small number of dummies in one terminal before adding
more by starting another dummyrunner in another terminal - the first one will
act as a measure of how lag changes with different loads. Also verify the
lag-times by entering commands manually in-game.</p></li>
<li><p>Check the CPU usage of your server using <code class="docutils literal notranslate"><span class="pre">top/htop</span></code> (linux). In-game, use the
<code class="docutils literal notranslate"><span class="pre">server</span></code> command.</p></li>
<li><p>You can run the server with <code class="docutils literal notranslate"><span class="pre">--profiler</span> <span class="pre">start</span></code> to test it with dummies. Note
that the profiler will itself affect server performance, especially memory
consumption.</p></li>
<li><p>Generally, the dummyrunner system makes for a decent test of general
performance; but it is of course hard to actually mimic human user behavior.
For this, actual real-game testing is required.</p></li>
</ul>
</section>
</section>
</section>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/evennia_logo.png" alt="Logo"/>
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
<p><h3><a href="../index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Profiling</a><ul>
<li><a class="reference internal" href="#introduction">Introduction</a></li>
<li><a class="reference internal" href="#simple-timer-tests">Simple timer tests</a></li>
<li><a class="reference internal" href="#using-cprofile">Using cProfile</a><ul>
<li><a class="reference internal" href="#analyzing-the-profile">Analyzing the profile</a></li>
</ul>
</li>
<li><a class="reference internal" href="#the-dummyrunner">The Dummyrunner</a><ul>
<li><a class="reference internal" href="#dummyrunner-hints">Dummyrunner hints</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="Unit-Testing.html"
title="previous chapter">Unit Testing</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="Quirks.html"
title="next chapter">Quirks</a></p>
<div role="note" aria-label="source link">
<!--h3>This Page</h3-->
<ul class="this-page-menu">
<li><a href="../_sources/Coding/Profiling.md.txt"
rel="nofollow">Show Page Source</a></li>
</ul>
</div><h3>Links</h3>
<ul>
<li><a href="https://www.evennia.com">Home page</a> </li>
<li><a href="https://github.com/evennia/evennia">Evennia Github</a> </li>
<li><a href="http://games.evennia.com">Game Index</a> </li>
<li>
<a href="https://discord.gg/AJJpcRUhtF">Discord</a> -
<a href="https://github.com/evennia/evennia/discussions">Discussions</a> -
<a href="https://evennia.blogspot.com/">Blog</a>
</li>
</ul>
<h3>Versions</h3>
<ul>
<li><a href="Profiling.html">1.0-dev (develop branch)</a></li>
<li><a href="../../0.9.5/index.html">0.9.5 (v0.9.5 branch)</a></li>
</ul>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="Quirks.html" title="Quirks"
>next</a> |</li>
<li class="right" >
<a href="Unit-Testing.html" title="Unit Testing"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Evennia 1.0-dev</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="Coding-Overview.html" >Coding and development help</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Profiling</a></li>
</ul>
<div class="develop">develop branch</div>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2020, The Evennia developer community.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink