<pclass="last">You are reading an old version of the Evennia documentation. <ahref="https://www.evennia.com/docs/latest/index.html">The latest version is here</a></p>.
<li><aclass="reference internal"href="#optional-evennia-settings-if-not-given-these-defaults-are-used">Optional Evennia settings (if not given, these defaults are used)</a></li>
<spanid="evennia-contrib-rpg-llm-llm-client"></span><h1>evennia.contrib.rpg.llm.llm_client<aclass="headerlink"href="#module-evennia.contrib.rpg.llm.llm_client"title="Permalink to this headline">¶</a></h1>
<p>LLM (Large Language Model) client, for communicating with an LLM backend. This can be used
for generating texts for AI npcs, or for fine-tuning the LLM on a given prompt.</p>
<p>Note that running a LLM locally requires a lot of power, and ideally a powerful GPU. Testing
this with CPU mode on a beefy laptop, still takes some 4s just on a very small model.</p>
<p>The server defaults to output suitable for a local server
<aclass="reference external"href="https://github.com/oobabooga/text-generation-webui">https://github.com/oobabooga/text-generation-webui</a>, but could be used for other LLM servers too.</p>
<p>See the LLM instructions on that page for how to set up the server. You’ll also need
a model file - there are thousands to try out on <aclass="reference external"href="https://huggingface.co/models">https://huggingface.co/models</a> (you want Text
<h2>Optional Evennia settings (if not given, these defaults are used)<aclass="headerlink"href="#optional-evennia-settings-if-not-given-these-defaults-are-used"title="Permalink to this headline">¶</a></h2>
<emclass="property">class </em><codeclass="sig-prename descclassname">evennia.contrib.rpg.llm.llm_client.</code><codeclass="sig-name descname">StringProducer</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">body</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#StringProducer"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.StringProducer"title="Permalink to this definition">¶</a></dt>
<codeclass="sig-name descname">__init__</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">body</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#StringProducer.__init__"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.StringProducer.__init__"title="Permalink to this definition">¶</a></dt>
<dd><p>Initialize self. See help(type(self)) for accurate signature.</p>
<codeclass="sig-name descname">startProducing</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">consumer</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#StringProducer.startProducing"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.StringProducer.startProducing"title="Permalink to this definition">¶</a></dt>
<codeclass="sig-name descname">pauseProducing</code><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#StringProducer.pauseProducing"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.StringProducer.pauseProducing"title="Permalink to this definition">¶</a></dt>
<codeclass="sig-name descname">stopProducing</code><spanclass="sig-paren">(</span><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#StringProducer.stopProducing"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.StringProducer.stopProducing"title="Permalink to this definition">¶</a></dt>
<emclass="property">class </em><codeclass="sig-prename descclassname">evennia.contrib.rpg.llm.llm_client.</code><codeclass="sig-name descname">SimpleResponseReceiver</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">status_code</span></em>, <emclass="sig-param"><spanclass="n">d</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#SimpleResponseReceiver"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.SimpleResponseReceiver"title="Permalink to this definition">¶</a></dt>
<codeclass="sig-name descname">__init__</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">status_code</span></em>, <emclass="sig-param"><spanclass="n">d</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#SimpleResponseReceiver.__init__"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.SimpleResponseReceiver.__init__"title="Permalink to this definition">¶</a></dt>
<dd><p>Initialize self. See help(type(self)) for accurate signature.</p>
<codeclass="sig-name descname">dataReceived</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">data</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#SimpleResponseReceiver.dataReceived"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.SimpleResponseReceiver.dataReceived"title="Permalink to this definition">¶</a></dt>
<dd><p>Called whenever data is received.</p>
<p>Use this method to translate to a higher-level message. Usually, some
callback will be made upon the receipt of each complete protocol
message.</p>
<dlclass="simple">
<dt>@param data: a string of indeterminate length. Please keep in mind</dt><dd><p>that you will probably need to buffer some data, as partial
(or multiple) protocol messages may be received! I recommend
that unit tests for protocols call through to this method with
differing chunk sizes, down to one byte at a time.</p>
<codeclass="sig-name descname">connectionLost</code><spanclass="sig-paren">(</span><emclass="sig-param">reason=<twisted.python.failure.Failure twisted.internet.error.ConnectionDone: Connection was closed cleanly.></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#SimpleResponseReceiver.connectionLost"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.SimpleResponseReceiver.connectionLost"title="Permalink to this definition">¶</a></dt>
<dd><p>Called when the connection is shut down.</p>
<p>Clear any circular references here, and any external references
to this Protocol. The connection has been closed.</p>
<emclass="property">class </em><codeclass="sig-prename descclassname">evennia.contrib.rpg.llm.llm_client.</code><codeclass="sig-name descname">QuietHTTP11ClientFactory</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">quiescentCallback</span></em>, <emclass="sig-param"><spanclass="n">metadata</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#QuietHTTP11ClientFactory"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.QuietHTTP11ClientFactory"title="Permalink to this definition">¶</a></dt>
<codeclass="sig-name descname">noisy</code><emclass="property"> = False</em><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.QuietHTTP11ClientFactory.noisy"title="Permalink to this definition">¶</a></dt>
<emclass="property">class </em><codeclass="sig-prename descclassname">evennia.contrib.rpg.llm.llm_client.</code><codeclass="sig-name descname">LLMClient</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">on_bad_request</span><spanclass="o">=</span><spanclass="default_value">None</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#LLMClient"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.LLMClient"title="Permalink to this definition">¶</a></dt>
<codeclass="sig-name descname">__init__</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">on_bad_request</span><spanclass="o">=</span><spanclass="default_value">None</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#LLMClient.__init__"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.LLMClient.__init__"title="Permalink to this definition">¶</a></dt>
<dd><p>Initialize self. See help(type(self)) for accurate signature.</p>
<codeclass="sig-name descname">get_response</code><spanclass="sig-paren">(</span><emclass="sig-param"><spanclass="n">prompt</span></em><spanclass="sig-paren">)</span><aclass="reference internal"href="../_modules/evennia/contrib/rpg/llm/llm_client.html#LLMClient.get_response"><spanclass="viewcode-link">[source]</span></a><aclass="headerlink"href="#evennia.contrib.rpg.llm.llm_client.LLMClient.get_response"title="Permalink to this definition">¶</a></dt>
<dd><p>Get a response from the LLM server for the given npc.</p>
<dlclass="field-list simple">
<dtclass="field-odd">Parameters</dt>
<ddclass="field-odd"><p><strong>prompt</strong> (<em>str</em><em> or </em><em>list</em>) – The prompt to send to the LLM server. If a list,
this is assumed to be the chat history so far, and will be added to the
prompt in a way suitable for the api.</p>
</dd>
<dtclass="field-even">Returns</dt>
<ddclass="field-even"><p><p><em>str</em>–</p>
<dlclass="simple">
<dt>The generated text response. Will return an empty string</dt><dd><p>if there is an issue with the server, in which case the
the caller is expected to handle this gracefully.</p>
<pclass="last">You are reading an old version of the Evennia documentation. <ahref="https://www.evennia.com/docs/latest/index.html">The latest version is here</a></p>.
</div>
<divclass="footer"role="contentinfo">
© Copyright 2023, The Evennia developer community.
Created using <ahref="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
