Andreas/evennia
1
0
Fork 0
mirror of https://github.com/evennia/evennia.git synced 2026-03-26 17:56:32 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
evennia/docs/1.0-dev/_modules/evennia/utils/dbserialize.html
Griatch 8936980973 Updated HTML docs
2020-10-19 08:58:16 +02:00

845 lines
No EOL
100 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>evennia.utils.dbserialize &#8212; Evennia 1.0-dev documentation</title>
<link rel="stylesheet" href="../../../_static/nature.css" type="text/css" />
<link rel="stylesheet" href="../../../_static/pygments.css" type="text/css" />
<script id="documentation_options" data-url_root="../../../" src="../../../_static/documentation_options.js"></script>
<script src="../../../_static/jquery.js"></script>
<script src="../../../_static/underscore.js"></script>
<script src="../../../_static/doctools.js"></script>
<script src="../../../_static/language_data.js"></script>
<link rel="shortcut icon" href="../../../_static/favicon.ico"/>
<link rel="index" title="Index" href="../../../genindex.html" />
<link rel="search" title="Search" href="../../../search.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../../index.html">Evennia 1.0-dev</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../../index.html" >Module code</a> &#187;</li>
<li class="nav-item nav-item-2"><a href="../../evennia.html" accesskey="U">evennia</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">evennia.utils.dbserialize</a></li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<h1>Source code for evennia.utils.dbserialize</h1><div class="highlight"><pre>
<span></span><span class="sd">&quot;&quot;&quot;</span>
<span class="sd">This module handles serialization of arbitrary python structural data,</span>
<span class="sd">intended primarily to be stored in the database. It also supports</span>
<span class="sd">storing Django model instances (which plain pickle cannot do).</span>
<span class="sd">This serialization is used internally by the server, notably for</span>
<span class="sd">storing data in Attributes and for piping data to process pools.</span>
<span class="sd">The purpose of dbserialize is to handle all forms of data. For</span>
<span class="sd">well-structured non-arbitrary exchange, such as communicating with a</span>
<span class="sd">rich web client, a simpler JSON serialization makes more sense.</span>
<span class="sd">This module also implements the `SaverList`, `SaverDict` and `SaverSet`</span>
<span class="sd">classes. These are iterables that track their position in a nested</span>
<span class="sd">structure and makes sure to send updates up to their root. This is</span>
<span class="sd">used by Attributes - without it, one would not be able to update mutables</span>
<span class="sd">in-situ, e.g `obj.db.mynestedlist[3][5] = 3` would never be saved and</span>
<span class="sd">be out of sync with the database.</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="kn">from</span> <span class="nn">functools</span> <span class="k">import</span> <span class="n">update_wrapper</span>
<span class="kn">from</span> <span class="nn">collections</span> <span class="k">import</span> <span class="n">defaultdict</span><span class="p">,</span> <span class="n">MutableSequence</span><span class="p">,</span> <span class="n">MutableSet</span><span class="p">,</span> <span class="n">MutableMapping</span>
<span class="kn">from</span> <span class="nn">collections</span> <span class="k">import</span> <span class="n">OrderedDict</span><span class="p">,</span> <span class="n">deque</span>
<span class="k">try</span><span class="p">:</span>
<span class="kn">from</span> <span class="nn">pickle</span> <span class="k">import</span> <span class="n">dumps</span><span class="p">,</span> <span class="n">loads</span>
<span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
<span class="kn">from</span> <span class="nn">pickle</span> <span class="k">import</span> <span class="n">dumps</span><span class="p">,</span> <span class="n">loads</span>
<span class="kn">from</span> <span class="nn">django.core.exceptions</span> <span class="k">import</span> <span class="n">ObjectDoesNotExist</span>
<span class="kn">from</span> <span class="nn">django.contrib.contenttypes.models</span> <span class="k">import</span> <span class="n">ContentType</span>
<span class="kn">from</span> <span class="nn">django.utils.safestring</span> <span class="k">import</span> <span class="n">SafeString</span>
<span class="kn">from</span> <span class="nn">evennia.utils.utils</span> <span class="k">import</span> <span class="n">uses_database</span><span class="p">,</span> <span class="n">is_iter</span><span class="p">,</span> <span class="n">to_str</span><span class="p">,</span> <span class="n">to_bytes</span>
<span class="kn">from</span> <span class="nn">evennia.utils</span> <span class="k">import</span> <span class="n">logger</span>
<span class="n">__all__</span> <span class="o">=</span> <span class="p">(</span><span class="s2">&quot;to_pickle&quot;</span><span class="p">,</span> <span class="s2">&quot;from_pickle&quot;</span><span class="p">,</span> <span class="s2">&quot;do_pickle&quot;</span><span class="p">,</span> <span class="s2">&quot;do_unpickle&quot;</span><span class="p">,</span> <span class="s2">&quot;dbserialize&quot;</span><span class="p">,</span> <span class="s2">&quot;dbunserialize&quot;</span><span class="p">)</span>
<span class="n">PICKLE_PROTOCOL</span> <span class="o">=</span> <span class="mi">2</span>
<span class="c1"># message to send if editing an already deleted Attribute in a savermutable</span>
<span class="n">_ERROR_DELETED_ATTR</span> <span class="o">=</span> <span class="p">(</span>
<span class="s2">&quot;</span><span class="si">{cls_name}</span><span class="s2"> </span><span class="si">{obj}</span><span class="s2"> has had its root Attribute deleted. &quot;</span>
<span class="s2">&quot;It must be cast to a </span><span class="si">{non_saver_name}</span><span class="s2"> before it can be modified further.&quot;</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">_get_mysql_db_version</span><span class="p">():</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> This is a helper method for specifically getting the version</span>
<span class="sd"> string of a MySQL database.</span>
<span class="sd"> Returns:</span>
<span class="sd"> mysql_version (str): The currently used mysql database</span>
<span class="sd"> version.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="kn">from</span> <span class="nn">django.db</span> <span class="k">import</span> <span class="n">connection</span>
<span class="n">conn</span> <span class="o">=</span> <span class="n">connection</span><span class="o">.</span><span class="n">cursor</span><span class="p">()</span>
<span class="n">conn</span><span class="o">.</span><span class="n">execute</span><span class="p">(</span><span class="s2">&quot;SELECT VERSION()&quot;</span><span class="p">)</span>
<span class="n">version</span> <span class="o">=</span> <span class="n">conn</span><span class="o">.</span><span class="n">fetchone</span><span class="p">()</span>
<span class="k">return</span> <span class="n">version</span> <span class="ow">and</span> <span class="nb">str</span><span class="p">(</span><span class="n">version</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="ow">or</span> <span class="s2">&quot;&quot;</span>
<span class="c1"># initialization and helpers</span>
<span class="n">_GA</span> <span class="o">=</span> <span class="nb">object</span><span class="o">.</span><span class="fm">__getattribute__</span>
<span class="n">_SA</span> <span class="o">=</span> <span class="nb">object</span><span class="o">.</span><span class="fm">__setattr__</span>
<span class="n">_FROM_MODEL_MAP</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">_TO_MODEL_MAP</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">_IGNORE_DATETIME_MODELS</span> <span class="o">=</span> <span class="kc">None</span>
<span class="n">_SESSION_HANDLER</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">def</span> <span class="nf">_IS_PACKED_DBOBJ</span><span class="p">(</span><span class="n">o</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">o</span><span class="p">,</span> <span class="nb">tuple</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">o</span><span class="p">)</span> <span class="o">==</span> <span class="mi">4</span> <span class="ow">and</span> <span class="n">o</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&quot;__packed_dbobj__&quot;</span>
<span class="k">def</span> <span class="nf">_IS_PACKED_SESSION</span><span class="p">(</span><span class="n">o</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">o</span><span class="p">,</span> <span class="nb">tuple</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">len</span><span class="p">(</span><span class="n">o</span><span class="p">)</span> <span class="o">==</span> <span class="mi">3</span> <span class="ow">and</span> <span class="n">o</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">==</span> <span class="s2">&quot;__packed_session__&quot;</span>
<span class="k">if</span> <span class="n">uses_database</span><span class="p">(</span><span class="s2">&quot;mysql&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="n">_get_mysql_db_version</span><span class="p">()</span> <span class="o">&lt;</span> <span class="s2">&quot;5.6.4&quot;</span><span class="p">:</span>
<span class="c1"># mysql &lt;5.6.4 don&#39;t support millisecond precision</span>
<span class="n">_DATESTRING</span> <span class="o">=</span> <span class="s2">&quot;%Y:%m:</span><span class="si">%d</span><span class="s2">-%H:%M:%S:000000&quot;</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">_DATESTRING</span> <span class="o">=</span> <span class="s2">&quot;%Y:%m:</span><span class="si">%d</span><span class="s2">-%H:%M:%S:</span><span class="si">%f</span><span class="s2">&quot;</span>
<span class="k">def</span> <span class="nf">_TO_DATESTRING</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Creates datestring hash.</span>
<span class="sd"> Args:</span>
<span class="sd"> obj (Object): Database object.</span>
<span class="sd"> Returns:</span>
<span class="sd"> datestring (str): A datestring hash.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="n">_GA</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">&quot;db_date_created&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">_DATESTRING</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
<span class="c1"># this can happen if object is not yet saved - no datestring is then set</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">obj</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
<span class="k">except</span> <span class="ne">AttributeError</span><span class="p">:</span>
<span class="c1"># we have received a None object, for example due to an erroneous save.</span>
<span class="k">return</span> <span class="kc">None</span>
<span class="k">return</span> <span class="n">_GA</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">&quot;db_date_created&quot;</span><span class="p">)</span><span class="o">.</span><span class="n">strftime</span><span class="p">(</span><span class="n">_DATESTRING</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_init_globals</span><span class="p">():</span>
<span class="sd">&quot;&quot;&quot;Lazy importing to avoid circular import issues&quot;&quot;&quot;</span>
<span class="k">global</span> <span class="n">_FROM_MODEL_MAP</span><span class="p">,</span> <span class="n">_TO_MODEL_MAP</span><span class="p">,</span> <span class="n">_SESSION_HANDLER</span><span class="p">,</span> <span class="n">_IGNORE_DATETIME_MODELS</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">_FROM_MODEL_MAP</span><span class="p">:</span>
<span class="n">_FROM_MODEL_MAP</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span>
<span class="n">_FROM_MODEL_MAP</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">dict</span><span class="p">((</span><span class="n">c</span><span class="o">.</span><span class="n">model</span><span class="p">,</span> <span class="n">c</span><span class="o">.</span><span class="n">natural_key</span><span class="p">())</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">ContentType</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">all</span><span class="p">()))</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">_TO_MODEL_MAP</span><span class="p">:</span>
<span class="kn">from</span> <span class="nn">django.conf</span> <span class="k">import</span> <span class="n">settings</span>
<span class="n">_TO_MODEL_MAP</span> <span class="o">=</span> <span class="n">defaultdict</span><span class="p">(</span><span class="nb">str</span><span class="p">)</span>
<span class="n">_TO_MODEL_MAP</span><span class="o">.</span><span class="n">update</span><span class="p">(</span>
<span class="nb">dict</span><span class="p">((</span><span class="n">c</span><span class="o">.</span><span class="n">natural_key</span><span class="p">(),</span> <span class="n">c</span><span class="o">.</span><span class="n">model_class</span><span class="p">())</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="n">ContentType</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">all</span><span class="p">())</span>
<span class="p">)</span>
<span class="n">_IGNORE_DATETIME_MODELS</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">src_key</span><span class="p">,</span> <span class="n">dst_key</span> <span class="ow">in</span> <span class="n">settings</span><span class="o">.</span><span class="n">ATTRIBUTE_STORED_MODEL_RENAME</span><span class="p">:</span>
<span class="n">_TO_MODEL_MAP</span><span class="p">[</span><span class="n">src_key</span><span class="p">]</span> <span class="o">=</span> <span class="n">_TO_MODEL_MAP</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">dst_key</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="n">_IGNORE_DATETIME_MODELS</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">src_key</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">_SESSION_HANDLER</span><span class="p">:</span>
<span class="kn">from</span> <span class="nn">evennia.server.sessionhandler</span> <span class="k">import</span> <span class="n">SESSION_HANDLER</span> <span class="k">as</span> <span class="n">_SESSION_HANDLER</span>
<span class="c1">#</span>
<span class="c1"># SaverList, SaverDict, SaverSet - Attribute-specific helper classes and functions</span>
<span class="c1">#</span>
<span class="k">def</span> <span class="nf">_save</span><span class="p">(</span><span class="n">method</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;method decorator that saves data to Attribute&quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">save_wrapper</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="vm">__doc__</span> <span class="o">=</span> <span class="n">method</span><span class="o">.</span><span class="vm">__doc__</span>
<span class="n">ret</span> <span class="o">=</span> <span class="n">method</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_save_tree</span><span class="p">()</span>
<span class="k">return</span> <span class="n">ret</span>
<span class="k">return</span> <span class="n">update_wrapper</span><span class="p">(</span><span class="n">save_wrapper</span><span class="p">,</span> <span class="n">method</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">_SaverMutable</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Parent class for properly handling of nested mutables in</span>
<span class="sd"> an Attribute. If not used something like</span>
<span class="sd"> obj.db.mylist[1][2] = &quot;test&quot; (allocation to a nested list)</span>
<span class="sd"> will not save the updated value to the database.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;store all properties for tracking the tree&quot;&quot;&quot;</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_parent</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;_parent&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_db_obj</span> <span class="o">=</span> <span class="n">kwargs</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="s2">&quot;_db_obj&quot;</span><span class="p">,</span> <span class="kc">None</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">def</span> <span class="nf">__bool__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Make sure to evaluate as False if empty&quot;&quot;&quot;</span>
<span class="k">return</span> <span class="nb">bool</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_save_tree</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;recursively traverse back up the tree, save when we reach the root&quot;&quot;&quot;</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_parent</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_parent</span><span class="o">.</span><span class="n">_save_tree</span><span class="p">()</span>
<span class="k">elif</span> <span class="bp">self</span><span class="o">.</span><span class="n">_db_obj</span><span class="p">:</span>
<span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_db_obj</span><span class="o">.</span><span class="n">pk</span><span class="p">:</span>
<span class="n">cls_name</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="vm">__name__</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">non_saver_name</span> <span class="o">=</span> <span class="n">cls_name</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s2">&quot;_Saver&quot;</span><span class="p">,</span> <span class="mi">1</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span>
<span class="k">except</span> <span class="ne">IndexError</span><span class="p">:</span>
<span class="n">non_saver_name</span> <span class="o">=</span> <span class="n">cls_name</span>
<span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
<span class="n">_ERROR_DELETED_ATTR</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
<span class="n">cls_name</span><span class="o">=</span><span class="n">cls_name</span><span class="p">,</span> <span class="n">obj</span><span class="o">=</span><span class="bp">self</span><span class="p">,</span> <span class="n">non_saver_name</span><span class="o">=</span><span class="n">non_saver_name</span>
<span class="p">)</span>
<span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_db_obj</span><span class="o">.</span><span class="n">value</span> <span class="o">=</span> <span class="bp">self</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">logger</span><span class="o">.</span><span class="n">log_err</span><span class="p">(</span><span class="s2">&quot;_SaverMutable </span><span class="si">%s</span><span class="s2"> has no root Attribute to save to.&quot;</span> <span class="o">%</span> <span class="bp">self</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">_convert_mutables</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;converts mutables to Saver* variants and assigns ._parent property&quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">process_tree</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="n">parent</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;recursively populate the tree, storing parents&quot;&quot;&quot;</span>
<span class="n">dtype</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">if</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">bool</span><span class="p">,</span> <span class="nb">tuple</span><span class="p">):</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">list</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverList</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">dict</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverDict</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">((</span><span class="n">key</span><span class="p">,</span> <span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">set</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverSet</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">return</span> <span class="n">process_tree</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">__repr__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__repr__</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__len__</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">__iter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__iter__</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__getitem__</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">__eq__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">==</span> <span class="n">other</span>
<span class="k">def</span> <span class="nf">__ne__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">!=</span> <span class="n">other</span>
<span class="k">def</span> <span class="nf">__lt__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">&lt;</span> <span class="n">other</span>
<span class="k">def</span> <span class="nf">__gt__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">&gt;</span> <span class="n">other</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">__setitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__setitem__</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_convert_mutables</span><span class="p">(</span><span class="n">value</span><span class="p">))</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">__delitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__delitem__</span><span class="p">(</span><span class="n">key</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">_SaverList</span><span class="p">(</span><span class="n">_SaverMutable</span><span class="p">,</span> <span class="n">MutableSequence</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> A list that saves itself to an Attribute when updated.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">=</span> <span class="nb">list</span><span class="p">()</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">__iadd__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">otherlist</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__add__</span><span class="p">(</span><span class="n">otherlist</span><span class="p">)</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span>
<span class="k">def</span> <span class="nf">__add__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">otherlist</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="p">)</span> <span class="o">+</span> <span class="n">otherlist</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">insert</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="n">index</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_convert_mutables</span><span class="p">(</span><span class="n">value</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">__eq__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="p">)</span> <span class="o">==</span> <span class="nb">list</span><span class="p">(</span><span class="n">other</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">TypeError</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">False</span>
<span class="k">def</span> <span class="nf">__ne__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">other</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">list</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="p">)</span> <span class="o">!=</span> <span class="nb">list</span><span class="p">(</span><span class="n">other</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">TypeError</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">True</span>
<span class="k">def</span> <span class="nf">index</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="n">value</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">sort</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="p">,</span> <span class="n">key</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">sort</span><span class="p">(</span><span class="n">key</span><span class="o">=</span><span class="n">key</span><span class="p">,</span> <span class="n">reverse</span><span class="o">=</span><span class="n">reverse</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">copy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
<span class="k">class</span> <span class="nc">_SaverDict</span><span class="p">(</span><span class="n">_SaverMutable</span><span class="p">,</span> <span class="n">MutableMapping</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> A dict that stores changes to an Attribute when updated</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">has_key</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="n">key</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span>
<span class="k">class</span> <span class="nc">_SaverSet</span><span class="p">(</span><span class="n">_SaverMutable</span><span class="p">,</span> <span class="n">MutableSet</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> A set that saves to an Attribute when updated</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">=</span> <span class="nb">set</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">__contains__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="fm">__contains__</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_convert_mutables</span><span class="p">(</span><span class="n">value</span><span class="p">))</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">discard</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">discard</span><span class="p">(</span><span class="n">value</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">_SaverOrderedDict</span><span class="p">(</span><span class="n">_SaverMutable</span><span class="p">,</span> <span class="n">MutableMapping</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> An ordereddict that can be saved and operated on.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">=</span> <span class="n">OrderedDict</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">has_key</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">):</span>
<span class="k">return</span> <span class="n">key</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span>
<span class="k">class</span> <span class="nc">_SaverDeque</span><span class="p">(</span><span class="n">_SaverMutable</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> A deque that can be saved and operated on.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span> <span class="o">=</span> <span class="n">deque</span><span class="p">()</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">append</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">appendleft</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">appendleft</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">clear</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">clear</span><span class="p">()</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">extendleft</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">extendleft</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="c1"># maxlen property</span>
<span class="k">def</span> <span class="nf">_getmaxlen</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">maxlen</span>
<span class="k">def</span> <span class="nf">_setmaxlen</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">value</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">maxlen</span> <span class="o">=</span> <span class="n">value</span>
<span class="k">def</span> <span class="nf">_delmaxlen</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">del</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">maxlen</span>
<span class="n">maxlen</span> <span class="o">=</span> <span class="nb">property</span><span class="p">(</span><span class="n">_getmaxlen</span><span class="p">,</span> <span class="n">_setmaxlen</span><span class="p">,</span> <span class="n">_delmaxlen</span><span class="p">)</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">pop</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">popleft</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">popleft</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">reverse</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">reverse</span><span class="p">()</span>
<span class="nd">@_save</span>
<span class="k">def</span> <span class="nf">rotate</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">rotate</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="n">_DESERIALIZE_MAPPING</span> <span class="o">=</span> <span class="p">{</span>
<span class="n">_SaverList</span><span class="o">.</span><span class="vm">__name__</span><span class="p">:</span> <span class="nb">list</span><span class="p">,</span>
<span class="n">_SaverDict</span><span class="o">.</span><span class="vm">__name__</span><span class="p">:</span> <span class="nb">dict</span><span class="p">,</span>
<span class="n">_SaverSet</span><span class="o">.</span><span class="vm">__name__</span><span class="p">:</span> <span class="nb">set</span><span class="p">,</span>
<span class="n">_SaverOrderedDict</span><span class="o">.</span><span class="vm">__name__</span><span class="p">:</span> <span class="n">OrderedDict</span><span class="p">,</span>
<span class="n">_SaverDeque</span><span class="o">.</span><span class="vm">__name__</span><span class="p">:</span> <span class="n">deque</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">def</span> <span class="nf">deserialize</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Make sure to *fully* decouple a structure from the database, by turning all _Saver*-mutables</span>
<span class="sd"> inside it back into their normal Python forms.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">_iter</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="n">typ</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="n">tname</span> <span class="o">=</span> <span class="n">typ</span><span class="o">.</span><span class="vm">__name__</span>
<span class="k">if</span> <span class="n">tname</span> <span class="ow">in</span> <span class="p">(</span><span class="s2">&quot;_SaverDict&quot;</span><span class="p">,</span> <span class="s2">&quot;dict&quot;</span><span class="p">):</span>
<span class="k">return</span> <span class="p">{</span><span class="n">_iter</span><span class="p">(</span><span class="n">key</span><span class="p">):</span> <span class="n">_iter</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">obj</span><span class="o">.</span><span class="n">items</span><span class="p">()}</span>
<span class="k">elif</span> <span class="n">tname</span> <span class="ow">in</span> <span class="n">_DESERIALIZE_MAPPING</span><span class="p">:</span>
<span class="k">return</span> <span class="n">_DESERIALIZE_MAPPING</span><span class="p">[</span><span class="n">tname</span><span class="p">](</span><span class="n">_iter</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">obj</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">is_iter</span><span class="p">(</span><span class="n">obj</span><span class="p">):</span>
<span class="k">return</span> <span class="n">typ</span><span class="p">(</span><span class="n">_iter</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">obj</span><span class="p">)</span>
<span class="k">return</span> <span class="n">obj</span>
<span class="k">return</span> <span class="n">_iter</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span>
<span class="c1">#</span>
<span class="c1"># serialization helpers</span>
<span class="k">def</span> <span class="nf">pack_dbobj</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Check and convert django database objects to an internal representation.</span>
<span class="sd"> Args:</span>
<span class="sd"> item (any): A database entity to pack</span>
<span class="sd"> Returns:</span>
<span class="sd"> packed (any or tuple): Either returns the original input item</span>
<span class="sd"> or the packing tuple `(&quot;__packed_dbobj__&quot;, key, creation_time, id)`.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="n">_init_globals</span><span class="p">()</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">item</span>
<span class="n">natural_key</span> <span class="o">=</span> <span class="n">_FROM_MODEL_MAP</span><span class="p">[</span>
<span class="nb">hasattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">&quot;id&quot;</span><span class="p">)</span>
<span class="ow">and</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">&quot;db_date_created&quot;</span><span class="p">)</span>
<span class="ow">and</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">&quot;__dbclass__&quot;</span><span class="p">)</span>
<span class="ow">and</span> <span class="n">obj</span><span class="o">.</span><span class="n">__dbclass__</span><span class="o">.</span><span class="vm">__name__</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span>
<span class="p">]</span>
<span class="c1"># build the internal representation as a tuple</span>
<span class="c1"># (&quot;__packed_dbobj__&quot;, key, creation_time, id)</span>
<span class="k">return</span> <span class="p">(</span>
<span class="n">natural_key</span>
<span class="ow">and</span> <span class="p">(</span><span class="s2">&quot;__packed_dbobj__&quot;</span><span class="p">,</span> <span class="n">natural_key</span><span class="p">,</span> <span class="n">_TO_DATESTRING</span><span class="p">(</span><span class="n">obj</span><span class="p">),</span> <span class="n">_GA</span><span class="p">(</span><span class="n">obj</span><span class="p">,</span> <span class="s2">&quot;id&quot;</span><span class="p">))</span>
<span class="ow">or</span> <span class="n">item</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">unpack_dbobj</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Check and convert internal representations back to Django database</span>
<span class="sd"> models.</span>
<span class="sd"> Args:</span>
<span class="sd"> item (packed_dbobj): The fact that item is a packed dbobj</span>
<span class="sd"> should be checked before this call.</span>
<span class="sd"> Returns:</span>
<span class="sd"> unpacked (any): Either the original input or converts the</span>
<span class="sd"> internal store back to a database representation (its</span>
<span class="sd"> typeclass is returned if applicable).</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="n">_init_globals</span><span class="p">()</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">obj</span> <span class="o">=</span> <span class="n">item</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="ow">and</span> <span class="n">_TO_MODEL_MAP</span><span class="p">[</span><span class="n">item</span><span class="p">[</span><span class="mi">1</span><span class="p">]]</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="nb">id</span><span class="o">=</span><span class="n">item</span><span class="p">[</span><span class="mi">3</span><span class="p">])</span>
<span class="k">except</span> <span class="n">ObjectDoesNotExist</span><span class="p">:</span>
<span class="k">return</span> <span class="kc">None</span>
<span class="k">except</span> <span class="ne">TypeError</span><span class="p">:</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;pk&quot;</span><span class="p">):</span>
<span class="c1"># this happens if item is already an obj</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">return</span> <span class="kc">None</span>
<span class="k">if</span> <span class="n">item</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="ow">in</span> <span class="n">_IGNORE_DATETIME_MODELS</span><span class="p">:</span>
<span class="c1"># if we are replacing models we ignore the datatime</span>
<span class="k">return</span> <span class="n">obj</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1"># even if we got back a match, check the sanity of the date (some</span>
<span class="c1"># databases may &#39;re-use&#39; the id)</span>
<span class="k">return</span> <span class="n">_TO_DATESTRING</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span> <span class="o">==</span> <span class="n">item</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="ow">and</span> <span class="n">obj</span> <span class="ow">or</span> <span class="kc">None</span>
<span class="k">def</span> <span class="nf">pack_session</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Handle the safe serializion of Sessions objects (these contain</span>
<span class="sd"> hidden references to database objects (accounts, puppets) so they</span>
<span class="sd"> can&#39;t be safely serialized).</span>
<span class="sd"> Args:</span>
<span class="sd"> item (Session)): This item must have all properties of a session</span>
<span class="sd"> before entering this call.</span>
<span class="sd"> Returns:</span>
<span class="sd"> packed (tuple or None): A session-packed tuple on the form</span>
<span class="sd"> `(__packed_session__, sessid, conn_time)`. If this sessid</span>
<span class="sd"> does not match a session in the Session handler, None is returned.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="n">_init_globals</span><span class="p">()</span>
<span class="n">session</span> <span class="o">=</span> <span class="n">_SESSION_HANDLER</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">item</span><span class="o">.</span><span class="n">sessid</span><span class="p">)</span>
<span class="k">if</span> <span class="n">session</span> <span class="ow">and</span> <span class="n">session</span><span class="o">.</span><span class="n">conn_time</span> <span class="o">==</span> <span class="n">item</span><span class="o">.</span><span class="n">conn_time</span><span class="p">:</span>
<span class="c1"># we require connection times to be identical for the Session</span>
<span class="c1"># to be accepted as actually being a session (sessids gets</span>
<span class="c1"># reused all the time).</span>
<span class="k">return</span> <span class="p">(</span>
<span class="n">item</span><span class="o">.</span><span class="n">conn_time</span>
<span class="ow">and</span> <span class="n">item</span><span class="o">.</span><span class="n">sessid</span>
<span class="ow">and</span> <span class="p">(</span><span class="s2">&quot;__packed_session__&quot;</span><span class="p">,</span> <span class="n">_GA</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;sessid&quot;</span><span class="p">),</span> <span class="n">_GA</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;conn_time&quot;</span><span class="p">))</span>
<span class="p">)</span>
<span class="k">return</span> <span class="kc">None</span>
<span class="k">def</span> <span class="nf">unpack_session</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> Check and convert internal representations back to Sessions.</span>
<span class="sd"> Args:</span>
<span class="sd"> item (packed_session): The fact that item is a packed session</span>
<span class="sd"> should be checked before this call.</span>
<span class="sd"> Returns:</span>
<span class="sd"> unpacked (any): Either the original input or converts the</span>
<span class="sd"> internal store back to a Session. If Session no longer</span>
<span class="sd"> exists, None will be returned.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="n">_init_globals</span><span class="p">()</span>
<span class="n">session</span> <span class="o">=</span> <span class="n">_SESSION_HANDLER</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">item</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="k">if</span> <span class="n">session</span> <span class="ow">and</span> <span class="n">session</span><span class="o">.</span><span class="n">conn_time</span> <span class="o">==</span> <span class="n">item</span><span class="p">[</span><span class="mi">2</span><span class="p">]:</span>
<span class="c1"># we require connection times to be identical for the Session</span>
<span class="c1"># to be accepted as the same as the one stored (sessids gets</span>
<span class="c1"># reused all the time).</span>
<span class="k">return</span> <span class="n">session</span>
<span class="k">return</span> <span class="kc">None</span>
<span class="c1">#</span>
<span class="c1"># Access methods</span>
<div class="viewcode-block" id="to_pickle"><a class="viewcode-back" href="../../../api/evennia.utils.dbserialize.html#evennia.utils.dbserialize.to_pickle">[docs]</a><span class="k">def</span> <span class="nf">to_pickle</span><span class="p">(</span><span class="n">data</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> This prepares data on arbitrary form to be pickled. It handles any</span>
<span class="sd"> nested structure and returns data on a form that is safe to pickle</span>
<span class="sd"> (including having converted any database models to their internal</span>
<span class="sd"> representation). We also convert any Saver*-type objects back to</span>
<span class="sd"> their normal representations, they are not pickle-safe.</span>
<span class="sd"> Args:</span>
<span class="sd"> data (any): Data to pickle.</span>
<span class="sd"> Returns:</span>
<span class="sd"> data (any): Pickled data.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">process_item</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Recursive processor and identification of data&quot;&quot;&quot;</span>
<span class="n">dtype</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">if</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">bool</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">,</span> <span class="n">SafeString</span><span class="p">):</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">tuple</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="nb">list</span><span class="p">,</span> <span class="n">_SaverList</span><span class="p">):</span>
<span class="k">return</span> <span class="p">[</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">]</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="nb">dict</span><span class="p">,</span> <span class="n">_SaverDict</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">dict</span><span class="p">((</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="nb">set</span><span class="p">,</span> <span class="n">_SaverSet</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">set</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="n">OrderedDict</span><span class="p">,</span> <span class="n">_SaverOrderedDict</span><span class="p">):</span>
<span class="k">return</span> <span class="n">OrderedDict</span><span class="p">((</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="n">deque</span><span class="p">,</span> <span class="n">_SaverDeque</span><span class="p">):</span>
<span class="k">return</span> <span class="n">deque</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;__iter__&quot;</span><span class="p">):</span>
<span class="c1"># we try to conserve the iterable class, if not convert to list</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="n">item</span><span class="o">.</span><span class="vm">__class__</span><span class="p">([</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">])</span>
<span class="k">except</span> <span class="p">(</span><span class="ne">AttributeError</span><span class="p">,</span> <span class="ne">TypeError</span><span class="p">):</span>
<span class="k">return</span> <span class="p">[</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">]</span>
<span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;sessid&quot;</span><span class="p">)</span> <span class="ow">and</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;conn_time&quot;</span><span class="p">):</span>
<span class="k">return</span> <span class="n">pack_session</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="n">pack_dbobj</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">TypeError</span><span class="p">:</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
<span class="n">logger</span><span class="o">.</span><span class="n">log_err</span><span class="p">(</span><span class="n">f</span><span class="s2">&quot;The object </span><span class="si">{item}</span><span class="s2"> of type {type(item)} could not be stored.&quot;</span><span class="p">)</span>
<span class="k">raise</span>
<span class="k">return</span> <span class="n">process_item</span><span class="p">(</span><span class="n">data</span><span class="p">)</span></div>
<span class="c1"># @transaction.autocommit</span>
<div class="viewcode-block" id="from_pickle"><a class="viewcode-back" href="../../../api/evennia.utils.dbserialize.html#evennia.utils.dbserialize.from_pickle">[docs]</a><span class="k">def</span> <span class="nf">from_pickle</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">db_obj</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;</span>
<span class="sd"> This should be fed a just de-pickled data object. It will be converted back</span>
<span class="sd"> to a form that may contain database objects again. Note that if a database</span>
<span class="sd"> object was removed (or changed in-place) in the database, None will be</span>
<span class="sd"> returned.</span>
<span class="sd"> Args:</span>
<span class="sd"> data (any): Pickled data to unpickle.</span>
<span class="sd"> db_obj (Atribute, any): This is the model instance (normally</span>
<span class="sd"> an Attribute) that _Saver*-type iterables (_SaverList etc)</span>
<span class="sd"> will save to when they update. It must have a &#39;value&#39; property</span>
<span class="sd"> that saves assigned data to the database. Skip if not</span>
<span class="sd"> serializing onto a given object. If db_obj is given, this</span>
<span class="sd"> function will convert lists, dicts and sets to their</span>
<span class="sd"> `_SaverList`, `_SaverDict` and `_SaverSet` counterparts.</span>
<span class="sd"> Returns:</span>
<span class="sd"> data (any): Unpickled data.</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">def</span> <span class="nf">process_item</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Recursive processor and identification of data&quot;&quot;&quot;</span>
<span class="n">dtype</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">if</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">bool</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">,</span> <span class="n">SafeString</span><span class="p">):</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">elif</span> <span class="n">_IS_PACKED_DBOBJ</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="c1"># this must be checked before tuple</span>
<span class="k">return</span> <span class="n">unpack_dbobj</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">_IS_PACKED_SESSION</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="k">return</span> <span class="n">unpack_session</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">tuple</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">dict</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">dict</span><span class="p">((</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">set</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">set</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="n">OrderedDict</span><span class="p">:</span>
<span class="k">return</span> <span class="n">OrderedDict</span><span class="p">((</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="o">.</span><span class="n">items</span><span class="p">())</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="n">deque</span><span class="p">:</span>
<span class="k">return</span> <span class="n">deque</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;__iter__&quot;</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># we try to conserve the iterable class if</span>
<span class="c1"># it accepts an iterator</span>
<span class="k">return</span> <span class="n">item</span><span class="o">.</span><span class="vm">__class__</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">except</span> <span class="p">(</span><span class="ne">AttributeError</span><span class="p">,</span> <span class="ne">TypeError</span><span class="p">):</span>
<span class="k">return</span> <span class="p">[</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">]</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">def</span> <span class="nf">process_tree</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="n">parent</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Recursive processor, building a parent-tree from iterable data&quot;&quot;&quot;</span>
<span class="n">dtype</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">if</span> <span class="n">dtype</span> <span class="ow">in</span> <span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="nb">int</span><span class="p">,</span> <span class="nb">float</span><span class="p">,</span> <span class="nb">bool</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">,</span> <span class="n">SafeString</span><span class="p">):</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">elif</span> <span class="n">_IS_PACKED_DBOBJ</span><span class="p">(</span><span class="n">item</span><span class="p">):</span>
<span class="c1"># this must be checked before tuple</span>
<span class="k">return</span> <span class="n">unpack_dbobj</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">tuple</span><span class="p">:</span>
<span class="k">return</span> <span class="nb">tuple</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">item</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">list</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverList</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">dict</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverDict</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">(</span>
<span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
<span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">set</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverSet</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="nb">set</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">))</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="n">OrderedDict</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverOrderedDict</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">(</span>
<span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
<span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="n">deque</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverDeque</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="s2">&quot;__iter__&quot;</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="c1"># we try to conserve the iterable class if it</span>
<span class="c1"># accepts an iterator</span>
<span class="k">return</span> <span class="n">item</span><span class="o">.</span><span class="vm">__class__</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">parent</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">except</span> <span class="p">(</span><span class="ne">AttributeError</span><span class="p">,</span> <span class="ne">TypeError</span><span class="p">):</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverList</span><span class="p">(</span><span class="n">_parent</span><span class="o">=</span><span class="n">parent</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">item</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">return</span> <span class="n">item</span>
<span class="k">if</span> <span class="n">db_obj</span><span class="p">:</span>
<span class="c1"># convert lists, dicts and sets to their Saved* counterparts. It</span>
<span class="c1"># is only relevant if the &quot;root&quot; is an iterable of the right type.</span>
<span class="n">dtype</span> <span class="o">=</span> <span class="nb">type</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
<span class="k">if</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">list</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverList</span><span class="p">(</span><span class="n">_db_obj</span><span class="o">=</span><span class="n">db_obj</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">data</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">dict</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverDict</span><span class="p">(</span><span class="n">_db_obj</span><span class="o">=</span><span class="n">db_obj</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">(</span>
<span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">data</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
<span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="nb">set</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverSet</span><span class="p">(</span><span class="n">_db_obj</span><span class="o">=</span><span class="n">db_obj</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">data</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="n">OrderedDict</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverOrderedDict</span><span class="p">(</span><span class="n">_db_obj</span><span class="o">=</span><span class="n">db_obj</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">update</span><span class="p">(</span>
<span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">process_tree</span><span class="p">(</span><span class="n">val</span><span class="p">,</span> <span class="n">dat</span><span class="p">))</span> <span class="k">for</span> <span class="n">key</span><span class="p">,</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">data</span><span class="o">.</span><span class="n">items</span><span class="p">()</span>
<span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">elif</span> <span class="n">dtype</span> <span class="o">==</span> <span class="n">deque</span><span class="p">:</span>
<span class="n">dat</span> <span class="o">=</span> <span class="n">_SaverDeque</span><span class="p">(</span><span class="n">_db_obj</span><span class="o">=</span><span class="n">db_obj</span><span class="p">)</span>
<span class="n">dat</span><span class="o">.</span><span class="n">_data</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">process_item</span><span class="p">(</span><span class="n">val</span><span class="p">)</span> <span class="k">for</span> <span class="n">val</span> <span class="ow">in</span> <span class="n">data</span><span class="p">)</span>
<span class="k">return</span> <span class="n">dat</span>
<span class="k">return</span> <span class="n">process_item</span><span class="p">(</span><span class="n">data</span><span class="p">)</span></div>
<div class="viewcode-block" id="do_pickle"><a class="viewcode-back" href="../../../api/evennia.utils.dbserialize.html#evennia.utils.dbserialize.do_pickle">[docs]</a><span class="k">def</span> <span class="nf">do_pickle</span><span class="p">(</span><span class="n">data</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Perform pickle to string&quot;&quot;&quot;</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="n">dumps</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">protocol</span><span class="o">=</span><span class="n">PICKLE_PROTOCOL</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
<span class="n">logger</span><span class="o">.</span><span class="n">log_err</span><span class="p">(</span><span class="n">f</span><span class="s2">&quot;Could not pickle data for storage: </span><span class="si">{data}</span><span class="s2">&quot;</span><span class="p">)</span>
<span class="k">raise</span></div>
<div class="viewcode-block" id="do_unpickle"><a class="viewcode-back" href="../../../api/evennia.utils.dbserialize.html#evennia.utils.dbserialize.do_unpickle">[docs]</a><span class="k">def</span> <span class="nf">do_unpickle</span><span class="p">(</span><span class="n">data</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Retrieve pickle from pickled string&quot;&quot;&quot;</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">return</span> <span class="n">loads</span><span class="p">(</span><span class="n">to_bytes</span><span class="p">(</span><span class="n">data</span><span class="p">))</span>
<span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span>
<span class="n">logger</span><span class="o">.</span><span class="n">log_err</span><span class="p">(</span><span class="n">f</span><span class="s2">&quot;Could not unpickle data from storage: </span><span class="si">{data}</span><span class="s2">&quot;</span><span class="p">)</span>
<span class="k">raise</span></div>
<div class="viewcode-block" id="dbserialize"><a class="viewcode-back" href="../../../api/evennia.utils.dbserialize.html#evennia.utils.dbserialize.dbserialize">[docs]</a><span class="k">def</span> <span class="nf">dbserialize</span><span class="p">(</span><span class="n">data</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Serialize to pickled form in one step&quot;&quot;&quot;</span>
<span class="k">return</span> <span class="n">do_pickle</span><span class="p">(</span><span class="n">to_pickle</span><span class="p">(</span><span class="n">data</span><span class="p">))</span></div>
<div class="viewcode-block" id="dbunserialize"><a class="viewcode-back" href="../../../api/evennia.utils.dbserialize.html#evennia.utils.dbserialize.dbunserialize">[docs]</a><span class="k">def</span> <span class="nf">dbunserialize</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">db_obj</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="sd">&quot;&quot;&quot;Un-serialize in one step. See from_pickle for help db_obj.&quot;&quot;&quot;</span>
<span class="k">return</span> <span class="n">from_pickle</span><span class="p">(</span><span class="n">do_unpickle</span><span class="p">(</span><span class="n">data</span><span class="p">),</span> <span class="n">db_obj</span><span class="o">=</span><span class="n">db_obj</span><span class="p">)</span></div>
</pre></div>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../../../index.html">
<img class="logo" src="../../../_static/evennia_logo.png" alt="Logo"/>
</a></p>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
<h3>Versions</h3>
<ul>
<li><a href="dbserialize.html">1.0-dev (develop branch)</a></li>
<li><a href="../../../../0.9.5/index.html">0.9.5 (master branch)</a></li>
</ul>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../../../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="nav-item nav-item-0"><a href="../../../index.html">Evennia 1.0-dev</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../../index.html" >Module code</a> &#187;</li>
<li class="nav-item nav-item-2"><a href="../../evennia.html" >evennia</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">evennia.utils.dbserialize</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2020, The Evennia developer community.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.2.1.
</div>
</body>
</html>
Reference in a new issue View git blame Copy permalink