So creating python objects for each element of a collection of data can use up far more memory than is needed.
A simple pattern for avoiding this wasted memory is to store the data in a array.array() then construct an object from the part of the data as you need it.
Using python classes to store an int.
Constructing python classes to store an int dynamically.
As you can see this method can save a *lot* of memory.
Here's some basic code demonstrating this technique... this isn't necessarily the API to use, but just demonstrates the memory savings. You can make a nicer to use API on top of that... or use your existing api with get magic properties.
# using python objects...
python collection_memory.py 100000 -object & sleep 2 ; ps aux | grep python
# using a collection of python objects, which constructs a class from the raw data.
python collection_memory.py 100000 & sleep 2 ; ps aux | grep python
Many uses of python could use this technique to save lots of memory. Anything that operates on a large number of python objects.