Internals

Classes here are not (yet) directly exposed to the user, but present here so that other user-visible documentation can link to them.

Key Encoding Library

Key

acid.keylib.Key is documented in the core interface chapter.

KeyList

class acid.keylib.KeyList

Represents a potentially lazy-decoded list of Key objects.

classmethod from_raw(packed, prefix=None, source=None)

Produce a KeyList from the buffer or bytestring packed, ignoring prefix bytes at the start of the string. If prefix does not match the actual prefix in packed, return None.

If source is not None, packed must be a buffer and source should be a source object implementing the Memsink Protocol.

Iterators

Iterator Result

class acid.iterators.Result

Interface for a single element from an iterator’s result set. Iterator classes do not return Result instances, they only return objects satisfying the same interface.

data = None

Object satisfying the buffer interface that represents the raw record data.

key = None

For a BatchIterator, the current key. Unused for a BasicIterator.

keys = None

List of acid.keylib.Key decoded from the physical engine key.

Iterator

class acid.iterators.Iterator
set_exact(key)

Set the lower and upper bounds such that key is the only key returned, if it exists.

set_hi(key, closed=False)

Set the upper bound to key. If closed is True, include the upper bound in the result set, otherwise exclude it.

set_lo(key, closed=True)

Set the lower bound to key. If closed is True, include the lower bound in the result set, otherwise exclude it.

set_max(max_)

Set the maximum size of the result set.

set_prefix(key)

Set the lower bound to >= key and the upper bound to < prefix_bound(key). If prefix_bound returns None, then set no upper bound (i.e. iterate to the end of the collection).

BasicIterator

class acid.iterators.BasicIterator(engine, prefix)

Provides bidirectional iteration of a range of keys.

engine:
acid.engines.Engine instance to iterate.
prefix:
Bytestring prefix for all keys.
forward()

Begin yielding objects satisfying the Result interface, from lo..`hi`. Note the yielded object is reused, so references to it should not be held.

reverse()

Begin yielding objects satisfying the Result interface, from hi..`lo`. Note the yielded object is reused, so references to it should not be held.

BatchIterator

class acid.iterators.BatchIterator(engine, prefix, compressor)

Provides bidirectional iteration of a range of keys, treating >1-length keys as batch records.

engine:
acid.engines.Engine instance to iterate.
prefix:
Bytestring prefix for all keys.
compressor:
acid.encoders.Compressor instance used to decompress batch keys.
batch_items()

Yield (key, value) pairs that are present in the current batch. Used to implement batch split, may be removed in future.

forward()

Begin yielding objects satisfying the Result interface, from lo..`hi`. Note the yielded object is reused, so references to it should not be held.

reverse()

Begin yielding objects satisfying the Result interface, from lo..`hi`. Note the yielded object is reused, so references to it should not be held.

set_max_phys(max_phys)

Set the maximum number of physical records to visit.

phys_key = None

Raw bytestring/buffer physical key for the current batch, or None.

Strategies

BasicStrategy

class acid.core.BasicStrategy(prefix)

Access strategy for ‘basic’ ordered collections, i.e. those containing no batch records, or the metadata collection.

delete(txn, key)

Implement delete() as Engine.delete(key).

get(txn, key)

Implement get() as Engine.get(key).

iter(txn)

Implement iter() using acid.iterators.BasicIterator.

pop(txn, key)

Implement pop() as Engine.pop(key).

put(txn, key, data)

Implement put() as Engine.put(key).

replace(txn, key, data)

Implement replace() as Engine.replace(key).

BatchStrategy

class acid.core.BatchStrategy(prefix, store, compressor)

Access strategy for ordered collections containing batch records.

ITERATOR_CLASS

alias of BatchIterator

batch(lo=None, hi=None, prefix=None, max_recs=None, max_bytes=None, max_keylen=None, preserve=True, max_phys=None, grouper=None)

Search the key range lo..hi for individual records, combining them into a batches.

Returns (found, made, last_key) indicating the number of records combined, the number of batches produced, and the last key visited before max_phys was exceeded.

Batch size is controlled via max_recs and max_bytes; at least one must not be None. Larger sizes may cause pathological behaviour in the storage engine (for example, space inefficiency). Since batches are fully decompressed before any member may be accessed via get() or iteritems(), larger sizes may slow decompression, waste IO bandwidth, and temporarily use more RAM.

lo:
Lowest search key.
hi:
Highest search key.
max_recs:
Maximum number of records contained by any single batch. When this count is reached, the current batch is saved and a new one is created.
max_bytes:
Maximum size in bytes of the batch record’s value after compression, or None for no maximum size. When not None, values are recompressed after each member is appended, in order to test if max_bytes has been reached. This is inefficient, but provides the best guarantee of final record size. Single records are skipped if they exceed this size when compressed individually.
preserve:

If True, then existing batch records in the database are left untouched. When one is found within lo..hi, the currently building batch is finished and the found batch is skipped over.

If False, found batches are exploded and their members contribute to the currently building batch.

max_phys:
Maximum number of physical keys to visit in any particular call. A collection may be incrementally batched by repeatedly invoking Collection.batch() with max set, and lo set to last_key of the previous run, until found returns 0. This allows batching to complete over several transactions without blocking other users.
grouper:
Specifies a grouping function used to decide when to avoid compressing unrelated records. The function is passed a record’s value. A new batch is triggered each time the function’s return value changes.
delete(txn, key)

Alias for pop() (which satisfies the delete() interface).

get(txn, key)

Implement get() using a range query over >= key.

iter(txn)

Implement iter() using acid.iterators.BatchIterator.

pop(txn, key)

Implement pop() using a range query to find the single or batch record key belongs to and splitting it, saving all records individually except for key. Return the data for key if it existed, otherwise None.

put(txn, key, data)

Alias for replace() (which satisfies the put() interface).

replace(txn, key, data)

Implement replace() by popping any existing value, then writing out the new record and returning the old value.

Contexts

TxnContext

class acid.core.TxnContext(engine)

Abstraction for maintaining the local context’s transaction. This implementation uses TLS.

GeventTxnContext

class acid.core.GeventTxnContext(engine)

Like TxnContext except using gevent.local.local().