MongoElector¶

Installation¶

$ pip install mongoelector

$ git clone git://github.com/zebpalmer/mongoelector

$ curl  -OL https://github.com/zebpalmer/mongoelector/tarball/master

$ python setup.py install

Usage¶

To use MongoElector in a project:

import mongoelector

# Instantiate MongoElector object
elector = MongoElector('CleverName', dbconn, ttl=15, dbname='coolproj',
                       onmaster=self.onmaster, onmasterloss=self.onmasterloss)

# example callbacks

def onmasterloss():
    logging.info("Master Lost, shutting down task scheduler)
    sched.shutdown() # shutdown APScheduler

def onmaster():
    logging.info("Elected master, starting task scheduler)
    sched.start() # start APScheduler

# start mongoelector
elector.start()

# shutdown mongoelector, release master lock
elector.stop()

# log if master
logging.debug('master status: {}'.format(elector.ismaster))

# release the master lock, allowing another instance to take it
elector.release()

# log if master exists (on any node)
logging.debug('Cluster master is running: {}'.format(elector.master_exists))

# create Pymongo dbconnection
dbconn = MongoClient(cfg.dbhost)
# create lock object with a ttl of 60.
# If the lock isn't refreshed within ttl seconds, it will auto-expire.
mlock = MongoLocker(self.key, dbconn, ttl=60, dbname='coolproj',
                    dbcollection='electorlocks')

# acquire the lock, raise AcquireTimeout after 30 seconds if not acquired.
mlock.acquire(timeout=30)

# release lock
mlock.release()

# check to see if lock is locked by any instance
print(mlock.locked())

# check to see if lock is owned by this instance
print(mlock.owned())

Code Documentation¶

class mongoelector.MongoLocker(key, dbconn, dbname='mongoelector', dbcollection='mongolocker', ttl=600, timeparanoid=True)[source]¶

Distributed lock object backed by MongoDB.

Intended to mimic standard lib Lock object as much as reasonable. This object is used by MongoElector, but is perfectly happy being used as a standalone distributed locking object.

Parameters:

key (str) – Name of distributed lock dbconn (PyMongo db connection) – Pymongo client connection to mongodb dbname (str) – name of database (defaults to ‘mongoelector’) dbname – name of collection (defaults to ‘mongolocker’) ttl (int) – Lock will expire (ttl seconds) after acquired unless renewed or released timeparanoid (bool) – Sanity check to ensure local server time matches mongodb server time (utc)

acquire(blocking=True, timeout=None, step=0.25, force=False)[source]¶

Attempts to acquire the lock, will block and retry indefinitely by default. Can be configured not to block, or to have a timeout. You can also force the acquisition if you have a really good reason to do so.

Parameters:	blocking (bool) – If true (default), will wait until lock is acquired. timeout (int) – blocking acquire will fail after timeout in seconds if the lock hasn’t been acquired yet. step (float or int) – delay between acquire attempts force (bool) – CAUTION: will forcibly take ownership of the lock

get_current()[source]¶

Returns the current (valid) lock object from the database, regardless of which instance it is owned by.

locked()[source]¶

Returns current status of the lock, but does not indicate if the current instance has ownership or not. (for that, use ‘self.owned()’) This is a ‘look before you leap’ option. For example, it can be used to ensure that some process is owns the lock and is doing the associated work. Obviously this method does not guarantee that the current instance will be successful in obtaining the lock on a subsequent acquire.

Returns:	Lock status
Return type:	bool

owned()[source]¶

Determines if self is the owner of the lock object. This verifies the instance uuid matches the uuid of the lock record in the db.

Returns:	Owner status
Return type:	bool

release(force=False)[source]¶

releases lock if owned by the current instance.

Parameters:	force – CAUTION: Forces the release to happen,

even if the local instance isn’t the lock owner. :type force: bool

status¶

touch()[source]¶

Renews lock expiration timestamp

Returns:	new expiration timestamp
Return type:	datetime

class mongoelector.MongoElector(key, dbconn, dbname='mongoelector', ttl=15, onmaster=None, onmasterloss=None, onloop=None, report_status=True)[source]¶

This object will do lots of awesome distributed master election coolness

Create a MongoElector instance

Parameters:

key (str) – Name of the distributed lock that is used for master election. should be unique to this type of daemon i.e. any instance for which you want to run exactly one master should all share this same name. dbconn (PyMongo DB Connection) – Connection to a MongoDB server or cluster dbname (str) – Name of the mongodb database to use (will be created if it doesn’t exist) ttl (int) – Time-to-live for the distributed lock. If the master node fails silently, this timeout must be hit before another node will take over. onmaster (Function or Method) – Function that will be run every time this instance is elected as the new master onmasterloss (Function or Method) – Function that will be run every time when this instance loses it’s master status onloop (Function or Method) – Function that will be run on every loop

cluster_detail¶

ismaster¶

Returns True if this instance is master

master_exists¶

Returns true if an instance (not necessarily this one) has master

node_status¶

Status info for current object

poll()[source]¶

Main polling logic, will refresh lock if it’s owned, or tries to obtain the lock if it’s available. Runs onloop callback after lock maintenance logic

In general, this should only be called by the elector thread

release()[source]¶

Releases master lock if owned and calls onmasterloss if provided.

report_status()[source]¶

running¶

Returns true if the elector logic is running

start(blocking=False)[source]¶

Starts mongo elector polling on a background thread then returns. If blocking is set to True, this will never return until stop() is

Parameters:	blocking (bool) – If False, returns as soon as the elector thread is started. If True, will only return after stop() is called i.e. by another thread.

stop()[source]¶

Cleanly stop the elector. Surrender master if owned

Contributing¶

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.

You can contribute in many ways:

$ python -m unittest tests.test_mongoelector

Authors¶

History¶