Using Redis is pretty easy to create a simple, distributed and robust scheduler. How? Just using sorted set structure. Sorted set allows you to put inside not only an object but also a score. If you use timestamp as score, you have done!
A scheduler needs these three primitives:
The former two primitives are pretty easy to implement, just use
ZREM command of redis, using the current timestamp (better in UTC) on ZADD:
and a basic ZREM
get_expired is a bit harder, it needs to pop a fired element from the set and lock it
for an amount of time, required by the worker to do the job with it.
Before that time, the worker should already have removed the item from the set.
Otherwise, if for any reason it failed (crashes for example), the object will
fire again on another worker, we will handle failover in this way.
To achieve this primitive, basic Redis commands are not useful, we need to use a script:
Used in this way:
Finally you need a process-worker, written in any language you want that every N seconds polls redis using get_expired() primitive, getting jobs and running the work.
Weakpoints of this scheduler are: polling approach and schedule time precision, which is in the range 0 ≤ precision ≤ polling_interval. But as a tradeoff, the result is a scheduler with no master/slaves synchonizations, simple and with good failover.
Copyright © 2014-2021 Luca Marturana. License.