diff options
| -rw-r--r-- | README.rst | 73 | ||||
| -rw-r--r-- | apscheduler/job.py | 4 | ||||
| -rw-r--r-- | apscheduler/schedulers/base.py | 35 | ||||
| -rw-r--r-- | docs/contributing.rst (renamed from CONTRIBUTING.rst) | 8 | ||||
| -rw-r--r-- | docs/extending.rst | 29 | ||||
| -rw-r--r-- | docs/index.rst | 436 | ||||
| -rw-r--r-- | docs/migration.rst | 42 | ||||
| -rw-r--r-- | docs/modules/schedulers/base.rst | 4 | ||||
| -rw-r--r-- | docs/userguide.rst | 407 | ||||
| -rw-r--r-- | docs/versionhistory.rst | 1 |
10 files changed, 544 insertions, 495 deletions
@@ -1,67 +1,66 @@ -Advanced Python Scheduler (APScheduler) is a light but powerful in-process task -scheduler that lets you schedule jobs (functions or any python callables) to be -executed at times of your choosing. +Advanced Python Scheduler (APScheduler) is a Python library that lets you schedule your Python code to be executed +later, either just once or periodically. You can add new jobs or remove old ones on the fly as you please. If you store +your jobs in a database, they will also survive scheduler restarts and maintain their state. When the scheduler is +restarted, it will then run all the jobs it should have run while it was offline [#f1]_. -This can be a far better alternative to externally run cron scripts for -long-running applications (e.g. web applications), as it is platform neutral -and can directly access your application's variables and functions. +Among other things, APScheduler can be used as a cross-platform, application specific replacement to platform specific +schedulers, such as the cron daemon or the Windows task scheduler. Please note, however, that APScheduler is **not** a +daemon or service itself, nor does it come with any command line tools. It is primarily meant to be run inside existing +applications. That said, there's nothing to stop you from using APScheduler to build a scheduler service or to run a +dedicated process for APScheduler. In fact, several users have reportedly already done so and the +:class:`~apscheduler.schedulers.blocking.BlockingScheduler` class was made specifically for that purpose. -The development of APScheduler was heavily influenced by the `Quartz -<http://www.quartz-scheduler.org/>`_ task scheduler written in Java. -APScheduler provides most of the major features that Quartz does, but it also -provides features not present in Quartz (such as multiple job stores). +APScheduler has three built-in scheduling systems you can use: +* Cron-style scheduling +* Interval-based execution (runs jobs on even intervals) +* One-off delayed execution (runs jobs once, on a set date/time) -Features -======== +You can mix and match scheduling systems and the backends where the jobs are stored any way you like. +Supported backends for storing jobs include: -* Thread-safe API -* Excellent test coverage (tested on CPython 2.6 - 2.7, 3.2 - 3.4, PyPy 2.3) -* Configurable scheduling mechanisms (triggers): +* Memory +* `SQLAlchemy <http://www.sqlalchemy.org/>`_ (any RDBMS supported by SQLAlchemy works) +* `MongoDB <http://www.mongodb.org/>`_ +* `Redis <http://redis.io/>`_ - * Cron-like scheduling - * Delayed scheduling of single run jobs (like the UNIX "at" command) - * Interval-based (run a job at specified time intervals) -* Integrates with several frameworks: +APScheduler also integrates with several common Python frameworks, like: - * `asyncio <http://docs.python.org/3.4/library/asyncio.html>`_ - (`PEP 3156 <http://www.python.org/dev/peps/pep-3156/>`_) - * `gevent <http://www.gevent.org/>`_ - * `Tornado <http://www.tornadoweb.org/>`_ - * `Twisted <http://twistedmatrix.com/>`_ - * `Qt <http://qt-project.org/>`_ (using either `PyQt <http://www.riverbankcomputing.com/software/pyqt/intro>`_ - or `PySide <http://qt-project.org/wiki/PySide>`_) -* Multiple, simultaneously active job stores: +* `asyncio <http://docs.python.org/3.4/library/asyncio.html>`_ (:pep:`3156`) +* `gevent <http://www.gevent.org/>`_ +* `Tornado <http://www.tornadoweb.org/>`_ +* `Twisted <http://twistedmatrix.com/>`_ +* `Qt <http://qt-project.org/>`_ (using either `PyQt <http://www.riverbankcomputing.com/software/pyqt/intro>`_ + or `PySide <http://qt-project.org/wiki/PySide>`_) - * Memory - * `SQLAlchemy <http://www.sqlalchemy.org/>`_ (any supported RDBMS works) - * `MongoDB <http://www.mongodb.org/>`_ - * `Redis <http://redis.io/>`_ +.. [#f1] The cutoff period for this is also configurable. Documentation -============= +------------- Documentation can be found `here <http://readthedocs.org/docs/apscheduler/en/latest/>`_. Source -====== +------ The source can be browsed at `Bitbucket <http://bitbucket.org/agronholm/apscheduler/src/>`_. Reporting bugs -============== +-------------- -A `bug tracker <https://bitbucket.org/agronholm/apscheduler/issues?status=new&status=open>`_ +A `bug tracker <https://bitbucket.org/agronholm/apscheduler/issues?status-new&status-open>`_ is provided by bitbucket.org. Getting help -============ +------------ If you have problems or other questions, you can either: -* Ask on the `APScheduler Google group <http://groups.google.com/group/apscheduler>`_, or * Ask on the ``#apscheduler`` channel on `Freenode IRC <http://freenode.net/irc_servers.shtml>`_ +* Ask on the `APScheduler Google group <http://groups.google.com/group/apscheduler>`_, or +* Ask on `StackOverflow <http://stackoverflow.com/questions/tagged/apscheduler>`_ and tag your question with the + ``apscheduler`` tag diff --git a/apscheduler/job.py b/apscheduler/job.py index 2abd230..576464d 100644 --- a/apscheduler/job.py +++ b/apscheduler/job.py @@ -82,6 +82,10 @@ class Job(object): return isinstance(self._jobstore, six.string_types) + # + # Private API + # + def _get_run_times(self, now): """ Computes the scheduled run times between ``next_run_time`` and ``now``. diff --git a/apscheduler/schedulers/base.py b/apscheduler/schedulers/base.py index 8ad7d36..53cbcf2 100644 --- a/apscheduler/schedulers/base.py +++ b/apscheduler/schedulers/base.py @@ -22,7 +22,17 @@ from apscheduler.events import ( class BaseScheduler(six.with_metaclass(ABCMeta)): - """Base class for all schedulers.""" + """ + Abstract base class for all schedulers. Takes the following keyword arguments: + + :param str|logging.Logger logger: logger to use for the scheduler's logging (defaults to apscheduler.scheduler) + :param str|datetime.tzinfo timezone: the default time zone (defaults to the local timezone) + :param dict job_defaults: default values for newly added jobs + :param dict jobstores: a dictionary of job store alias -> job store instance or configuration dict + :param dict executors: a dictionary of executor alias -> executor instance or configuration dict + + .. seealso:: :ref:`scheduler-config` + """ _stopped = True @@ -188,12 +198,17 @@ class BaseScheduler(six.with_metaclass(ABCMeta)): def add_listener(self, callback, mask=EVENT_ALL): """ + add_listener(callback, mask=EVENT_ALL) + Adds a listener for scheduler events. When a matching event occurs, ``callback`` is executed with the event object as its sole argument. If the ``mask`` parameter is not provided, the callback will receive events of all types. :param callback: any callable that takes one argument :param int mask: bitmask that indicates which events should be listened to + + .. seealso:: :mod:`apscheduler.events` + .. seealso:: :ref:`scheduler-events` """ with self._listeners_lock: @@ -211,6 +226,10 @@ class BaseScheduler(six.with_metaclass(ABCMeta)): coalesce=undefined, max_instances=undefined, jobstore='default', executor='default', replace_existing=False, **trigger_args): """ + add_job(func, trigger=None, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined, \ + coalesce=undefined, max_instances=undefined, jobstore='default', executor='default', \ + replace_existing=False, **trigger_args) + Adds the given job to the job list and wakes up the scheduler if it's already running. Any argument that defaults to ``undefined`` will use scheduler defaults if no value is provided. @@ -279,7 +298,15 @@ class BaseScheduler(six.with_metaclass(ABCMeta)): def scheduled_job(self, trigger, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined, coalesce=undefined, max_instances=undefined, jobstore='default', executor='default', **trigger_args): - """A decorator version of :meth:`add_job`, except that ``replace_existing`` is always ``True``.""" + """ + scheduled_job(trigger, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined, \ + coalesce=undefined, max_instances=undefined, jobstore='default', executor='default',**trigger_args) + + A decorator version of :meth:`add_job`, except that ``replace_existing`` is always ``True``. + + .. important:: The ``id`` argument must be given if scheduling a job in a persistent job store. The scheduler + cannot, however, enforce this requirement. + """ def inner(func): self.add_job(func, trigger, args, kwargs, id, name, misfire_grace_time, coalesce, max_instances, jobstore, @@ -420,6 +447,8 @@ class BaseScheduler(six.with_metaclass(ABCMeta)): def print_jobs(self, jobstore=None, out=None): """ + print_jobs(jobstore=None, out=sys.stdout) + Prints out a textual listing of all jobs currently scheduled on either all job stores or just a specific one. :param str|unicode jobstore: alias of the job store, ``None`` to list jobs from all stores @@ -445,7 +474,7 @@ class BaseScheduler(six.with_metaclass(ABCMeta)): print(six.u(' No scheduled jobs'), file=out) # - # Protected API + # Private API # def _configure(self, config): diff --git a/CONTRIBUTING.rst b/docs/contributing.rst index ee661b3..c983ab8 100644 --- a/CONTRIBUTING.rst +++ b/docs/contributing.rst @@ -1,3 +1,11 @@ +########################### +Contributing to APScheduler +########################### + +If you wish to add a feature or fix a bug in APScheduler, you need to follow certain procedures and rules to get your +changes accepted. This is to maintain the high quality of the code base. + + Contribution Process ==================== diff --git a/docs/extending.rst b/docs/extending.rst index fb60f84..b328c4d 100644 --- a/docs/extending.rst +++ b/docs/extending.rst @@ -1,12 +1,12 @@ +##################### Extending APScheduler -===================== +##################### -This document is meant to explain how to add extra functionality to -APScheduler, such as custom triggers or job stores. +This document is meant to explain how to develop your custom triggers, job stores, executors or schedulers. -Writing and using custom triggers ---------------------------------- +Custom triggers +--------------- Triggers determine the times when the jobs should be run. APScheduler comes with three built-in triggers -- @@ -25,16 +25,15 @@ according to whatever scheduling logic you wish to implement. If no such datetime can be computed, it should return ``None``. To schedule a job using your custom trigger, you can either extends the -:class:`~apscheduler.scheduler.Scheduler` class to include your own shortcuts, -or use the generic :meth:`~apscheduler.scheduler.Scheduler.add_job` method to +:class:`~apscheduler.schedulers.base.BaseScheduler` class to include your own shortcuts, +or use the generic :meth:`~apscheduler.schedulers.base.BaseScheduler.add_job` method to add your jobs. -Writing and using custom job stores ------------------------------------ +Custom job stores +----------------- -Job store classes should preferably inherit from -:class:`apscheduler.jobstores.base.JobStore`. This class provides stubbed out +Job store classes must inherit from :class:`apscheduler.jobstores.base.BaseJobStore`. This class provides stubbed out methods which any implementation should override. These methods also contain useful documentation regarding the responsibilities of a job store. It is recommended that you look at the existing job store implementations for @@ -44,3 +43,11 @@ To use your job store, you must add it to the scheduler as normal:: jobstore = MyJobStore() scheduler.add_jobstore(jobstore, 'mystore') + + +Custom executors +---------------- + + +Custom schedulers +----------------- diff --git a/docs/index.rst b/docs/index.rst index cbfc5b7..81d42ed 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,438 +1,24 @@ Advanced Python Scheduler ========================= -.. contents:: - -Introduction ------------- - -Advanced Python Scheduler (APScheduler) is a light but powerful in-process task -scheduler that lets you schedule functions (or any other python callables) to be -executed at times of your choosing. - -This can be a far better alternative to externally run cron scripts for -long-running applications (e.g. web applications), as it is platform neutral -and can directly access your application's variables and functions. - -The development of APScheduler was heavily influenced by the `Quartz -<http://www.quartz-scheduler.org/>`_ task scheduler written in Java. -APScheduler provides most of the major features that Quartz does, but it also -provides features not present in Quartz (such as multiple job stores). - - -Features --------- - .. include:: ../README.rst - :start-after: Features - ======== :end-before: Documentation - ============= - -Usage -===== - -Installing APScheduler ----------------------- - -The preferred installation method is by using `pip <http://pypi.python.org/pypi/pip/>`_:: - - $ pip install apscheduler - -or `easy_install <http://pypi.python.org/pypi/distribute/>`_:: - - $ easy_install apscheduler - -If that doesn't work, you can manually `download the APScheduler distribution -<http://pypi.python.org/pypi/APScheduler/>`_ from PyPI, extract and then -install it:: - $ python setup.py install - -Starting the scheduler ----------------------- - -To start the scheduler with default settings:: - - from apscheduler.scheduler import Scheduler - - sched = Scheduler() - sched.start() - -The constructor takes as its first, optional parameter a dictionary of "global" -options to facilitate configuration from .ini files. All APScheduler options -given in the global configuration must begin with "apscheduler." to avoid name -clashes with other software. The constructor also takes options as keyword -arguments (without the prefix). - -You can also configure the scheduler after its instantiation, if necessary. -This is handy if you use the decorators for scheduling and must have a -Scheduler instance available from the very beginning:: - - from apscheduler.scheduler import Scheduler - - sched = Scheduler() - - @sched.scheduled_job('interval', {'hours': 3}) - def some_job(): - print "Decorated job" - - sched.configure(options_from_ini_file) - sched.start() - - -Scheduling jobs ---------------- - -The simplest way to schedule jobs using the built-in triggers is to use one of -the shortcut methods provided by the scheduler: +Table of Contents +================= .. toctree:: - :maxdepth: 1 - - dateschedule - intervalschedule - cronschedule - -These shortcuts cover the vast majority of use cases. However, if you need -to use a custom trigger, you need to use the -:meth:`~apscheduler.scheduler.Scheduler.add_job` method. - -When a scheduled job is triggered, it is handed over to the thread pool for -execution. - -You can request a job to be added to a specific job store by giving the target -job store's alias in the ``jobstore`` option to -:meth:`~apscheduler.scheduler.Scheduler.add_job` or any of the above shortcut -methods. - -You can schedule jobs on the scheduler **at any time**. If the scheduler is not -running when the job is added, the job will be scheduled `tentatively` and its -first run time will only be computed when the scheduler starts. Jobs will not -run retroactively in such cases. - -.. warning:: Scheduling new jobs from existing jobs is not currently reliable. - This will likely be fixed in the next major release. - - -.. _job_options: - -Job options ------------ - -The following options can be given as keyword arguments to -:meth:`~apscheduler.scheduler.Scheduler.add_job` or one of the shortcut -methods, including the decorators. - -===================== ========================================================= -Option Definition -===================== ========================================================= -name Name of the job (informative, does not have to be unique) -misfire_grace_time Time in seconds that the job is allowed to miss the the - designated run time before being considered to have - misfired (see :ref:`coalescing`) - (overrides the global scheduler setting) -coalesce Run once instead of many times if the scheduler - determines that the job should be run more than once in - succession (see :ref:`coalescing`) - (overrides the global scheduler setting) -max_runs Maximum number of times this job is allowed to be - triggered before being removed -max_instances Maximum number of concurrently running instances allowed - for this job (see :ref:`_max_instances`) -===================== ========================================================= - - -Shutting down the scheduler ---------------------------- - -To shut down the scheduler:: - - sched.shutdown() - -By default, the scheduler shuts down its thread pool and waits until all -currently executing jobs are finished. For a faster exit you can do:: - - sched.shutdown(wait=False) - -This will still shut down the thread pool but does not wait for any running -tasks to complete. Also, if you gave the scheduler a thread pool that you want -to manage elsewhere, you probably want to skip the thread pool shutdown -altogether:: - - sched.shutdown(shutdown_threadpool=False) - -This implies ``wait=False``, since there is no way to wait for the scheduler's -tasks to finish without shutting down the thread pool. - -A neat trick to automatically shut down the scheduler is to use an :py:mod:`atexit` -hook for that:: - - import atexit - - sched = Scheduler(daemon=True) - atexit.register(lambda: sched.shutdown(wait=False)) - # Proceed with starting the actual application - - -Scheduler configuration options -------------------------------- - -======================= ========== ============================================== -Directive Default Definition -======================= ========== ============================================== -misfire_grace_time 1 Maximum time in seconds for the job execution - to be allowed to delay before it is considered - a misfire (see :ref:`coalescing`) -coalesce False Roll several pending executions of jobs into one - (see :ref:`coalescing`) -standalone False If set to ``True``, - :meth:`~apscheduler.scheduler.Scheduler.start` - will run the main loop in the calling - thread until no more jobs are scheduled. - See :ref:`modes` for more information. -daemonic True Controls whether the scheduler thread is - daemonic or not. This option has no effect when - ``standalone`` is ``True``. - - If set to ``False``, then the - scheduler must be shut down explicitly - when the program is about to finish, or it will - prevent the program from terminating. - - If set to ``True``, the scheduler will - automatically terminate with the application, - but may cause an exception to be raised on - exit. - - Jobs are always executed in non-daemonic - threads. -threadpool (built-in) Instance of a :pep:`3148` compliant thread - pool or a dot-notation (``x.y.z:varname``) - reference to one -threadpool.core_threads 0 Maximum number of persistent threads in the pool -threadpool.max_threads 20 Maximum number of total threads in the pool -threadpool.keepalive 1 Seconds to keep non-core worker threads waiting - for new tasks -jobstores.X.class Class of the jobstore named X (specified as - module.name:classname) -jobstores.X.Y Constructor option Y of jobstore X -======================= ========== ============================================== - - -.. _modes: - -Operating modes: embedded vs standalone ---------------------------------------- - -The scheduler has two operating modes: standalone and embedded. In embedded mode, -it will spawn its own thread when :meth:`~apscheduler.scheduler.Scheduler.start` -is called. In standalone mode, it will run directly in the calling thread and -will block until there are no more pending jobs. - -The embedded mode is suitable for running alongside some application that requires -scheduling capabilities. The standalone mode, on the other hand, can be used as a -handy cross-platform cron replacement for executing Python code. A typical usage -of the standalone mode is to have a script that only adds the jobs to the scheduler -and then calls :meth:`~apscheduler.scheduler.Scheduler.start` on the scheduler. - -All of the examples in the examples directory demonstrate usage of the standalone -mode, with the exception of ``threaded.py`` which demonstrates the embedded mode -(where the "application" just prints a line every 2 seconds). - - -Job stores ----------- - -APScheduler keeps all the scheduled jobs in *job stores*. Job stores are -configurable adapters to some back-end that may or may not support persisting -job configurations on disk, database or something else. Job stores are added -to the scheduler and identified by their aliases. The alias ``default`` is -special in that if the user does not explicitly specify a job store alias when -scheduling a job, it goes to the ``default`` job store. If there is no job -store in the scheduler by that name when the scheduler is started, a new job -store of type :class:`~apscheduler.jobstores.ram_store.RAMJobStore` is created -to serve as the default. - -The other built-in job stores are: - -* :class:`~apscheduler.jobstores.shelve_store.ShelveJobStore` -* :class:`~apscheduler.jobstores.sqlalchemy_store.SQLAlchemyJobStore` -* :class:`~apscheduler.jobstores.mongodb_store.MongoDBJobStore` -* :class:`~apscheduler.jobstores.redis_store.RedisJobStore` - -Job stores can be added either through configuration options or the -:meth:`~apscheduler.scheduler.Scheduler.add_jobstore` method. The following -are therefore equal:: - - config = {'apscheduler.jobstore.file.class': 'apscheduler.jobstores.shelve_store:ShelveJobStore', - 'apscheduler.jobstore.file.path': '/tmp/dbfile'} - sched = Scheduler(config) - -and:: - - from apscheduler.jobstores.shelve_store import ShelveJobStore - - sched = Scheduler() - sched.add_jobstore(ShelveJobStore('/tmp/dbfile'), 'file') - -The example configuration above results in the scheduler having two -job stores -- one -:class:`~apscheduler.jobstores.ram_store.RAMJobStore` and one -:class:`~apscheduler.jobstores.shelve_store.ShelveJobStore`. - - -Job persistency ---------------- - -The built-in job stores (other than -:class:`~apscheduler.jobstores.ram_store.RAMJobStore`) store jobs in a durable -manner. This means that when you schedule jobs in them, shut down the scheduler, -restart it and readd the job store in question, it will load the previously -scheduled jobs automatically. - -Persistent job stores store a reference to the target callable in text form -and serialize the arguments using pickle. This unfortunately adds some -restrictions: - -* You cannot schedule static methods, inner functions or lambdas. -* You cannot update the objects given as arguments to the callable. - -Technically you *can* update the state of the argument objects, but those -changes are never persisted back to the job store. - -.. note:: None of these restrictions apply to ``RAMJobStore``. - - -.. _max_instances: - -Limiting the number of concurrently executing instances of a job ----------------------------------------------------------------- - -By default, no two instances of the same job will be run concurrently. This -means that if the job is about to be run but the previous run hasn't finished -yet, then the latest run is considered a misfire. It is possible to set the -maximum number of instances for a particular job that the scheduler will let -run concurrently, by using the ``max_instances`` keyword argument when adding -the job. - - -.. _coalescing: - -Missed job executions and coalescing ------------------------------------- - -Sometimes the scheduler may be unable to execute a scheduled job at the time -it was scheduled to run. The most common case is when a job is scheduled in a -persistent job store and the scheduler is shut down and restarted after the job -was supposed to execute. When this happens, the job is considered to have -"misfired". The scheduler will then check each missed execution time against -the job's ``misfire_grace_time`` option (which can be set on per-job basis or -globally in the scheduler) to see if the execution should still be triggered. -This can lead into the job being executed several times in succession. - -If this behavior is undesirable for your particular use case, it is possible -to use `coalescing` to roll all these missed executions into one. In other -words, if coalescing is enabled for the job and the scheduler sees one or more -queued executions for the job, it will only trigger it once. The "bypassed" -runs of the job are not considered misfires nor do they count towards any -maximum run count of the job. - - -Scheduler events ----------------- - -It is possible to attach event listeners to the scheduler. Scheduler events are -fired on certain occasions, and may carry additional information in them -concerning the details of that particular event. It is possible to listen to -only particular types of events by giving the appropriate ``mask`` argument to -:meth:`~apscheduler.scheduler.Scheduler.add_listener`, OR'ing -the different constants together. The listener callable is called with one -argument, the event object. The type of the event object is tied to the event -code as shown below: - -========================== ============== ========================================== -Constant Event class Triggered when... -========================== ============== ========================================== -EVENT_SCHEDULER_START SchedulerEvent The scheduler is started -EVENT_SCHEDULER_SHUTDOWN SchedulerEvent The scheduler is shut down -EVENT_JOBSTORE_ADDED JobStoreEvent A job store is added to the scheduler -EVENT_JOBSTORE_REMOVED JobStoreEvent A job store is removed from the scheduler -EVENT_JOBSTORE_JOB_ADDED JobStoreEvent A job is added to a job store -EVENT_JOBSTORE_JOB_REMOVED JobStoreEvent A job is removed from a job store -EVENT_JOB_EXECUTED JobEvent A job is executed successfully -EVENT_JOB_ERROR JobEvent A job raised an exception during execution -EVENT_JOB_MISSED JobEvent A job's execution time is missed -========================== ============== ========================================== - -See the documentation for the :mod:`~apscheduler.events` module for specifics -on the available event attributes. - -Example:: - - def my_listener(event): - if event.exception: - print 'The job crashed :(' - else: - print 'The job worked :)' - - scheduler.add_listener(my_listener, EVENT_JOB_EXECUTED | EVENT_JOB_ERROR) - - -Getting a list of scheduled jobs --------------------------------- - -If you want to see which jobs are currently added in the scheduler, you can -simply do:: - - sched.print_jobs() - -This will print a human-readable listing of scheduled jobs, their triggering -mechanisms and the next time they will fire. If you supply a file-like object -as an argument to this method, it will output the results in that file. - -To get a machine processable list of the scheduled jobs, you can use the -:meth:`~apscheduler.scheduler.Scheduler.get_jobs` scheduler method. It will -return a list of :class:`~apscheduler.job.Job` instances. - - -Extending APScheduler -===================== - -It is possible to extend APScheduler to support alternative job stores and -triggers. See the :doc:`Extending APScheduler <extending>` document for details. - - -FAQ -=== - -Q: Why do my processes hang instead of exiting when they are finished? - -A: A scheduled job may still be executing. APScheduler's thread pool is wired -to wait for the job threads to exit before allowing the interpreter to exit to -avoid unpredictable behavior caused by the shutdown procedures of the Python -interpreter. A more thorough explanation -`can be found here <http://joeshaw.org/2009/02/24/605>`_. - - -Reporting bugs -============== - -A `bug tracker <http://bitbucket.org/agronholm/apscheduler/issues/>`_ -is provided by bitbucket.org. - - -Getting help -============ - -If you have problems or other questions, you can either: + :maxdepth: 1 -* Ask on the `APScheduler Google group - <http://groups.google.com/group/apscheduler>`_, or -* Ask on the ``#apscheduler`` channel on - `Freenode IRC <http://freenode.net/irc_servers.shtml>`_ + userguide + versionhistory + migration + contributing + extending -.. include:: ../CHANGES.rst +Indices and tables +================== +* :ref:`API reference <modindex>` diff --git a/docs/migration.rst b/docs/migration.rst index a338373..2d0d226 100644 --- a/docs/migration.rst +++ b/docs/migration.rst @@ -1,32 +1,44 @@ -Migrating from APScheduler v2.x to 3.0 -====================================== +############################################### +Migrating from previous versions of APScheduler +############################################### -The 3.0 series is API incompatible with previous releases due to a design -overhaul. +From v2.x to 3.0 +================ -API changes ------------ +The 3.0 series is API incompatible with previous releases due to a design overhaul. + +Configuration changes +--------------------- + +* The "operating mode" concept is gone -- use either BlockingScheduler or BackgroundScheduler directly + +Scheduler API changes +--------------------- * The trigger-specific scheduling methods have been removed entirely from the Scheduler class. Instead, you have to use the - :meth:`~apscheduler.scheduler.Scheduler.add_job` method or the - :meth:`~apscheduler.scheduler.Scheduler.scheduled_job` decorator, giving the + :meth:`~apscheduler.schedulers.base.BaseScheduler.add_job` method or the + :meth:`~apscheduler.schedulers.base.BaseScheduler.scheduled_job` decorator, giving the entry point name of the trigger, or an already constructed instance. It should also be noted that the signature of - :meth:`~apscheduler.scheduler.Scheduler.add_job` has changed due to this. -* Adding a job before the scheduler has been started can now fail if no next - execution time can be calculated for it. Previously it would only fail when - the scheduler was started. + :meth:`~apscheduler.schedulers.base.BaseScheduler.add_job` has changed due to this. * The "simple" trigger has been renamed to "date" +* The old thread pool is gone and it is replaced by :class:`~apscheduler.executors.pool.PoolExecutor` + +Job store changes +----------------- +The job store system was completely overhauled for both efficiency and forwards compatibility. +Unfortunately, this means that the old data is not compatible with the new job stores. +If you need to migrate existing data from APScheduler 2.x to 3.x, contact the APScheduler author. -Migrating from APScheduler v1.x to 2.0 -====================================== + +From v1.x to 2.0 +================ There have been some API changes since the 1.x series. This document explains the changes made to v2.0 that are incompatible with the v1.x API. - API changes ----------- diff --git a/docs/modules/schedulers/base.rst b/docs/modules/schedulers/base.rst index 63ec90c..7f6c686 100644 --- a/docs/modules/schedulers/base.rst +++ b/docs/modules/schedulers/base.rst @@ -8,7 +8,3 @@ API .. autoclass:: BaseScheduler :members: - :exclude-members: add_job - :member-order: alphabetical - - .. automethod:: add_job(func, trigger=None, args=None, kwargs=None, id=None, name=None, misfire_grace_time=undefined, coalesce=undefined, max_runs=undefined, max_instances=undefined, jobstore='default', executor='default', replace_existing=False, **trigger_args) diff --git a/docs/userguide.rst b/docs/userguide.rst new file mode 100644 index 0000000..693ea7f --- /dev/null +++ b/docs/userguide.rst @@ -0,0 +1,407 @@ +########## +User guide +########## + + +Installing APScheduler +---------------------- + +The preferred installation method is by using `pip <http://pypi.python.org/pypi/pip/>`_:: + + $ pip install apscheduler + +If you don't have pip installed, you can easily install it by downloading and running +`get-pip.py <https://bootstrap.pypa.io/get-pip.py>`_. + +If, for some reason, pip won't work, you can manually `download the APScheduler distribution +<https://pypi.python.org/pypi/APScheduler/>`_ from PyPI, extract and then install it:: + + $ python setup.py install + + +Code examples +------------- + +The source distribution contains the :file:`examples` directory where you can find many working examples for using +APScheduler in different ways. The examples can also be +`browsed online <https://bitbucket.org/agronholm/apscheduler/src/master/examples/?at=master>`_. + + +Basic concepts +-------------- + +APScheduler has four kinds of components: + +* triggers +* job stores +* executors +* schedulers + +*Triggers* contain the scheduling logic. Each job has its own trigger which determines when the job should be run next. +Beyond their initial configuration, triggers are completely stateless. + +*Job stores* house the scheduled jobs. The default job store simply keeps the jobs in memory, but others store them in +various kinds of databases. A job's data is serialized when it is saved to a persistent job store, and deserialized when +it's loaded back from it. Job stores (other than the default one) don't keep the job data in memory, but act as +middlemen for saving, loading, updating and searching jobs in the backend. Job stores must never be shared between +schedulers. + +*Executors* are what handle the running of the jobs. They do this typically by submitting the designated callable in a +job to a thread or process pool. When the job is done, the executor notifies the scheduler which them emits an +appropriate event. + +*Schedulers* are what bind the rest together. You typically have only one scheduler running in your application. +The application developer doesn't normally deal with the job stores, executors or triggers directly. Instead, the +scheduler provides the proper interface to handle all those. Configuring the job stores and executors is done through +the scheduler, as is adding, modifying and removing jobs. + + +Choosing the right scheduler, job stores and executors +------------------------------------------------------ + +Your choice of scheduler depends mostly on your programming environment and what you'll be using APScheduler for. +Here's a quick guide for choosing a scheduler: + +* :class:`~apscheduler.schedulers.blocking.BlockingScheduler`: + use when the scheduler is the only thing running in your process +* :class:`~apscheduler.schedulers.background.BackgroundScheduler`: + use then you're not using any of the frameworks below, and want the scheduler to run in the background inside your + application +* :class:`~apscheduler.schedulers.asyncio.AsyncIOScheduler`: + use if your application uses the asyncio module +* :class:`~apscheduler.schedulers.gevent.GeventScheduler`: + use if your application uses gevent +* :class:`~apscheduler.schedulers.tornado.TornadoScheduler`: + use if you're building a Tornado application +* :class:`~apscheduler.schedulers.twisted.TwistedScheduler`: + use if you're building a Twisted application +* :class:`~apscheduler.schedulers.qt.QtScheduler`: + use if you're building a Qt application + +Simple enough, yes? + +To pick the appropriate job store, you need to determine whether you need job persistence or not. If you always recreate +your jobs at the start of your application, then you can probably go with the default +(:class:`~apscheduler.jobstores.memory.MemoryJobStore`). But if you need your jobs to persist over scheduler restarts or +application crashes, then your choice usually boils down to what tools are used in your programming environment. +If, however, you are in the position to choose freely, then +:class:`~apscheduler.jobstores.sqlalchemy.SQLAlchemyJobStore` on a `PostgreSQL <http://www.postgresql.org/>`_ backend is +the recommended choice due to its strong data integrity protection. + +Likewise, the choice of executors is usually made for you if you use one of the frameworks above. +Otherwise, the default :class:`~apscheduler.executors.pool.PoolExecutor` should be good enough for most purposes. +If your workload involves CPU intensive operations, you should configure your PoolExecutor to use process pooling +instead of thread pooling to make use of multiple CPU cores. You can add a second PoolExecutor for this purpose, and +only configure one of them for process pooling. + + +.. _scheduler-config: + +Configuring the scheduler +------------------------- + +APScheduler provides many different ways to configure the scheduler. You can use a configuration dictionary or you can +pass in the options as keyword arguments. You can also instantiate the scheduler first, add jobs and configure the +scheduler afterwards. This way you get maximum flexibility for any environment. + +The full list of scheduler level configuration options can be found on the API reference of the +:class:`~apscheduler.schedulers.base.BaseScheduler` class. Scheduler subclasses may also have additional options which +are documented on their respective API references. Configuration options for individual job stores and executors can +likewise be found on their API reference pages. + +Let's say you want to run BackgroundScheduler in your application with the default job store and the default executor:: + + from apscheduler.schedulers.background import BackgroundScheduler + + + scheduler = BackgroundScheduler() + + # Initialize the rest of the application here, or before the scheduler initialization + +This will get you a BackgroundScheduler with a MemoryJobStore named "default" and a PoolExecutor named "default" with a +default maximum thread count of 10. + +Now, suppose you want more. You want to have *two* job stores using *two* executors and you also want to tweak the +default values for new jobs and set a different timezone. +The following three examples are completely equivalent, and will get you: + +* a MongoDBJobStore named "mongo" +* an SQLAlchemyJobStore named "default" (using SQLite) +* a PoolExecutor using threads, named "default", with a worker count of 20 +* a PoolExecutor using subprocesses, named "processpool", with a worker count of 5 +* UTC as the scheduler's timezone +* coalescing turned off for new jobs by default +* a default maximum instance limit of 3 for new jobs + +Method 1:: + + from apscheduler.schedulers.background import BackgroundScheduler + from apscheduler.jobstores.mongodb import MongoDBJobStore + from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore + from apscheduler.executors.pool import PoolExecutor + from pytz import utc + + + jobstores = { + 'mongo': MongoDBJobStore(), + 'default': SQLAlchemyJobStore(url='sqlite:///jobs.sqlite') + } + executors = { + 'default': PoolExecutor('thread', 20), + 'processpool': PoolExecutor('process', 5) + } + job_defaults = { + 'coalesce': False, + 'max_instances': 3 + } + scheduler = BackgroundScheduler(jobstores=jobstores, executors=executors, job_defaults=job_defaults, timezone=utc) + +Method 2:: + + from apscheduler.schedulers.background import BackgroundScheduler + + + # The "apscheduler." prefix is hard coded + scheduler = BackgroundScheduler({ + 'apscheduler.jobstores.mongo': { + 'class': 'apscheduler.jobstores.mongodb:MongoDBJobStore' + }, + 'apscheduler.jobstores.default': { + 'class': 'apscheduler.jobstores.mongodb:SQLAlchemyJobStore', + 'url': 'sqlite:///jobs.sqlite' + }, + 'apscheduler.executors.default': { + 'class': 'apscheduler.executors.pool:PoolExecutor', + 'type': 'thread', + 'max_workers': '20' + }, + 'apscheduler.executors.processpool': { + 'class': 'apscheduler.executors.pool:PoolExecutor', + 'type': 'process', + 'max_workers': '5' + }, + 'apscheduler.job_defaults.coalesce': 'false', + 'apscheduler.job_defaults.max_instances': '3', + 'apscheduler.timezone': 'UTC', + }) + +Method 3:: + + from apscheduler.schedulers.background import BackgroundScheduler + from apscheduler.jobstores.mongodb import MongoDBJobStore + from apscheduler.jobstores.sqlalchemy import SQLAlchemyJobStore + from apscheduler.executors.pool import PoolExecutor + from pytz import utc + + + jobstores = { + 'mongo': MongoDBJobStore(), + 'default': SQLAlchemyJobStore(url='sqlite:///jobs.sqlite') + } + executors = { + 'default': PoolExecutor('thread', max_workers=20), + 'processpool': PoolExecutor('process', max_workers=5) + } + job_defaults = { + 'coalesce': False, + 'max_instances': 3 + } + scheduler = BackgroundScheduler() + + # .. do something else here, maybe add jobs etc. + + scheduler.configure(jobstores=jobstores, executors=executors, job_defaults=job_defaults, timezone=utc) + +.. note:: If you add jobs before configuring the scheduler, any undefined values will be filled with the hardcoded + defaults. + + +Starting the scheduler +---------------------- + +Starting the scheduler is done by simply calling :meth:`~apscheduler.schedulers.base.BaseScheduler.start` on the +scheduler. For schedulers other than `~apscheduler.schedulers.blocking.BlockingScheduler`, this call will return +immediately and you can continue the initialization process of your application, possibly adding jobs to the scheduler. + +For BlockingScheduler, you will only want to call :meth:`~apscheduler.schedulers.base.BaseScheduler.start` after you're +done with any initialization steps. + +.. note:: After the scheduler has been started, you can no longer alter its settings. + + +Adding jobs +----------- + +There are two ways to add jobs to a scheduler: + +#. by calling :meth:`~apscheduler.schedulers.base.BaseScheduler.add_job` +#. by decorating a function with :meth:`~apscheduler.schedulers.base.BaseScheduler.scheduled_job` + +The first way is the most common way to do it. The second way is mostly a convenience to declare jobs that don't change +during the application's run time. The :meth:`~apscheduler.schedulers.base.BaseScheduler.add_job` method returns a +:class:`apscheduler.job.Job` instance that you can use to modify or remove the job later. + +You can schedule jobs on the scheduler **at any time**. If the scheduler is not yet running when the job is added, the +job will be scheduled *tentatively* and its first run time will only be computed when the scheduler starts. + +It is important to note that if you use an executor or job store that serializes the job, it will add a couple +requirements on your job: + +#. The target callable must be globally accessible +#. Any arguments to the callable must be serializable + +Of the builtin job stores, only MemoryJobStore doesn't serialize jobs. +Of the builtin executors, only a PoolExecutor configured for process pooling will serialize jobs. + +.. important:: If you schedule jobs in a persistent job store during your application's initialization, you **MUST** + define an explicit ID for the job and use ``replace_existing=True`` or you will get a new copy of the job every time + your application restarts! + +.. tip:: To run a job immediately, omit ``trigger`` argument when adding the job. + + +Removing jobs +------------- + +When you remove a job from the scheduler, it is removed from its associated job store and will not be executed anymore. +There are two ways to make this happen: + +#. by calling :meth:`~apscheduler.schedulers.base.BaseScheduler.remove_job` with the job's ID and job store alias +#. by calling :meth:`~apscheduler.job.Job.remove` on the Job instance you got from + :meth:`~apscheduler.schedulers.base.BaseScheduler.add_job` + +The latter method is probably more convenient, but it requires that you store somewhere the +:class:`~apscheduler.job.Job` instance you received when adding the job. For jobs scheduled via the +:meth:`~apscheduler.schedulers.base.BaseScheduler.scheduled_job`, the first way is the only way. + +If the job's schedule ends (i.e. its trigger doesn't produce any further run times), it is automatically removed. + +Example:: + + job = scheduler.add_job(myfunc, 'interval', minutes=2) + job.remove() + +Same, using an explicit job ID:: + + scheduler.add_job(myfunc, 'interval', minutes=2, id='my_job_id') + scheduler.remove_job('my_job_id') + + +Pausing and resuming jobs +------------------------- + +You can easily pause and resume jobs through either the :class:`~apscheduler.job.Job` instance or the scheduler itself. +When a job is paused, its next run time is cleared and no further run times will be calculated for it until the job is +resumed. To pause a job, use either method: + +* :meth:`apscheduler.job.Job.pause` +* :meth:`apscheduler.schedulers.base.BaseScheduler.pause_job` + +To resume: + +* :meth:`apscheduler.job.Job.resume` +* :meth:`apscheduler.schedulers.base.BaseScheduler.resume_job` + + +Getting a list of scheduled jobs +-------------------------------- + +To get a machine processable list of the scheduled jobs, you can use the +:meth:`~apscheduler.schedulers.base.BaseScheduler.get_jobs` method. It will return a list of +:class:`~apscheduler.job.Job` instances. If you're only interested in the jobs contained in a particular job store, +then give a job store alias as the second argument. + +As a convenience, you can use the :meth:`~apscheduler.schedulers.base.BaseScheduler.print_jobs` method which will print +out a formatted list of jobs, their triggers and next run times. + + +Modifying jobs +-------------- + +You can modify any job attributes by calling either :meth:`apscheduler.job.Job.modify` or +:meth:`~apscheduler.schedulers.base.BaseScheduler.modify_job`. You can modify any Job attributes except for ``id``. + +Example:: + + job.modify(max_instances=6, name='Alternate name') + +If you want to reschedule the job -- that is, change its trigger, you can use either +:meth:`apscheduler.job.Job.reschedule` or :meth:`~apscheduler.schedulers.base.BaseScheduler.reschedule_job`. +These methods construct a new trigger for the job and recalculate its next run time based on the new trigger. + +Example:: + + scheduler.reschedule_job('my_job_id', trigger='cron', minute='*/5') + + +Shutting down the scheduler +--------------------------- + +To shut down the scheduler:: + + scheduler.shutdown() + +By default, the scheduler shuts down its job stores and executors and waits until all currently executing jobs are +finished. If you don't want to wait, you can do:: + + scheduler.shutdown(wait=False) + +This will still shut down the job stores and executors but does not wait for any running +tasks to complete. + + +Limiting the number of concurrently executing instances of a job +---------------------------------------------------------------- + +By default, only one instance of each job is allowed to be run at the same time. +This means that if the job is about to be run but the previous run hasn't finished yet, then the latest run is +considered a misfire. It is possible to set the maximum number of instances for a particular job that the scheduler will +let run concurrently, by using the ``max_instances`` keyword argument when adding the job. + + +Missed job executions and coalescing +------------------------------------ + +Sometimes the scheduler may be unable to execute a scheduled job at the time it was scheduled to run. +The most common case is when a job is scheduled in a persistent job store and the scheduler is shut down and restarted +after the job was supposed to execute. When this happens, the job is considered to have "misfired". +The scheduler will then check each missed execution time against the job's ``misfire_grace_time`` option (which can be +set on per-job basis or globally in the scheduler) to see if the execution should still be triggered. +This can lead into the job being executed several times in succession. + +If this behavior is undesirable for your particular use case, it is possible to use `coalescing` to roll all these +missed executions into one. In other words, if coalescing is enabled for the job and the scheduler sees one or more +queued executions for the job, it will only trigger it once. No misfire events will be sent for the "bypassed" runs. + + +.. _scheduler-events: + +Scheduler events +---------------- + +It is possible to attach event listeners to the scheduler. Scheduler events are fired on certain occasions, and may +carry additional information in them concerning the details of that particular event. +It is possible to listen to only particular types of events by giving the appropriate ``mask`` argument to +:meth:`~apscheduler.schedulers.base.BaseScheduler.add_listener`, OR'ing the different constants together. +The listener callable is called with one argument, the event object. + +See the documentation for the :mod:`~apscheduler.events` module for specifics on the available events and their +attributes. + +Example:: + + def my_listener(event): + if event.exception: + print('The job crashed :(') + else: + print('The job worked :)') + + scheduler.add_listener(my_listener, EVENT_JOB_EXECUTED | EVENT_JOB_ERROR) + + +Reporting bugs +-------------- + +.. include:: ../README.rst + :start-after: Reporting bugs + -------------- diff --git a/docs/versionhistory.rst b/docs/versionhistory.rst new file mode 100644 index 0000000..d9e113e --- /dev/null +++ b/docs/versionhistory.rst @@ -0,0 +1 @@ +.. include:: ../CHANGES.rst |