This article will cover fully asynchronous pipelines. The term ‘asynchronous’ is misleading here — all piplines are asynchronous in the sense that yielding a pipeline is a non-blocking operation. An asynchronous refers to a pipeline that remains in a RUN state until outside action is taken, for example, a button is clicked or a task is executed.
Marking a pipeline as an asynchronous pipeline is as simple as setting the
async
class property to True.
|
|
Once this pipeline starts, it will remain in the RUN state until the
pipeline is transitioned to another state. You transition a pipeline to
another state by calling the complete
method, using a callback.
complete()
is a method only available to asynchronous pipelines. Calling
complete will fill the pipelines output slots and, if all slots have been
filled, mark the pipeline complete. Any barriers related to the slots
being filled are notified as described in the previous article.
|
|
Callback URLs
The pipeline API provides convenience methods for calling the callback method.
get_callback_url
returns a URL that, when accessed, passes any query
parameters to the callback method. For example, to generate a URL to our
pipeline with a choice
parameter we can call get_callback_url as follows:
|
|
This will generate a URL of the form:
|
|
Accessing this URL will pass the choice
parameter to the callback function of
the pipeline with pipeline_id fd789852183b4310b5f1353205a967fe
.
|
|
Running the pipeline above will log the Callback URL to the console. By visiting
that URL, the callback
method will execute, completing your pipeline. You can
refer to the EmailToContinue Pipeline for a more robust example.
Callback Tasks
The second way to execute a callback method is via a callback task. The Pipelines API provides another convenience method to generate a callback task that will execute our pipeline. In the following example, a task is created to trigger in the future, adding an artificial delay to our pipeline.
|
|
Note that the task is queued using the pipeline_id in the task name. This helps ensure our run method is idempotent. Full source code for an asynchronous pipeline follows. This pipeline will delay for 10 seconds, and then log a callback_url to the console. Visiting the callback URL will complete the pipeline.
|
|