App Engine Pipelines API - Part 6: The Pipeline UI

View all articles in the Pipeline API Series. This article will serve as a reminder of the Pipeline UI as much for the writer as for the reader. The Pipeline UI requires the MapMeduce library to be installed. If you are not familiar with MapReduce please refer to the MapReduce API Series of articles. Once MapReduce is installed you will need to add a few indices to index. [Read More]

App Engine Pipelines API - Part 5: Asynchronous Pipelines

View all articles in the Pipeline API Series. This article will cover fully asynchronous pipelines. The term ‘asynchronous’ is misleading here — all piplines are asynchronous in the sense that yielding a pipeline is a non-blocking operation. An asynchronous refers to a pipeline that remains in a RUN state until outside action is taken, for example, a button is clicked or a task is executed. Marking a pipeline as an asynchronous pipeline is as simple as setting the async class property to True. [Read More]

App Engine Pipelines API - Part 4: Pipeline Internals

View all articles in the Pipeline API Series. We’ve learned how to execute and chain together pipelines, now let’s take a look at how pipelines execute under the hood. If necessary, you can refer to the source code of the pipelines project to clarify any details. The Pipeline Data Model Let’s start with the pipeline data model. Note that each Kind defined by the pipelines API is prefixed by _AE_Pipeline, making it easy to view individual pipeline details by viewing the datastore entity. [Read More]

App Engine Pipelines API - Part 1: The Basics

View all articles in the Pipeline API Series. The Pipelines API is a general purpose workflow engine for App Engine applications. With the Pipelines API we can connect together complex workflows into a coherent run time backed by the Datastore. This article provides a basic overview of the Pipelines API and how it can be used for abritrary computational workflows. In the most basic sense a Pipeline is an object that takes input, performs some logic or computation on that input, and produces output. [Read More]