Yet Another S3 Static Site

Here it is. My version of the S3 static site. This one is publishable through CloudFormation and uses CodeCommit and CodeBuild to regenerate and publish the site with every push to the host Git repository. Any change to the CodeCommit Git repository automatically triggers a build through CodeCommit. This build runs the Hugo static site generator on your repo and syncs the results to an S3 bucket configured for serving a static site. [Read More]

Packaging a Custom Boomi Connector

Having created a custom Boomi connector, the next step is packaging it as a Jar file for testing and release. This requires setting some configuration files that allow the Atom process to load and run your files. Connector configuration file The first configuration we need to set is the META-INF/connector-config.xml file. This file tells the Atom process which class implements your custom connector. This file must have a root XML element named GenericConnector and specify the class name of your connector — which must be the class that extends the BaseConnector class. [Read More]

Creating a Custom Dell Boomi Connector

This article will show you how to create a custom connector for reading data from Dell Boomi. The connector will read the list of GitHub follower’s from the public GitHub API. This should provide an overview of how to write your own custom connector for a unique I/O source. Prerequisites To follow along, you need to have a valid Boomi licence (or free trial) to setup the Boomi Connector SDK. [Read More]

Paper Review: Implementing Data Cubes Efficiently

Business intelligence and analytics use cases involve complex queries on potentially very large databases. To minimize query response times, query optimization is critical. One approach to optimizing query response times is to precompute relevant values ahead of time, and to use those precomputed results to answer queries. Unfortunately, it is not always feasible to precompute every potential value that is required to answer arbitrary queries. This paper describes a framework and presents algorithms that pick a good subset of queries to precompute to optimize response time. [Read More]

Getting Started With Dell Boomi

This tutorial describes how to setup a test integration running Dell Boomi. For those not familiar, Boomi is an integration platform as a service that implements many of the integration patterns described in the book Enterprise Integration Patterns. It is a commercial alternative to the Apache Camel open source integration framework. Boomi Overview Boomi allows developers to run an integration Process, which is composed of an input stage, followed by data transformations and any business logic, followed by one or more output stages where the data is written out to a final destination. [Read More]

Optimistic Locking in a REST API

In a REST application, it’s often the case that several clients might interact with a single resource, each holding a copy of the resources state. At any point in time, these client’s understanding of resource state may differ from each other or from the server. Without some way of realigning resource state, changes requested by a client based on an out-of-date understanding of resource may have undesired effects, from repeating computationally expensive requests to overwriting and losing another client’s changes. [Read More]

Checking for null in an API Gateway transform

Amazon’s API Gateway provides the ability to transform a response from an endpoint into a different format for return to the client. One thing I wished to accomplish with this was to return a value to the client, only if it was set in the response. Essentially, I wanted to check for the existence of a JSON property in the response, and react accordingly. The solution was not obvious to me, and this post serves to record that solution for posterity. [Read More]

Effective Cache Control

The web supports a global network of billions of devices and users, and a key component of this support is effectively caching frequently accessed data along the request-response path. One of the key benefits in adopting a REST architectural style for your system is being able to leverage this existing infrastructure, taking advantage of the billions of dollars of investment already made in the web, and promoting loose coupling, performance and scalability. [Read More]

Configuring REST-assured for Amazon API Gateway

As part of testing our Amazon API Gateway deployment, we set up JUnit tests to run automated Swagger/OpenAPI validation using Swagger Request Validator and REST-assured. This allows us to write simple tests in a fluent style, with automatic validation that requests and responses match the Swagger API specification deployed to API Gateway. given() .log().all() .filter(validationFilter) .when() .post("/oauth2/token") .then() .assertThat() .statusCode(200); Out of the box, REST-assured does not work with Amazon’s API Gateway endpoints using Java 8. [Read More]

Paper Review: Robust Query Processing through Progressive Optimization

Title and Author of Paper Robust Query Processing through Progressive Optimization. Markl et al. Summary Traditional query optimizers choose an execution plan for a query by using estimates of current database statistics. However, these estimates may be inaccurate, leading to overly expensive query plans being chosen and executed. This paper presents progressive query optimization, allowing query execution to detect and recover from estimation errors during processing. During each execution step, progressive query optimization (POP) detects differences between the cardinality of the currently processed tuple and compares that to the estimated cardinality that was used to define the original execution plan. [Read More]