I’ve been working on an application to read data from a Kinesis using the Kinesis Client Library for Java. One requirement was to be able to run the application locally during development. This requires configuring Kinesis, DynamoDB, and CloudWatch to work locally.

Dynalite

Getting a local copy of DynamoDB running is easily done using the dynalite library.

> npm install -g dynalite
> dynalite --port 7000

Kinesalite

A local copy of Kinesis is found with the kinesalite library.

> npm install -g kinesalite
> kinesalite --port 7001

Configuring the application

With both dynalite and kinesalite running, it’s time to configure our application to use them. This is done by instantiating AmazonDynamoDBClient and AmazonKinesisClient objects with local end-points. You also need to use an instance of NullMetricsFactory to disable CloudWatch reporting while running locally.

void main(String[] args) throws Exception {

    AmazonDynamoDB dynamoClient = new AmazonDynamoDBClient();
    dynamoClient.setEndpoint("http://localhost:7000");

    AmazonKinesis kinesisClient = new AmazonKinesisClient();
    kinesisClient.setEndpoint("http://localhost:7001");

    KinesisClientLibConfiguration config =
            new KinesisClientLibConfiguration("my-app", "my-stream", credentialsProvider, "my-worker")
                    .withInitialPositionInStream(InitialPositionInStream.LATEST);

    final Worker worker = new Worker.Builder()
            .recordProcessorFactory(new DruidRecordProcessorFactory())
            .config(config)
            .dynamoDBClient(dynamoClient)
            .kinesisClient(kinesisClient)
            .metricsFactory(new NullMetricsFactory())
            .build();

    worker.run();
}