Aws

One of the larger hurdles to climb when becoming FedRamp moderate is encrypting all data in transit using FIPS validated encryption modules. This article describes one approach to satisfying this requirement for anyone running a Kubernetes cluster on AWS. For those lucky souls who are unfamiliar with FIPS, it stands for Federal Information Processing Standard (FIPS), which is a joint US and Canadian government standard that specifies the security requirements for cryptographic modules that protect sensitive information. FedRamp specifically requires you to satisfy two related controls: SC-8 and SC-13. ...

A common pattern when scaling a SaaS application is to deploy your stack to multiple regions across the globe. Deploying to multiple regions helps distribute the application and data closer to the user, and isolates infrastructure in case of an outage in a single region. If your application is deployed in this configuration, you have the new problem of routing users to the correct region based on their tenancy or geography. This post shows how to create a global login portal that directs users to the correct local region where they can access their application. ...

DynamoDB provides built-in support for cross regional data replication using a solution AWS calls global tables. This article shows how to build and run an application in Kubernetes that uses global tables to replicate data between regions. In the event of a regional disaster, a secondary Kubernetes cluster in a secondary region has all the data from DynamoDB replicated locally to continue operation. How global tables work A DynamoDB global table is a set of multiple replica tables. Each replica table exists in a different AWS region, but all replicas have the same name and primary key. Whenever data is written to any replica table, the data is automatically replicated to all other replica tables that have been added to the global table. ...

AWS provides four different storage options for your Kubernetes cluster: EBS, EFS, FSx for Lustre, and Amazon File Cache. Each of these CSI drivers has different performance characteristics, depending on your workload. This post quantifies those performance differences using the flexible I/O tester FIO. Note: For an overview of the different CSI options available on AWS, see Picking the right AWS CSI driver for your Kubernetes application. Before we start, please note that these results come with multiple caveats, and you should absolutely test performance on your own workloads before making any final decisions. For example, a workload with many random reads is typically latency sensitive and would benefit from higher IOPS, whereas a workload consisting of mostly streaming reads is typically throughput sensitive and would benefit from higher bandwidth and at large I/O sizes with relatively lower IOPS. ...

Dealing with storage is a core challenge when running complex applications on Kubernetes. While many applications operate just fine using a cloud database or blob storage, some applications have performance or design requirements requiring local storage. Note: For performance benchmarks see [Benchmarking AWS CSI Drivers](<{{ relref “/post/kubernetes/benchmarking-aws-csi-drivers” }}). When this is the case, developers and cluster operators rely on Container Storage Interface (CSI) implementations to provide local storage for Pods. When running on the AWS cloud, no less than four CSI providers are available for us to use: Elastic Block Storage, Elastic File System, FSx for Lustre, and AWS File Cache. This article compares these four different local storage options to help you choose the right option for your application. ...

Amazon’s Elastic Compute Cloud (EC2). EC2 is a web service that provides resizable, on-demand computing capacity — literally, servers in Amazon’s data centers — that you use to build and host your software. It’s important to understand that EC2 is a virtual computing environment. In a virtual environment, there is one physical server with all of the necessary hardware — CPU, memory, hard disk, network controller and more. This single physical server can host multiple operating systems and applications through a hypervisor that runs directly on top of the physical machine. The following simplified diagram shows a hypervisor architecture based on the Xen project: Xen interfaces directly with the hardware, and has a special control domain (dom0) for interfacing with Xen. Domain 0 is designed to access hardware directly and manage devices and device drivers. It has the ability to talk to the hypervisor to instruct it to start and stop guest VMs. ...

At first glance, Amazon’s First-In-First-Out (FIFO) message queues provide an excellent feature set for business-critical scenarios. With FIFO, the order in which messages are sent and received is strictly preserved. With exactly-once processing, message duplicates are not introduced into the queue, and consumers control when a message is made available for redelivery. Reading past the marketing hype, how well do FIFO queues work in the real world? This article takes a deep dive into SQS FIFO queues to test the claims of message ordering and exactly-once processing, paying particular care to the conditions under which these claims hold. I show how exactly-once processing can be achieved, and how that message ordering is preserved through failure conditions. I conclude with an examination of the relationship between in-order and exactly-once processing and atomic broadcast and consensus. ...

For the past year I’ve been making a concerted effort to learn French using the methods from the book Fluent Forever, which is an excellent resource for learning how to learn a language. For those not familiar with the method, it boils down to this: Learn Pronunciation: knowing how to correctly pronounce words in your target language makes everything else easier. Learn Frequently Used Words: not all words are created equal, learn the most frequently used words first. Learn Grammar: put together grammatical sentences using the words you already know. If you turn your head to the side and squint at that list, it somewhat resembles the steps you would take to learn a language as an infant — first understand the sounds of the language, then learn words (“mommy”, “daddy”), and finally put together correct sentences. In addition, as an infant you have a constant source of high quality input helping you learn words and grammar. You can imagine the following “conversation” between an adult and a hungry child: ...

This tutorial walks through how to create a fully functional Virtual Private Cloud in AWS using CloudFormation. At the end of the tutorial, you will have a reproducible way to create a virtual cloud with three subnets, a security group, and an internet gateway with SSH access for your IP address. I’ve found this template useful for creating an isolated environment to develop and test software. Full code for this tutorial is available on Github. ...

When designing Workiva’s durable messaging system we took a hard look at using Amazon’s Kinesis as the message storage and delivery mechanism. At first glance, Kinesis has a feature set that looks like it can solve any problem: it can store terabytes of data, it can replay old messages, and it can support multiple message consumers. But if you dig a little deeper you will find that Kinesis is well suited for a very particular use case, and if your application doesn’t fit this use case, Kinesis may be a lot more trouble than it’s worth. ...