Hi there 👋

DynamoDB provides built-in support for cross regional data replication using a solution AWS calls global tables. This article shows how to build and run an application in Kubernetes that uses global tables to replicate data between regions. In the event of a regional disaster, a secondary Kubernetes cluster in a secondary region has all the data from DynamoDB replicated locally to continue operation. How global tables work A DynamoDB global table is a set of multiple replica tables....

Disaster events are one of the biggest challenges that a software organization can face. Natural disasters like earthquakes or floods, technical failures such as power or network loss, and human actions such as unauthorized attacks can disable an entire fleet of systems, leading to complete failure for a business. To deal with disaster scenarios requires a proactive approach to prepare and recover from failure. One of the key benefits of running in the cloud is how easy it is to run workloads in multiple regions....

Check out the project on Github. I recently published a new project for repeatable filesystem benchmarking. This code is based on the benchmarks available in the AWS Mountpoint-s3 project and the Flexible I/O Tester, made into a more general purpose utility for benchmarking arbitrary filesystems. The benchmark consists of a mix of read and write workloads, each run for ten iterations. The average result of the ten iterations is reported as the final result of the benchmark....

AWS provides four different storage options for your Kubernetes cluster: EBS, EFS, FSx for Lustre, and Amazon File Cache. Each of these CSI drivers has different performance characteristics, depending on your workload. This post quantifies those performance differences using the flexible I/O tester FIO. Note: For an overview of the different CSI options available on AWS, see Picking the right AWS CSI driver for your Kubernetes application. Before we start, please note that these results come with multiple caveats, and you should absolutely test performance on your own workloads before making any final decisions....

Dealing with storage is a core challenge when running complex applications on Kubernetes. While many applications operate just fine using a cloud database or blob storage, some applications have performance or design requirements requiring local storage. Note: For performance benchmarks see [Benchmarking AWS CSI Drivers](<{{ relref “/post/kubernetes/benchmarking-aws-csi-drivers” }}). When this is the case, developers and cluster operators rely on Container Storage Interface (CSI) implementations to provide local storage for Pods. When running on the AWS cloud, no less than four CSI providers are available for us to use: Elastic Block Storage, Elastic File System, FSx for Lustre, and AWS File Cache....

eBPF is a relatively new extension of the Linux kernel that can run sandboxed programs in a privileged context. It is used to safely and efficiently extend the capabilities of the kernel at runtime without requiring changes to kernel source code or the loading of kernel modules. Because of eBPFs tight integration with the networking stack at the kernel level, it is seeing adoption in networking applications. This includes Kubernetes networking through eBPF implementations of the Kubnernetes networking stack like Cilium....

I was recently introduced to a simple article on system design: Users are almost always right. In it, a simple rule is stated: When the users keep doing it wrong, the users are right and your system is wrong. This rule reminded me of the concept of desire paths: unplanned trails created by human or animal traffic that usually represent the shortest or easiest route between two places. Desire paths in an urban setting often run counter to what a planner might suggest, leading to a natural conflict between the pedestrian and the planner....

I was recently testing file upload performance, and needed several large files of different sizes to test with. To make the math easier, it was helpful if I had files with round numbered sizes like 10MB, 20MB, or 100MB. After searching around for files of the right size, it turns out the easiest solution is to generate one yourself using the Linux command line. Depending on your needs, you can use two different methods of generating files using some simple commands....

A JWT (JSON Web Token) is a type of security token that is used to securely transmit information between parties. It is a compact and self-contained way of representing claims to be transferred between two parties. Many API providers require you to generate a cryptographically signed JWT assertion that includes the authorization you are requesting from the server. If the server accepts the assertions you are claiming in the JWT, it will respond with an access token granting access to the API....

Application Portfolio Management (APM) draws inspiration from financial portfolio management, which has been around since at least the 1970s. By looking at all applications and services in the organization and analyzing their costs and benefits, you can determine the most effective way to manage them as part of a larger overall strategy. This allows the architect or engineering leader to take a more strategic approach to managing their application portfolio backed by data....