Check out the project on Github. I recently published a new project for repeatable filesystem benchmarking. This code is based on the benchmarks available in the AWS Mountpoint-s3 project and the Flexible I/O Tester, made into a more general purpose utility for benchmarking arbitrary filesystems. The benchmark consists of a mix of read and write workloads, each run for ten iterations. The average result of the ten iterations is reported as the final result of the benchmark. ...

AWS provides four different storage options for your Kubernetes cluster: EBS, EFS, FSx for Lustre, and Amazon File Cache. Each of these CSI drivers has different performance characteristics, depending on your workload. This post quantifies those performance differences using the flexible I/O tester FIO. Note: For an overview of the different CSI options available on AWS, see Picking the right AWS CSI driver for your Kubernetes application. Before we start, please note that these results come with multiple caveats, and you should absolutely test performance on your own workloads before making any final decisions. For example, a workload with many random reads is typically latency sensitive and would benefit from higher IOPS, whereas a workload consisting of mostly streaming reads is typically throughput sensitive and would benefit from higher bandwidth and at large I/O sizes with relatively lower IOPS. ...

Dealing with storage is a core challenge when running complex applications on Kubernetes. While many applications operate just fine using a cloud database or blob storage, some applications have performance or design requirements requiring local storage. Note: For performance benchmarks see [Benchmarking AWS CSI Drivers](<{{ relref “/post/kubernetes/benchmarking-aws-csi-drivers” }}). When this is the case, developers and cluster operators rely on Container Storage Interface (CSI) implementations to provide local storage for Pods. When running on the AWS cloud, no less than four CSI providers are available for us to use: Elastic Block Storage, Elastic File System, FSx for Lustre, and AWS File Cache. This article compares these four different local storage options to help you choose the right option for your application. ...

eBPF is a relatively new extension of the Linux kernel that can run sandboxed programs in a privileged context. It is used to safely and efficiently extend the capabilities of the kernel at runtime without requiring changes to kernel source code or the loading of kernel modules. Because of eBPFs tight integration with the networking stack at the kernel level, it is seeing adoption in networking applications. This includes Kubernetes networking through eBPF implementations of the Kubnernetes networking stack like Cilium. ...

I was recently introduced to a simple article on system design: Users are almost always right. In it, a simple rule is stated: When the users keep doing it wrong, the users are right and your system is wrong. This rule reminded me of the concept of desire paths: unplanned trails created by human or animal traffic that usually represent the shortest or easiest route between two places. Desire paths in an urban setting often run counter to what a planner might suggest, leading to a natural conflict between the pedestrian and the planner. ...

I was recently testing file upload performance, and needed several large files of different sizes to test with. To make the math easier, it was helpful if I had files with round numbered sizes like 10MB, 20MB, or 100MB. After searching around for files of the right size, it turns out the easiest solution is to generate one yourself using the Linux command line. Depending on your needs, you can use two different methods of generating files using some simple commands. ...

A JWT (JSON Web Token) is a type of security token that is used to securely transmit information between parties. It is a compact and self-contained way of representing claims to be transferred between two parties. Many API providers require you to generate a cryptographically signed JWT assertion that includes the authorization you are requesting from the server. If the server accepts the assertions you are claiming in the JWT, it will respond with an access token granting access to the API. ...

Application Portfolio Management (APM) draws inspiration from financial portfolio management, which has been around since at least the 1970s. By looking at all applications and services in the organization and analyzing their costs and benefits, you can determine the most effective way to manage them as part of a larger overall strategy. This allows the architect or engineering leader to take a more strategic approach to managing their application portfolio backed by data. Portfolio management is crucial for creating a holistic view of your team’s technology landscape and making sure that it aligns with business goals. ...

Shortly after deploying a new Kubernetes cluster, one of the first things you will likely want to do is collect some metrics and data about how it operates. There are two projects that are typically used for this, and since they are named similarly it can be confusing to know which one you should use and why. This post hopes to clear up any confusion between the Kubernetes Metrics Server and kube-state-metrics. ...

At some point in your career you will be asked to manage a project. This can be intimidating, it can be scary, but it doesn’t have to be. We can leverage some time-honoured techniques, and adapt them to the unique approach required for software projects to deliver on time, on budget, and with success. This article is a collection of techniques I’ve learned for managing projects over time, that attempts to combine agile best practices with project management best practices. If you study project management to any level of depth, it is inevitable that you will come across the Project Management Institute, or PMI. The PMI is a global professional body that provides training and certification in project management topics. PMI provides different certification options, including some for Agile processes and Scrum. ...