Compute Engine

This article provides a startup script for deploying Kafka to a Google Compute Engine instance. This isn’t meant to be a production-ready system — it uses the Zookeeper instance embedded with Kafka and keeps most of the default settings. Instead, treat this as a quick and easy way do Kafka development using a live server. This article uses Compute Engine startup scripts to install and run Kafka on instance startup. Startup scripts allow you to run arbitrary Bash commands whenever an instance is created or restarted. Since this script is run on every restart, we lead with a check that makes sure we have not already ran the startup script and, if we have, we simply exit. ...

Sometimes you have a data analysis problem that is just too big for your desktop or laptop. The limiting factor here is generally RAM. Thankfully, services like Google Compute Engine allow you to lease servers with up to 208GB of RAM, large enough for a wide variety of intensive tasks. An ancillary benefit of using a service like Compute Engine is that it allows you to easily load your data from a Cloud Storage Bucket, meaning you don’t need to keep a copy of the large dataset locally at all times. ...

A recurring issue when working with Compute Engine is that newly created Instances have only 10GB of free space available. To take advantage of the full disk size you need to manually partition and resize it. This article shows one method of accomplishing this task. ...

Overview The Google Compute Engine core concepts video provides a great overview of the technology. Please take a moment to watch it. As the video shows, Compute Engine is defined by Resources, and each of these Resources is available through the Compute Engine API. You can access this API through the Google API Explorer or by installing the gcloud sdk. ...

Google Compute Engine allows you to make custom images from a running virtual machine. The documentation provides a sufficient example but is a little bit scattered. This article collects and presents all the steps necessary to create your own Compute Engine images that you can use as a base for virtual machines. ...