Configuring an Azure VNET to use AZTK in mixed mode

Configuring an Azure VNET to use AZTK in mixed mode

In my last post, I showed you how to provision a low-cost Apache Spark cluster on Microsoft Azure, with the help of the Azure Batch service, Low Priority Virtual Machines, and the Azure Distributed Data Engineering Toolkit (AZTK).

But have you tried to mix a cluster with Dedicated-, as well as Low Priority-Virtual Machines?

If you did, you propably run into an error…

Read more

How to create a low-cost Apache Spark cluster on Microsoft Azure

How to create a low-cost Apache Spark cluster on Microsoft Azure

A few months ago, I found a nice little open-source tool on GitHub called AZTK, which provides a fast and easy way to provision low-cost Apache Spark clusters on Microsoft Azure.

In this blog post, I would like to show you, how to install the Azure Distributed Data Engineering Toolkit (AZTK) on your Windows-, Linux- or MacOS-based system, and how to provision your first Apache Spark cluster with it.

Read more