Cloud Infrastructure for Data Analytics Platform

Country

Greece


Description

The client is an early-stage startup with a data analytics platform providing a highly customizable Enterprise alternative to Google Analytics.


Challenges

  • Strict budget: the client came with an MVP that had yet to get investment

  • Expertise: the client had a clear concept for the cloud setup they wanted but did not have the engineering expertise to pull it off

  • Data safety and GDPR compliance: it was crucial to use only European providers on all levels and store customers’ data in Europe only to achieve certification later on

  • Easy management of the stack: the infrastructure had to be easily manageable for the end customers. One of the considered business strategies was to make the product open source, and the client wanted to be able to provide a quickstart configuration for administrators


Solution

Together with the client, we picked Hetzner as a cloud provider, it was a rational and budget-friendly choice at the time.

The application itself consisted of a number of container workloads, and the client wanted Nomad for the orchestration. We had extensive previous experience with Nomad, and considering the scale, it was a great choice and a breath of fresh air from our more common Kubernetes-powered setups.

Since there was no managed service in Hetzner to host Nomad (or any public cloud, for that matter) we implemented the entire cloud setup with Infrastructure as Code using Hashicorp toolkit: Packer to build instance images for Nomad components, Terraform to provision VMs from Packer-built images and orchestrate Nomad cluster assembly. The eventual experience is coherent enough to feel like using a managed service: set a number of services, toggle High Availability (we run staging without failover, and production as an HA cluster) and apply the code to get full networking setup with a production-ready Nomad cluster in Hetzner.

BunnyCDN is used for DNS and CDN/reverse proxy, which can cache data but is GDPR compliant, so legal matters were covered. To get these key features from both platforms (caching and automatic TLS services from BunnyCDN, as well as DNS management), a DNS records chain was established.

As usual, the client received an easy-to-manage infrastructure covered with detailed documentation, and a week of warranty period to boot.

A bit of statistics from the final setup:

  • time to deploy the entire stack from scratch in a new region is 16 minutes

  • infrastructure updates via provisioning of the machines from rebuilt images takes 7 minutes

  • TLS security grade: A+ (based on Qualys SSL labs test)

CONTACT US