I’ve started working on some of my free time in a project to build the smallest useful analytics platform on the cloud (starting with azure).
The purpose is to use it a sa PoC to show to colleagues, managers, prospective customers or just to have fun and play
It’s publicly available on my github repo and any collaboration is welcome. You can fork it, improve it, send PR’s and do whatever you want!
The first version will run solely on azure. The objective is to show the following technologies/disciplines:
* Infrastructure as a Code (IaaC), by using Terraform
* Cloud architecture anc Cloud Ops by using an azure cloud environment
* Data Engineering by using a Spark powered Databricks Notebook and an ADF Pipeline (future)
* DevOps to trigger some pipelines based on changes (future)
* Basic Security concepts (keyvault, service principals, least privileged rbac accesses…)
* FinOps keeping the costs at minimum and choosing the proper tools for the job
* Reporting and Dashboarding on data in the platform
* Data management: We will use an adls storage account and azure sql db
TOOLS:
* Terraform to deploy all the infra as a code
* Azure Cloud to host our resources
You have the code plus all the information on my github repo:
