I have been a senior engineer with Uber for over 2 years. I have had a history of being part of high velocity teams, and using functional languages to build systems that perform at scale and correctness. Data engineering with functional languages is a field that I did some work on recently. LinkedIn: @anmshkmr
In order to create a canonical framework for real-time and batch data processing in the order of 10s of TBs per day, a general purpose framework to instantiate and extend data pipelines is presented here that can be used to ingest, process, and derive ML based inferences from the base data.