Pros
Spearheaded the redesign of Azure-based ETL/ELT pipelines using Azure Data Factory and Azure Data Lake Storage (ADLS) to process 10M+ records daily, improving data throughput and scalability by 40%. • Designed and implemented a star schema enterprise data warehouse in Azure Synapse Analytics, reducing report generation time by 50% and enabling faster analytics consumption. • Built real-time data streaming pipelines integrating Apache Kafka with Azure Databricks lowering processing latency by 35% and supporting distributed workloads over 20TB of data. • Developed and integrated REST APIs to ingest external financial data into Azure Data Lake, improving cross-system data availability and reducing insurance policy issuance time by 30%. • Partnered with analytics teams to build SQL-based transformation and data modeling layers in Azure Synapse, developing DAX measures and data modeling practices to deliver analytics
Cons
Client requirements were always vague