Shagoul Hameed

Experience

1. Lead

   Kantar

   Sep 2020 — Present · Hyderabad, IN

   Designed and optimized scalable data pipelines using Azure Data Factory (ADF), Dataflows, Databricks, improving data processing efficiency by 40%. • Developed and managed ETL/ELT workflows to ingest data from APIs, flat files, databases into Azure Data Lake Storage (ADLS Gen2) and Delta Lake.• Implemented Medallion Architecture to transform raw data into curated Gold-layer insights.• Optimized Databricks cluster configurations, reducing execution time and memory overhead for large-scale processing.• Built real-time streaming pipelines using Auto Loader and Structured Streaming for incremental data ingestion.• Developed CI/CD pipelines using Databricks Repos, GitHub Actions, and Azure DevOps, enabling seamless deployment of data workflows.• Utilized dbt to build modular, reusable transformations on top of Delta tables, improving data modeling and maintainability.• Implemented SCD Type 1, Type 2, and Type 3 strategies to track historical changes in dimension tables, ensuring data integrity.• Fine-tuned Databricks cluster configurations, optimizing memory management and autoscaling for large-scale data processing workloads.• Collaborated with cross-functional teams to design data models, ensuring high data availability, lineage tracking, and governance in Unity Catalog.• Developed Python scripts to automate data ingestion, extracting data from internal/external APIs and storing it in ADLS.• Orchestrated data workflows using Azure Data Factory and Apache Airflow, scheduling incremental loads and automating dependencies.

Education

• SNMV College of Arts and Science

  Diploma of Education, WEB TECHNOLOGY

• Shri Nehru Maha Vidyalaya College of Arts & Sciences

  Bachelor of Science (BSc), Information Technology

  2007 — 2010

Skills

• Analysis
• Data Processing
• Research
• Statistics
• Css
• Survey Design
• Market Research
• C
• Programming
• Competitive Analysis
• Market Intelligence
• Advertising Research
• Html
• Vendor Management
• Survey Research
• Data Analysis
• Data Collection
• Segmentation
• Sql
• C++
• Business Analysis
• Crm
• Databases
• Quality Control
• Primary Research
• Qualitative Research
• Javascript
• Project Management
• Business Development
• Online Media Management
• Java
• Market Analysis
• Customer Insight
• Focus Groups
• Team Management
• Questionnaire Design
• Confirmit
• Analytics
• Data Mining
• Quantitative Research