Bilal Shaikh

Experience

1. System Engineer

   Tata Consultancy Services

   Feb 2022 — Present · Pune, IN

   Developed and optimized ADF pipelines to ingest and process healthcare claim data from REST APIs, JSON, SQL, and CSV sources—improving data integration efficiency by 20%.• Designed a Medallion architecture (Landing → Bronze → Silver → Gold) using Azure Data Lake Storage Gen2 and Delta Lake, reducing data retrieval time by 30% and enhancing data traceability.• Built metadata-driven ETL pipelines in Azure Data Factory for dynamic ingestion of structured and semi-structured data using config-based orchestration.• Created PySpark transformation pipelines in Azure Databricks with incremental loading and parallel processing to handle high-volume EMR and claims data.• Implemented SCD Type 2 logic in dimension tables for historical tracking, supporting time-based analytics and audit requirements.• Standardized schemas across disparate datasets (EMR, claims, NPI) to establish a Common Data Model (CDM) in the Silver layer, ensuring consistent and analytics-ready data.• Applied complex business transformations (joins, filters, aggregations) and supported MDM processes to ensure data quality and completeness.• Secured pipelines with Azure Key Vault, managing sensitive credentials in compliance with HIPAA standards.• Integrated GitHub CI/CD for version control, automated deployments, and collaboration, improving release reliability and traceability.• Used Python Faker to generate synthetic data for testing, ensuring robustness and reliability of pipeline logic before production deployment.

Education

• Swami Ramanand Teerth Marathwada University (SRTMU), Nanded

  B.E

  2017 — 2020