Divya Gupta
- Role
- Senior Data Engineer at Tata Consultancy Services
- Location
- San Francisco, CA, US
- LinkedIn followers
- 500 followers
About Divya Gupta
Senior Data Engineer | 8+ Years | Retail • E-Commerce • Healthcare Experienced…
Experience
Senior Data Engineer
Mar 2024 — Present
Orchestrating end-to-end migration of healthcare data from on-premises systems to Azure Databricks, leveraging Python, PySpark and SQL to enhance data processing capabilities by 65%.• Architecting and developing robust ETL pipelines in Azure Databricks using Azure Data Factory and Azure Databricks Delta Lake, automating the extraction, transformation, and loading of diverse healthcare datasets.• Implementing advanced data validation frameworks utilizing Azure Data Lake Storage and Databricks Notebooks to ensure the integrity and accuracy of patient data, meeting stringent HIPAA compliance requirements.• Designing and deploying dynamic, real-time Power BI dashboards connected to Azure Synapse Analytics to visualize healthcare data, providing critical insights to clinical and administrative teams for improved decision-making.• Utilizing Azure Boards from Azure DevOps to collaborate within a Scrum framework, coordinating with cross-functional teams to achieve project milestones and deliver high-quality migration solutions on time.• Ensuring data security and compliance by implementing Azure Key Vault for encryption and Azure Active Directory for access controls throughout the migration process.• Engineered custom data connectors and transformation scripts in Python to streamline the migration process and ensure compatibility with legacy on-premises systems like Streamsets.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.