Get your Emails warmed up and ready to send
Build sender reputation and reach more inboxes with automated email warmup.
Warm up your emails
Starting at $49/month
Manohar R
Senior Data Engineer @CVS Health
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @CVS Health
Building streaming pipelines in Databricks using Pyspark and involved in establishing deployment processes for structure streaming.• Involved in migrating existing pipelines from AWS Glue studio to Databricks Workflows.• Built ETL pipelines in DataBricks using CDC and migration from hive metastore to Unity Catalog.• Involved in security design review in Databricks and shifting from Table ACLs to Unity Catalog.• Built dashboards, SQL alerts in Databricks and did integrations to 3rd party alerting systems such as PagerDuty.• Performed data analysis and developed analytic solutions. Data investigation to discover correlations / trends and the ability to explain them.• Worked with Data Engineers, Data Architects, to define back-end requirements for data products (aggregations, materialized views, tables – visualization)• Developed frameworks and processes to analyze unstructured information.• Onboarded new ETL tool Fivetran for extracting data from different data sources.• Performed batch ingestion of the data to Amazon RDS using PySpark by creating spark sessions and writing scripts to load the data to RDS.• Performed aggregation, sub-queries, and window functions on the stored data using AWS Glue Data• Catalog\'s query editor to generate Batch Views.• Utilizing Glue Console created a Glue job to generate batch views, transformed the data into batch views, and performed data mapping, cleaning, and aggregation using Glue ETL.• Proficient in utilizing AWS Batch in designing and implementing batch processing workflows, to automate and manage batch computing workloads, ensuring high-performance computing.• Created Athena tables for batch and speed layers, created an external table to merge both tables and stored them in S3 using Athena (serving layer).• Used Athena to query data from batch and speed layers and performed ad-hoc analysis to identify potentially fraudulent activities quickly.
EDUCATION
University of Mumbai
Bachelor of Engineering - BE, Information Technology
ABOUT MANOHAR R
I am a Senior Data Engineer at CVS Health, where I design and implement data pipelines to…
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.