Meghana M
Data Engineer Scientist @Disney Streaming
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer Scientist @Disney Streaming
Spearheaded initiatives to enhance data quality across Disney Plus, Hulu, and Hotstar platforms.• Developed and maintained data quality metrics using PySpark and Databricks, ensuring adherence to standards.• Engineered a streamlined Quality of Experience (QOE) pipeline to efficiently manage and transmit user platform quality scores.• Implemented parallel processing techniques to expedite data checks, enhancing operational efficiency. Led GCP Data Migration from Teradata to BigQuery, ensuring optimized data structures and seamless transitions. Developed ETL pipelines using Cloud Composer, BigQuery, PySpark, and SQL to process and transform high-volume datasets.• Orchestrated seamless data flow from AWS S3 to MySQL database, ensuring reliable storage and accessibility.• Enhanced codebase integrity by eliminating redundant functions, improving consistency and maintainability.• Managed deprecation of outdated jobs using Databricks CLI, keeping infrastructure lean and up- to- date.• Proactively monitored and resolved issues in production pipelines, including data backfilling for failed tasks.• Resolved critical data retrieval issues impacting session runs, ensuring uninterrupted analytics processes.• Conducted regular meetings with stakeholders to gather requirements, provide updates on data projects, and align on project goals.• Demonstrated project progress and results through stakeholder demos and presentations.• Leveraged Metabase dashboards for effective data visualization and analysis.• Designed and implemented robust CI/CD pipelines for code, automating deployment and testing of data processing workflows.• Utilized AWS S3 for efficient data loading, SQL tables for structured data storage, Metabase for comprehensive data visualization, and PySpark/Databricks for advanced data processing.
EDUCATION
JNTUH College of Engineering Hyderabad
Bachelor's degree
ABOUT MEGHANA M
Experienced IT professional with around 11 years as a Senior Data Engineer, I specialize in designing, developing, and maintaining data-driven solutions on various cloud platforms. My expertise spans data technologies, cloud services, and data modeling, enabling effective collaboration with cross-functional teams to deliver valuable insights. Proficient in a range of services and technologies, including ETL tools like PySpark and Kafka, data storage solutions, and Hadoop distributions, I excel in designing efficient and scalable data transformations across diverse environments. I have hands-on experience in Python and Spark scripting for data analysis, warehousing, mining, and ETL transformations. Additionally, I am skilled in scripting languages like Linux/Unix shell and Python, implementing CI/CD using tools like Jenkins for seamless deployment.Leveraging visualization tools such as Tableau, I ensure efficient data visualization and analysis. Proficient in both Waterfall and Agile methodologies, I contribute to the SDLC, covering requirements analysis, design specification, and testing. My experience equips me with versatile skills to address diverse challenges and deliver robust data solutions meeting business needs across various cloud providers.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.