Tejaswini N

About Tejaswini N

I am a Data Engineer with extensive experience in building Big Data systems primarily…

Experience

1. Data Engineer

   Corteva Agriscience

   Dec 2021 — Present · IA, US

   Design and developing the data pipelines using Azure Data Factory, Azure Synapse and Azure Data Lake for Harvesting, planting, Sample manager and Seed Quality ML models and reports. • Successfully completed Data migration from Oracle, Neo4J, Access and SQL Server to Azure. • Used CosmosDB and Storage Tables for creating pipelines using Azure Data Factory. • Design, Develop and deploying ML models using Azure ML. • Creating ETL pipelines, dataflows, data transformations and manipulations using ADF • Strong experience in Python and PySpark for writing code to enhance and optimize data processes. Proficient in developing data transformation routines and data quality checks. • Improved PySpark job performance by optimizing transformations, leveraging Spark\'s caching and persisting mechanisms, and tuning cluster configurations. • Extensive experience in working with data lakes and loading the data in various layers, including raw, staging, curated, and consumption layers. Proficient in managing data at each layer for optimal data processing. • Experience in working with the Delta Lake by creating the Delta Tables and analysing the data.