Naveen K

Azure Data Engineer @Eli Lilly and Company

Indianapolis, IN, US

Implemented data migration from Hadoop to Azure Data Lake Storage.\\\\n• Used Azure Data Lake Storage Connector to map the current Hadoop environment to Azure\\\\nHDInsight.\\\\n• Created Azure HDInsight for computation power to transfer data from Hadoop to Azure Data Lake •\\\\nImplemented Azure Logic apps workflows for performing event-based data movement.\\\\n• Worked on creating batch pipeline in azure data factory from MongoDB to Azure Synapse Analytics.\\\\n• Integrated ADLS with snowpipe to stream real-time Kafka data to Snowflake.\\\\n• Invoked Azure Databricks for running Spark jobs in Azure Data Factory, to perform complex data\\\\nprocessing and analysis tasks on streaming data.\\\\n• Transformed data from Blob storage to Azure Synapse Analytics using Azure Data Factory.\\\\n• Created many Databricks Spark jobs with PySpark to perform table-to-table operations.\\\\n• Experience in writing SQL queries using Azure SQL DB to retrieve data in Azure Synapse Analytics.\\\\n• Worked on Azure Log Analytics to know the status of Azure services and respond to issues occurring\\\\nin workflows.\\\\n• Automated time-based and event-based triggers using Azure Functions in Azure Data Factory.\\\\n• Improvised analytical insights in Power BI by live connections from databases.\\\\n• Integrated ci/cd pipeline with Azure Git in Azure DevOps to handle repetitive build, test, and\\\\ndeployment tasks.\\\\n• Used Linked services to create a link from data store to Azure Data Factory.\\\\n• Worked on creating Airflow DAGs to automate workflows in pipelines using CRON syntax.\\\\n• Worked on dependencies and operators to re-arrange nodes and process the data.\\\\n• Optimized data storage and reduced costs by using Azure CDN Compression and Azure Archive\\\\nStorage techniques.\\\\n• Improved performance of the database process by indexing, implementing recovery procedures,\\\\nupdating information, managing user access permissions.\\\\n• Experienced in Agile methodology involved in bi-weekly sprints, daily scrum meetings with backlogs\\\\nand story points.\\\\n \\\\n\\\\n \\\\n \\\\n\\\\n \\\\n

University of Missouri-Kansas City

Master's degree, Computer Science