Meghana Reddy

Azure Data Engineer @Broadridge

Highly Involved into Data Architecture and Application Design using Cloud and Big Data solutions on Microsoft Azure.\\\\n• Leading the effort for migration of Legacy-system to Microsoft Azure cloud based solution. Re-designing the Legacy Application solutions with minimal changes to run on cloud platform.\\\\n• Built the data pipeline using Azure Service like Data Factory to load the data from Legacy SQL server to Azure Data Base using Data Factories, API Gateway Services, SSIS Packages, Talend Jobs, custom.Net and Python codes.\\\\n• Built Azure Web Job for Product Management teams to connect to different APIs and sources to extract the data and load into Azure Data Warehouse using Azure Web Job and Functions.\\\\n• Build various pipeline to integrate the Azure Cloud to AWS S3 to get the data into Azure DataBase.\\\\n• Set up the Hadoop and Spark cluster for the various POCs, specifically to load the Cookie level data and real-time streaming. Integrate with other ecosystems like Hive, Hbase, Spark, HDFS/DataLake/Blob Storage.\\\\n• Set up the Spark Cluster to process the more than 2 Tb of data and dumped into SQL Server. In addition, built various Spark jobs to run Data Transformations and Actions.\\\\n• Writing a different APIs to connect with the different Media Data feeds like, Prisma, Double Click Management, Twitter, Facebook, Instagram and Amnet to get the Data using Azure Web Job and Functions integration with Cosmos DB.\\\\n• Built the trigger based Mechanism to reduce the cost of different resources like Web Job and Data Factories using Azure Logic Apps and Functions.\\\\n• Extensively worked on Relational DataBase, Postgres SQL as well as MPP database like Redshift.\\\\n• Experience in Custom Process design of Transformation via Azure Data Factory & Automation Pipelines. Extensively used the Azure Service like Azure Data Factory and Logic App for ETL, to push in/out the data from DB to Blob storage, HDInsight - HDFS, Hive Tables.\\\\n \\\\n\\\\n \\\\n \\\\n\\\\n \\\\n