Vinay K.
Senior Data Engineer @Microsoft
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @Microsoft
Toronto, ON, CA
EDUCATION
Coursera
DeepLearning.AI Data Engineering Professional Certificate
Jawaharlal Nehru Technological University
Master's degree
ABOUT VINAY K.
open to work #actively looking for new opportunities Over 7 years of experience as a Seasoned Data Engineer (Analytical Thinker & Collaborative Problem Solver) with a proven track record of designing and implementing scalable and optimized data pipelines capable of handling petabytes of data, across both batch and real-time processing frequencies. Skilled in building end-to-end data pipelines from ingestion to consumption, ensuring data consistency, reliability, and performance at every stage of the data lifecycle. Throughout my career, I have developed a deep understanding of Spark architecture and possess strong capabilities in Spark tuning. Additionally, my knowledge of the Hadoop Framework and its comprehensive ecosystem, including Linux, Map-Reduce, HDFS, Hive, Pig, Sqoop, Flume, and PySpark, has allowed me to drive successful data analysis initiatives. Technical Proficiency: Proficient in Python, Azure services (Azure Data Factory, Azure Databricks, Azure Synapse, Azure Data Lake), AWS services (AWS Glue, Amazon Redshift, Amazon S3, AWS Lambda, Amazon Kinesis), and Snowflake. Consistently deliver scalable and efficient data solutions tailored to business needs. Tech Stack- BigData frameworks: Hadoop, Spark, Hive, Kafka, Databricks - Programming Languages: Python, Java,Scala - Databases: MySQL, Postgresql, MongoDB, Cassandra - Azure Service: Azure Data Factory, Data Lake, Azure Functions, Logic Apps, Stream Analytics,AzureOpenAI - AWS Services: S3, Lambda, EMR, Glue, Cloudwatch, Redshift, SNS, SQS, Athena, Appflow - Monitoring Tools: Airflow
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.