Priyannk S.

About Priyannk S.

I am a skilled Big Data engineer with 5+ years of experience designing and implementing large-scale data processing systems. My expertise lies in leveraging cutting-edge tools and technologies to build robust and efficient data pipelines that can handle massive volumes of data.Throughout my career, I have worked with a wide range of data technologies, including Hadoop, Spark, Kafka, Cassandra,SnowFlake,HBase, Kinesis and Elasticsearch. I am well-versed in programming languages such as Java, Python, and Scala, and have a strong understanding of distributed computing, data modeling, and data warehousing.In my previous roles, I have designed and developed data processing solutions that have enabled organizations to extract valuable insights from their data, improve decision-making processes, and drive business growth. I have also collaborated with data scientists and other stakeholders to identify requirements and implement solutions that meet their specific needs.I am passionate about staying up-to-date with the latest developments in the Big Data ecosystem and enjoy exploring new technologies and tools to improve my skills and knowledge. I am a strong communicator and collaborator, and thrive in fast-paced, dynamic environments where I can leverage my expertise to solve complex data problems.Overall, I am a dedicated and results-driven Big Data engineer with a proven track record of delivering high-quality solutions that enable organizations to unlock the full potential of their data.

Experience

1. Data Engineer - Iii

   dunnhumby

   Nov 2023 — Present · Gurugram, IN

   Designed and developed terabyte-scale ETL pipelines on Databricks and PySpark, transforming structured and semi-structured data to power customer and retail analytics.Optimized Spark workloads through effective partitioning, caching strategies, and broadcast joins, cutting runtime of critical data pipelines by up to 60%.Implemented Delta Lake for reliable data management with schema enforcement and time travel, enabling safe and auditable data updates.Built near real-time streaming pipelines using Kafka and Spark Structured Streaming, supporting analytics and insights delivery with <5-minute latency.Automated end-to-end pipeline deployment with Databricks Workflows, GitHub Actions, and parameterized notebooks, improving release efficiency and consistency.Migrated legacy on-premise data workflows to Azure and GCP, improving scalability and reducing infrastructure costs by 25%.Tuned and optimized complex SQL transformations, boosting query performance and data retrieval speeds by 50%. Used Stored Procedures here.Led a team of 3 data engineers, guiding code reviews, architectural design decisions, and establishing best practices for testing and continuous integration.

Education

• ITS Engineering College, Greater Noida

  Bachelor of Technology - BTech, Computer Science

  2014 — 2018

• Kendriya Vidyalaya

  SSC

  2006 — 2014