Aditya Narayan Daw
Serving Notice Period | Big Data Engineer | Expert in Talend, Spark, Scala, Amazon Bedrock, Kafka, Python
- Role
- System Engineer at Tata Consultancy Services
- Location
- Kolkata, WB, IN
- LinkedIn followers
- 500 followers
About Aditya Narayan Daw
Developed fault-tolerant frameworks to ingest huge volumes of data from various sources…
Experience
System Engineer
Oct 2021 — Present · Kolkata, IN
Developed fault-tolerant frameworks to ingest huge volumes of data from various sources into theHadoop environment which was 30% more efficient than the previous approach. Performed validation checks on ingested data, such as removing extra new lines and rejecting databased on metadata, before storing it in Hive tables for further processing which reduced the manualfiltering of the data, which reduced the downstream processing time by 70%. Built a micro-batch streaming pipeline using Kafka and Spark to ingest data in Hive after flatteningnested data into a structured format which helped downstream users to perform analytics on the nesteddata efficiently. Helped to migrate a large banking client from traditional Apache Pig based application to Spark basedapplication successfully which not only resulted in reducing the processing time of the data by 20% alsohelped the client to be more flexible with the modern changes in technology. Worked also in an environment as a sole developer of a business team, who communicates with thebusiness users and other teams to work on new POCs and to validate and perform requirements.
Education
University Institute of Technology, The University of Burdwan
Engineer's degree
2017 — 2021
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.