Satish Nandhigum
Data Engineer @easyJet
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @easyJet
Luton, GB
Responsible for the delivery of data ingestion pipelines for loading data into the enterprise data lake stored in HDFS from various sources like SQL Server, AWS S3 buckets and FTP Servers.• Responsible for the delivery of a metadata driven framework to load data into the enterprise data lake stored in HDFS and parse the raw data based on various input file patterns and persist a refined version of the data set with using performance optimization techniques to be consumed by downstream systems and business users• Developed Spark Programs in Python to perform data transformation, creating data frames and running Spark SQL. Worked on creation of batch and real-time data for data ingestion and data products.• Responsible for surfacing the data stored on HDFS in Hive using both internal and external tables with applying performance optimization techniques like partitioning and parquet compression for end user consumption.And upload the metadata information into Cloudera Navigator using API’s and Scala.• Creation of oozie Xml’s and executing workflows and coordinators on the Cloudera platform.• Developed a generic data ingestion framework of loading data into a temporary landing area of the data lake stored in HDFS from Amazon S3 Buckets. By creating generic but modularized bash scripts, data ingestion pipelines remained flexible.• Responsible for the delivery of Oozie workflows and coordinators to schedule and execute end to end ingestions of data pipelines with features like email notification in case of failure and parallel action execution for performance improvements.• Responsible for the delivery of knowledge transfer sessions on structural and technical aspects of the end to end data ingestion and deployment process to other members of the delivery team and also using on CI/CD environment using Jenkins and GIT Reprository for Code merging • Creation of Single property file to remove the duplication of multiple config files across all the environments.
EDUCATION
Bangalore University
Master of Science - MS, Computer Science
University of Essex
Master's degree, science
ABOUT SATISH NANDHIGUM
Profile:Professional IT Experience with rich knowledge of Big Data Technologies in Aviation, financial and retail domain. Designed End to End Big Data flows focused on delivering data pipelines to ingest data from various sources into an Enterprise Data Lake by leveraging tools from the Hadoop ecosystem. Involved in most stages of the development life-cycle such as analysis, design, implementation and testing. Excellent BI development skills with SSIS, SSAS and SSRS. Strong production DBA skills with deep understanding of SQL Server 20••••16, DR and HR techniques and end to end BI project implementation using SSIS, SSAS and SSRS. Good knowledge on RDBMS concepts. And good understanding of Data Warehousing concepts, strong exposure to dimensional modelling and Datawarehouse design principles. Very good understanding of Agile/Scrum methodologies. Certified Scrum Master and Scrum Developer from Srum.org.Possesses excellent communication and interpersonal skills and enjoys the challenge of designing and developing assignments, strongly motivated by achievement and committed to delivering a high-quality reports which not only meets but also exceeds the clients expectations. A successfully communicative team player who puts emphasis on collaboration and also possesses strong learning and problem-solving skills.Seeks contract work being immediately available for both interviews and starts in London and across the UK and Europe.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.