Akshay S

Data Engineer @Emids

Participated in Agile Scrum meetings as a Data Engineer to support, manage, and work with a team of developers, analysts, and QA. Consistently took part in enhancement talks, maintenance of present systems and pipelines, testing, and bug-fixing activities. Together with the business analysts, converted business requirements into technical requirements and produced low-level and high-level documentation. With the aid of PySpark jobs and Oozie, a data workflow on-premise Hadoop system was designed and installed for the purpose of orchestrating the workflow for new clients. Used shell scripting to trigger Oozie workflows and FTP/SFTP the files within the directories. Modified existing workflows to implement the delta lake feature to reduce writing the entire data back on hive tables