Rachit Pal

About Rachit Pal

Data engineer having 6 years of experience, specialized in Big Data technologies like spark, hive, Hadoop,SQL, Pyspark, GCP

Experience

1. Senior Software Engineer

   Tredence Inc.

   Aug 2024 — Present

   Here I am working as a Data Engineer and I am mainly involved in the migration project where i am responsible to get the 1010 tool code which is in XML( extensible markup language) which I have to convert into the big query and write the stored procedure for that table replicating the same logic used in the XML code for the given table.Then, I have to create the end-to-end pipeline to migrate the data from 1010 to big query and also do the UAT( User acceptance testing) for that table and need to validate the source and target data. The evidences of this UAT needs to be captured in the confluence documentation. The, whole end-to-end pipeline get scheduled and managed with the help of airflow for which I have to create a DAG which get orchestrated by using the cloud composer(Airflow).Also worked on the Omnibot ( a mini chat-bot) in which I moved the data from snowflake to BigQuery and integrated that data with VertexAi 2.5 Flash to generate the data insights.

Education

• Meerut Institute of Engineering and Technology(MIET)

  Bachelor of Technology, C language, Java

  2015 — 2019