Mrunmaya Dash

Associate Consultant @Tata Consultancy Services

Kolkata, IN

Role (Client: Amgen, California): As AWS & Bigdata Developer (Amgen EDL 2.0 – Enterprise Data Lake) As a part of Migration of EDL 1.0 to EDL 2.0/Plant Simulation and Scheduling/eTMF • Attended multiple scrum calls to finalize requirements with client • Prepared Technical Design Specifications and Design Deocuments • Analysed Data Ingestion to Enterprise Data Lake (EDL 2.0) from Teradata through EMR Sqoop • Transformation of data in EDL working layer and move to Publish layer • Made Data available for customer in Redshift Spectrum with data located in S3 • Used S3 storage for each layer data store in EDL 2.0 • Symantic layer data moved to Redshift for reporting purpose once available in spectrum • Used Databricks Python notebook, for Data transformation through Pyspark • Used Airflow scheduler to schedule the Jobs • DAG creation for Airflow using Python • Used Rest API call through PythonOperator in DAG • Used BashOperator, BranchPythonOperator, DummyOperator, TriggerDAGRunOperator etc inAirflow DAG code • Debug and Complete Unit testing during project phase • Optimizing Jobs to use S3 by using various compression mechanisms like Avro, Parquet format • Configured periodic Incremental data ingestion in EDL 2.0 along with full refresh • Developed IV(Infrastructure Verification) and IQ(Infrastructure Quality) Script for System changes • Managed system administration with Unix system development and maintenance through EDL manager • Attended daily scrum calls and worked in agile methodology to deliver project • Used Jeera for Task assignment during Agile delivery • Code deployment through CI/CD using Jenkins Environment: AWS(Amazon Web Service), Rest API, EMR Sqoop, S3, Databricks, Redshift, Spectrum, Python, Spark, Unix, Airflow, Crontab, Databricks Notebook, Agile, Jenkins, Jeera, RDS Oracle, RDS postgreSQL

Bijupattnaik Utkal University

Bachelor's degree, Computer Science, 1st Class