Rakshita Tarigopala

Data Engineer @Johnson & Johnson

US

Developed automated AWS Glue jobs to ingest data coming from SFTP vendor systems. Implemented Crawler and catalog tables to automatically manage the Schema changes of incoming files.• Involved in designing and deploying multitude applications utilizing almost all AWS stack (including EC2, S3, Route 53, RDS, SNS, SQS) focusing fault tolerance and high availability in AWS CloudFormation.• Optimized volumes and EC2 instances using API Gateway. Integrated Datadog to AWS Cloud using IAM roles and Lambda functions to Route 53 level logs and metrics directly to Splunk heavy forwarder.• Data ingestion modules utilizing Python modules, AWS Step functions, and AWS glue. To implement data transformation logics on AWS, PySpark Glue tasks were created, and the output was stored in a Redshift cluster.• The raw datasets of transactional verticals are merged based on business criteria using Spark to create a master dataset for building Dashboard. Used Python UDFs to implement business logic and the UDF are embedded into MAP transformation using PySpark.• Worked with Spark SQL context to create data frames to filter the input data for model execution and also performed transformations on the data frames.• Implemented persistence layer using Spring JDBC template to store the data in oracle cloud and also used Spring JDBC template with hibernate to access the data from the database.• Translate java based ElasticSearch and MongoDB ingest modules to spring boot.• Experience with different file formats – Avro, Parquet, JSON, XML etc.• Created Data validation scripts that would be used for verification correctness of ETL logic and transformations between source system and dimension tables.• Enhanced the vendors ETL frameworks to ensure a more dynamic and parameterized design to extract data from source tables using ETL config tables.• Performed EDA using Python and SQL to extract and extrapolate vital.