Amit Londhe

Lead Software Engineer @G2O

Columbus, OH, US

Services - Built Airflow application that runs in AWS MWAA environment. As part of conforming to NIST standards, hosted the workflow in an AWS Island type of environment that does not have internet connectivity. Came up with the firewall exceptions and configuring them in AWS VPNs so that Python DAG can run on MWAA. Airflow communicated securely with backend NoSQL database, Marklogic to execute various tasks.Used K8 for on-prem implementation and installed Airflow backed by Postgres installed using Crunchydata Postgres Operator.* Core Platform - Next generation data hub based on Data Mesh architecture was built using AWS Services such as EMR, IAM, S3, Kinesis, EC2, Step functions, Dynamo DB and Lambda Functions. Used CDK and Cloudformation to stand up the platform in AWS.Stateless processing was done using Spark whereas used Apache Flink for stateful stream processing. Later on refactored the system to run on premise using K8 cluster. In K8 universe, employed MongoDB, Kafka and Spark Operator build data pipelines. AWS Lambda features are redesigned using Mongo DB Streams and Spark Streaming jobs. Used Helm as IaaC for building K8 applications.Argo CD used for CICD. Argo Workflows in place of AWS Step function based scenarios. REST Endpoints to access Data Mesh solution built using Spring Boot APIs. Developed multiple readers/writers using Spark Datasource V2 APIs.* Enterprise Data Hub - Built highly scalable and streaming Data pipelines using Spark, Kafka on CDH ( Cloudera cluster). Batch and streaming pipelines process TBs of data to deliver curated content for product\'s consumption. Operationalize these pipelines using Jenkins DSL.* As an individual contributor in a team, worked on building a search platform using Hadoop and Cascading for creating Solr/Lucene Index. Also use Apache Pig for Ad-hoc data analysis. Follow the Agile Scrum methodology and Acceptance Test Driven Development (ATDD) using Ruby, Cucumber and Gherkin.

Fergusson College

12th

New English School Ramanbaug,Pune

10th, School

Amrita Institute of Advanced Computing,Coimbatore

DAC, Computer Software

Vishwakarma Institute of Technology,Pune

B.E, Mechanical Engineering