Aditya Lingangari

Sr Data Engineer @Liberty Mutual Insurance

Dallas, TX, US

Developed end-to-end data pipelines using Spark, Hive, Airflow, and Snowflake to support analytics across insurance and retirement/401K systems.Utilized Informatica for ETL workflow creation, transformation logic, and performance optimization with Snowflake and on-prem sources.Designed and optimized dimensional models (Star and Snowflake schemas) for reporting and data warehouse performance.Built real-time streaming solutions using Spark Streaming, Kafka, Scala, and NoSQL stores (HBase, Cassandra) with storage in HDFS.Created batch processing scripts using Spark-Scala and PySpark integrated with Databricks, S3, and PostgreSQL for data transformation.Utilized C#.NET modules to integrate external reporting tools and support legacy system compatibility.Tuned Snowflake queries and optimized resource consumption using clustering keys, partitions, and caching.Developed Sqoop and Pig jobs for ingesting structured/unstructured data from RDBMS into Hadoop.Managed infrastructure using AWS (EC2, S3, EMR, Redshift, DynamoDB, RDS) and leveraged CloudWatch, ELB, IAM for monitoring and security.Built interactive dashboards and operational reports using Power BI and Tableau to support stakeholders with insights into retirement and 401K data.Collaborated with business teams to gather requirements, map data sources, and deliver actionable insights—applying strong Business Analyst skills.Automated job orchestration using Apache Airflow and Oozie, managing dependencies across data ingestion, cleansing, and transformation tasks.Supported high-throughput applications by building scalable Spark jobs and optimizing performance for large-volume datasets.Worked in Agile and Waterfall environments, delivering sprint-based solutions while ensuring quality, documentation, and reproducibility.