Naveen S

Experience

1. Data Engineer

   Verizon

   Nov 2024 — Present · Tampa, FL, US

   Migrated on-prem Hadoop workloads to AWS-based Databricks, reducing data processing time from 6 hours to 45 minutes using Photon Engine and Delta Caching.Built scalable ETL frameworks with AWS Glue, Lambda, and Step Functions, cutting infrastructure costs by 40%.Developed high-throughput pipelines using Apache Iceberg, Trino, and Spark, supporting petabyte-scale analytical workloads.Enabled real-time fraud detection with Kafka, Spark Structured Streaming, and Snowflake, delivering sub-second analytics.Designed secure Snowflake environments with RBAC, row-level security, and data masking, ensuring GDPR/HIPAA compliance.Automated ELT orchestration using Airflow, dbt, and Snowflake Tasks, reducing data refresh latency by 85%.Built self-healing, metadata-driven pipelines using Python, Jenkins, and GitHub Actions, achieving 99.99% reliability.Implemented a data quality framework with Great Expectations and PySpark, automating validation across 100+ sources.Delivered actionable insights through MicroStrategy dashboards and reports, directly integrated with AWS-based data pipelines.Supported business reporting by delivering trusted datasets to MicroStrategy, Power BI, and Tableau, aligning technical solutions with business KPIs.Drove FinOps initiatives, optimizing Snowflake and Databricks usage to save $1.5M annually via auto-scaling and cost tracking.Wrote complex SQL and PL/SQL queries for data validation, anomaly detection, and reconciliation across staging and reporting layers.Prepared test plans, test cases, and defect reports, collaborating with UAT and business teams for sign-off.