Harsha Vardhan
Data Engineer
- Role
- Data Engineer at Palantir Technologies
- Location
- Cleveland, OH, US
- LinkedIn followers
- 500 followers
About Harsha Vardhan
Data Engineer with 5+ years of experience in designing and building cloud-native, enterprise-scale data platforms and real-time streaming pipelines across finance, healthcare, and aerospace domains. Expertise in Apache Spark, PySpark, SQL, Python, Kafka, Palantir Foundry, and event-driven architectures to deliver high-performance ETL/ELT pipelines, ML feature engineering pipelines, and feature stores. Proven ability to optimize performance, reduce latency, and lower cloud compute costs while supporting fraud detection, risk scoring, and advanced analytics at scale. Strong hands-on experience with AWS, Azure, CI/CD, Docker, Kubernetes, and Infrastructure-as-Code (IaC). Demonstrated success in enterprise system integration, data governance, data lineage, metadata management, and regulatory compliance (SOC 2, HIPAA, FedRAMP, ITAR). Effective collaborator and mentor with a track record of delivering reliable, secure, and scalable data solutions.
Experience
Data Engineer
May 2024 — Present · NY, US
Integrated enterprise data from SAP ERP, Oracle WMS, Snowflake, and AWS S3 into a unified data ontology, enabling supply chain analytics across 300+ distribution centers and 1,500+ retail locations. • Built real-time data streaming pipelines using Apache Kafka and Spark Streaming, reducing inventory and out-of-stock alert latency from 45 minutes to under 10 minutes. • Engineered a Python-based semantic data mapping framework using OpenAI embeddings and FAISS, reducing manual mapping effort by 40% and accelerating onboarding timelines by 2–3 weeks. • Designed and maintained 100+ ETL/ELT pipelines using PySpark and SQL, processing 2 TB of daily data with 99% pipeline reliability and automated monitoring/alerting. • Implemented a centralized data transformation layer using dbt Core and Snowflake, creating reusable SQL models and data quality tests that reduced reporting errors by 30% across 8+ business domains.• Developed scalable Foundry Functions and REST APIs handling 20K+ daily requests, implementing authentication, rate limiting, and retry logic for downstream applications and analysts.
Education
Cleveland State University
Master of Science - MS, Information Technology
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.