Karthik Lakshmanan
Senior Consultant (EY) & LEAD Data Engineer (Dell) | Prompt Engineer | Ex-PS | Ex-TCS| Spark | Kafka | Pyspark | Scala | Databrick | SQL | Snowflake | Hive | Oracle | Azure | AWS | Airflow | Tibco | Autosys | Freelancer
- Role
- Senior Consultant (S3) - Technology Consultant - Lead Data Engineer at EY
- Location
- Bengaluru, KA, IN
- LinkedIn followers
- 500 followers
About Karthik Lakshmanan
Over all, I have completed 7+ years of work experience as a Data Engineer. I\'ve worked…
Experience
Senior Consultant (S3) - Technology Consultant - Lead Data Engineer
May 2024 — Present · Bengaluru, IN
Projects: Quote Deal Planning, Discrete Demand, Large Order Planning, and Collab PlanningThese projects are part of enterprise-level solutions that support critical business operations such as understanding customer requirements, handling large order deals, planning supply chain operations, asset allocation, and inventory management.Roles & Responsibilities:Designed and developed multiple generic Spark frameworks using Scala, tailored for modular and reusable pipeline components.Migrated Spark applications from on-premise Cloudera infrastructure to IOMete platform, improving performance and scalability.Converted Spark codebase from Scala to PySpark to align with IOMete platform compatibility and modern coding standards.Executed test plans and led implementation phases, ensuring smooth transitions and minimal production impact.Acted as a team lead, assigning tasks based on priority, reviewing deliverables, and ensuring project milestones were met.Successfully migrated backend from Cassandra database to Oracle, streamlining data access and improving data reliability.Upgraded applications from Spark 2 to Spark 3, leveraging improved performance and feature enhancements.Optimized Scala code to reduce processing time and minimize SLA breaches.Experience in managing CI/CD pipelines using tools like GitLab/GitHub for version control and deployment automation.Hands-on experience with Control-M for scheduling data pipelines and job orchestration.Worked extensively on Shell scripting within edge nodes for automation and task management.Supported development teams across the Software Development Life Cycle (SDLC) with technical expertise and troubleshooting support.
Education
Shanmugha Arts, Science, Technology and Research Academy
Master of Computer Applications
2015 — 2017
Bishop Heber College, Tiruchirappalli - 620 017.
Bachelor of Science - BS
2012 — 2015
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.