Vijay C
Senior Data Engineer in LPL Financial
- Role
- Senior Data Engineer at LPL Financial
- Location
- Austin, TX, US
- LinkedIn followers
- 500 followers
About Vijay C
With over 8 years of experience in IT, I specialize in developing advanced data solutions within cloud environments, notably AWS and Azure. My expertise spans architecture, design, and implementation, with a focus on Hadoop frameworks and associated technologies. Proficient in Python, PySpark, and SQL, I excel in crafting robust ETL pipelines and driving insights through analytics. Holding certifications as an AWS Certified Data Engineer Associate and Microsoft certified Azure Data Engineer Associate, I remain committed to staying updated with the latest industry standards.In every project, I bring a unique blend of technical proficiency, strategic insight, and an unwavering dedication to excellence. From orchestrating seamless data migrations to optimizing cloud-based data systems, I ensure optimal outcomes and drive organizational success through innovation and efficiency.
Experience
Senior Data Engineer
May 2022 — Present · Austin, TX, US
As a Data Engineer at LPL Financial, I spearheaded the design and optimization of Hadoop-based applications utilizing Spark and Kafka for efficient data processing. Within AWS, I managed a comprehensive data stack including Snowflake, DynamoDB, S3, and Spark SQL, proficiently handling data modeling tasks. My expertise in Python scripting facilitated seamless integration for API calls, web scraping, and data processing, while PySpark proficiency enabled high-performance analysis. I engineered real-time data streaming solutions with Spark Structured Streaming and Scala, managing AWS services like EMR, EC2, IAM, Lambda, and S3 for optimized processing and storage. I developed PySpark code for AWS Glue jobs and EMR instances, streamlined workflows, and orchestrated Docker container deployments with Kubernetes. Moreover, I designed Lambda handlers for efficient data processing in Amazon EMR and implemented ETL processes using Informatica Cloud. My experience with Snowflake, coupled with skills in building scalable Data Pipelines and implementing streaming workflows, ensures real-time data processing and compliance with data standards. Additionally, I possess familiarity with various build management and version control tools like SBT, Maven, GIT, and have hands-on experience with Apache Flink, Apache Beam, and Scalding frameworks.
Education
Chaitanya Bharathi Institute Of Technology
Bachelor of Engineering - BE, Chemical Engineering
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.