Virender Singh Deshwal
Architect - Spark Developer @Virtusa
Signup · Get unlimited contacts
WORK HISTORY
Architect - Spark Developer @Virtusa
Pune, IN
Client: CitigroupProject: GLRS is a standardized platform which caters to all the liquidity reporting needs for the Corporate Treasury and FRI as a part of the Treasury Liquidity Monitoring and Reporting program.The GLRS 2.0 application is designed on the Hadoop ecosystem. This project is developed for Investment banking, we are getting everyday EOD reconciliation files from different processing origination, which contains Transactions data in pipe separated file in compressed files. As part of data Ingestion, Our Spark program read files from RDBMS and load into hdfs then Enrichment program read those data and load into a Hive table. If we didn’t get EOD data on time then do root cause analysis and fix the issue. Once data store in HIVE then get those data in Spark/Scala and make DataFrame and do analysis and make various report and store in HIVE table.Responsibilities:• Developed Archival Ingestion code in spark to load historical data.• Developed Daily Ingestion code in spark to read data from oracle and validate then post validation load into hive table.• Developed Schema validation code which generate report by comparing schema between oracle and hive tables.• Deployed Spark Code through RLM.• Responsible for developing scalable distributed data solutions using spark.• Involved in performance tuning of spark applications for fixing right batch interval time and memory tuning.• Responsible for troubleshooting and resolving the issues related to the performance of Hadoop cluster.• Partitioned hive tables for improving the query performance.
EDUCATION
Jiwaji University
BCA, Computer Programming, Specific Applications
Sanghvi Institute of Mnanagement & Science
MCA, Computer Application
ABOUT VIRENDER SINGH DESHWAL
Total 12+ Years of experience in IT and 5 Years in Spark and Hadoop• Expertise in Hadoop echo systems - HDFS, Sqoop, Hive, Impala, Shell script.• Expertise in Spark (Scala & Python) for scalability, distributed computing and high performance computing.• Extensive experience in Agile methodology and Software development life Cycle.• Expert in SDLC Tools like Confluence, Jira, GIT, SVN & HP Quality Centre.• Worked for reputed client Citigroup Inc. Gained work experience at Client place Citi group, Belfast, Northern Ireland, UK and Singapore.• Quick turn around on Defect fixes in both Prod and Non Prod environment including hot fixes.• All around experience from Requirement Gathering, Development, Testing, UAT and Implementation of different software components.• Good experience in Project/Team management and also worked on different RFP’s.• A Team Player, Keen Planner & Implementer who has significantly helped project execution.• Self-motivated and ready to learn new Technologies.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.