Gopavarapu Divya
Senior Software Dev Engineer @Verizon
Signup · Get unlimited contacts
WORK HISTORY
Senior Software Dev Engineer @Verizon
Developed Spark(Scala & Pyspark) ETL pipeline jobs, ingestion streaming applications to migrate historical and incremental data load from different source file formats on distributed environment * Involved in requirements gathering, analysis, design, coding, code reviews and unit and integration testing* Done Extensive end to end Analysis on input feeds which includes cleaning,cleansing and Structuring raw data, data ingestion, data integration and mining data, also involves deriving the value gain from given Raw data * Experience in Executing Spark on Cloud Dataproc, optimising dataproc storage, Templates and Dataproc Autoscaling * Migrated on premise spark ETL pipelines to google cloud platform such as Dataproc, Bigquery,cloud storage* Worked on reading and writing data from bigquery to spark dataframes and hive tables for advanced ETL capabilities* Developed a framework to generate daily adhoc reports and extracts from big query enterprise data, also generate daily email alerts upon ETL job failures* Reducing data processing by refactoring existing ETL pipeline and identifying common components. One of refactoringreduced 30% of preprocessing time and reduced 70% network traffic involved in that task* Independently developed End to End spark scala applications as per business requirementsExperience in developing and scheduling ETL workflows with oozie* Designed and coordinated with Data scientists team in implementing advanced analytical models against large datasets* Good knowledge in using cloud shell for various tasks and deployment activities* Developed Several complex hive,spark sql queries to create, partition, merge data from several sources and perform analytics to derive valuable insights
EDUCATION
Yogi vemana university
B tech, computer science engineering
SKILLS
ABOUT GOPAVARAPU DIVYA
As a Senior Software Dev Engineer at Yahoo/Verizon, I leverage my Big data, Hadoop,Spark skills to build scalable and robust ETL pipelines that process terabytes of data from Verizon network and corporate data warehouses. I have over 8 years of experience in building and migrating big data solutions in a production environment, using batch and near real time processing techniques.I am passionate about data engineering and machine learning, and I enjoy discovering, analyzing, structuring, and mining data to derive insights and value for the business. I have experience in designing and implementing data layer solutions, performing data analysis and integration, developing Spark ETL pipelines, and deploying and monitoring machine learning models. I also have expertise in using various BI tools, such as DataStage, Jasper Reports, and Oracle SQL Developer, to create reports, dashboards, and applications. I am always eager to learn new technologies and tools, and to collaborate with other teams and stakeholders to deliver high-quality and innovative solutions.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.