Jun Wang
Lead Program Engineer at Backblaze Cloud Storage & Backup
- Role
- Sr Data Engineer at Backblaze
- Location
- Mountain View, CA, US
- LinkedIn followers
- 500 followers
About Jun Wang
Responsible for data analytic schema design on Hadoop Ecosystem and Teradata databases at LinkedIn. Develop ETL processes and make data available and accessible to entire company to make data driven decisions. Extensively used Spark, Hive, Presto, Teradata, python, etc. Lead a few projects for building and maintaining the state-of-the-art ETL infrastructure. Initiated and developed a few framework programs to automate dimension data download, transformation of HDFS formatted data to traditional databases. Participated the development of various large data warehouses at Yahoo. Responsibilities include architecture design, database schema design and implementation, and data load programming. Extensively used PIG and Hive for big data solution.Successfully re-architected a data loading and pre-processing by migrating the process from traditional C program on Unix machine to Grid clusters. Significantly improved SLA from days to hours for a large data warehouse system. Used Hadoop HDFS, PIG and Hive.Based on the requirements from product group, analyzed various data sources at the company, and successfully designed all dimension tables for a new data warehouse application system. Implemented all procedures with SQL to load dimension data.Used Java, Python, Perl implemented various tools to automatically deploy database codes, loading data into database, communicating with HDFS file systems, etc. These tools have been adopted in many groups in Yahoo.Responsible for designing and implementing a database application infrastructure. With this infrastructure, we can easily deploy large database codes to many databases on different environments. We adopted the basic principles of software engineering, considered the special needs of the database application, defined many good policies and processes based on our experience and lessons, and tried to provide the best solutions for the large database application.
Experience
Sr Data Engineer
Feb 2020 — Present · San Diego County, CA, US
Education
University of Wyoming
Master's Degree, Computer Science
Nanjing University
Master, Computer Science.
Skills
- Hive
- Sql
- Linux
- Agile Methodologies
- Software Development
- Databases
- Big Data
- Global Data Synchronization
- Software Engineering
- Mapreduce
- Data Warehousing
- Mysql
- Distributed Systems
- Perl
- Hadoop
- Oracle
- Java
- Unix
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.