Oscar Yang
Data Engineer Ii @Highspot
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer Ii @Highspot
US
As a data engineer in Machine Learning Data Platform team at Highspot, I contributed to Highspot\'s new data platform, included ETL for batch and streaming processing. Meanwhile I was fully responsible for containerizing the whole new data platform project, automated the Kubernetes container life cycle management, used a varity of tools to build CI/CD pipeline, and also collaborated across departments to achieve observabilities in production, and meet all the policies. I also closely worked with PM to understand the key metrics and exported those as dashboards.
EDUCATION
Beijing University of Posts and Telecommunications
Bachelor of Science - BS, Computer Sicrnce and Technology
Beijing University of Posts and Telecommunications
Master's degree, Computer Technology
The University of Texas at Arlington
Master's degree, Computer Software Engineering
ABOUT OSCAR YANG
Having 6 years of experience in the IT industry, which includes hands-on experience in Big Data Technologies, SQL, and BackEnd Development with Python, Java and Scala. Extensive experience in Architecting & Building Data platforms from scratch. Strong Knowledge of the architecture and components of Spark, and efficient in working with Spark Core, SparkSQL, and Spark streaming. Capable of building large scale distributed applications, which includes Stream and Batch Processing a large set of structured, semi-structured and unstructured data with Spark Scala and Python. Experienced in building, deploying and scaling Rest API’s in AWS. Certified AWS Solution Architect Associate and Extensive experience in utilizing AWS services such as EMR, S3, Lambda, Step functions, EC2, EMR, DynamoDB, Redshift, Kinesis, Athena, and more for various Usecases. Skilled in Data Manipulation and transformation techniques, including data cleansing, aggregation, and statistical analysis with Python. Strong experience in data modeling, ETL processes, and data warehousing concepts Expertise in Snowflake SQL with extensive experience in creating complex database joins, optimizing queries, and designing efficient databases. Experience with cloud-based data warehousing solutions, such as Snowflake, and ability to develop and maintain data pipelines from various data sources to the data warehouse. Expertise in building the Realtime Distributed Stream Processing Pipelines with Spark, Flink and Kafka, Kinesis. Good experience on Building & Managing Data Infrastructure, Monitoring and Observabilty. Hands on experience in Orchestrating datapipelines with Airflow, AWS Lambda, Glue Scheduler and Step Functions. Proficiency in coding in different tools and technologies i.e. Python, Java, Scala, Java, Unix shell scripting, Jira, Bitbucket, GIT, Jenkins, Docker, Kubernetes
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.