Srija G
Looking for C2C / C2H basis positions | Senior Data Engineer | Data Engineer | Big Data | Python | SQL | AWS | Azure | Hadoop | Kafka | ETL | Tableau| PySpark |Scala| NoSql
- Role
- Sr Data Engineer at Early Warning Labs
- Location
- North Brunswick, NJ, US
- LinkedIn followers
- 500 followers
About Srija G
Data Engineer with 8+ years of experience in Analysis, Design, Development, and…
Experience
Sr Data Engineer
Jan 2022 — Present · Scottsdale, AZ, US
Responsible for developing a highly scalable and flexible authority engine for all customer data. • Worked on Resetting of customer attributes that provide insight about customer. Purchase frequency, marketing channel, Groupon deal categorization. Advocate different sources of data using SQL, HIVE, SCALA. • Integrated 3rd party data agencies (Gender, age, other purchase history from other sites) try to integrate that data to existing data store. • Used Kafka HDFS Connector to export data from Kafka topic to HDFS files in a variety of formats and integrates with Apache hive to make data immediately available for SQO querying. • Normalized the data according to the business needs like data cleansing, modifying the data types and various transformations using Spark, Scala and GCP Dataproc. • Implemented dynamic partitioning in BigQuery tables and used appropriate file format, compression technique to improve the performance of Pyspark jobs in the DATAPROC. • Built a system for analyzing the column names from all tables and identifying personal information columns of data across on-premises Databases (data migration) to GCP • Process and load bound and unbound Data from Google Pub/Sub topic to Big-Query using cloud Dataflow with Python. • Worked on partitions of Pub/Sub messages and setting up the replication factors. • Operation focused, including building proper monitoring on data process and data quality. • Effectively worked and communicated with product, marketing, business owners, and business intelligence and the data infrastructure and warehouse teams. • Performed analysis on data discrepancies and recommended solutions based upon root cause. • Designed and developed job flow using Apache Air flow. • Worked on Intellij IDE, Eclipse IDE, Maven, SBT, GIT • Working on data pipe line which is built on top of Spark using Scala
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.