Vamshidhar Reddy Pogula
Senior Azure Data Engineer @Boyd Group Services Inc.
Signup · Get unlimited contacts
WORK HISTORY
Senior Azure Data Engineer @Boyd Group Services Inc.
Chicago, IL, US
Developed ETL pipelines using Azure Data Lake, HDFS, Hive, Spark, DB2, Teradata, Scala, Sqoop, Java, and Python for data ingestion and transformation, working with SQL and NoSQL databases like Snowflake, MS SQL, and MongoDB. Utilized PySpark SQL API for data extraction, loaded data lakes, and optimized performance with Java and Python, efficiently managing large transactional datasets. Configured Azure Data Factory and Airflow to optimize SQL-based data ingestion from diverse sources. Created custom Spark, Hive, and Sqoop adapters for various data sources, enabling both batch and real-time data ingestion into HDFS using Java and Python. Leveraged Snowflake\'s automatic scaling capabilities, focusing on SQL operations to handle fluctuating data workloads effectively. Worked on building data pipelines using Azure Data Factory, Azure Databricks, loading data to Azure Data Lake. Designed and developed mappings with complex transformation rules in Redpoint Used Parameters in the mappings and successfully generalized the mapping for other data points. Used the DataStage Designer to develop processes for extracting, cleansing, transforming, integrating and loading data into staging tables.Hive and spark tuning with partitioning/bucketing of Parquet and executors/driver\'s memory. Crated Azure Data Factory (ADF) Streaming pipelines to consume data from Azure Event Hub using Spark and load into Azure Data Lake Storage (ADLS). High Availability cluster that hosts dozens of database servers including Sybase and Oracle. Developed SnowSQL scripts to deploy new objects and update changes into Snowflake. Create Python Flask login and dashboard with Neo4j, GraphQL database and execute various cypher queries for data analytics. Performed thorough data analysis for the purpose of overhauling the database using ANSI SQL. Experienced in Performance Engineering and testing using Adobe Analytics and Splunk.
ABOUT VAMSHIDHAR REDDY POGULA
With over 5 years of experience in data engineering, I am dedicated to building robust ETL pipelines and optimizing cloud solutions. At Kohl\'s, my mission is to streamline data processes, leveraging my expertise in Spark, Java, and Python to align with our organization\'s culture of innovation. I am committed to enhancing our data infrastructure, contributing to the team with diverse experiences and a focus on scalable, efficient data management. In my most recent role at Kohl\'s, I have developed and optimized data ingestion and transformation processes using Azure Data Lake and Snowflake, improving performance and scalability. I\'ve configured Azure Data Factory and Airflow, creating custom adapters with Java and Python, which has empowered our team to manage large transactional datasets more effectively and enabled real-time data insights.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.