V Krishna

About V Krishna

With over a decade of experience as a Senior Data Engineer and Big Data Developer, I have honed my expertise in designing, developing, and maintaining robust data systems across various platforms. My proficiency spans a wide range of technologies, including Hadoop, Spark, and the Hadoop ecosystem, where I\'ve been instrumental in configuring and utilizing components like HDFS, MapReduce, Hive, and Kafka. I have successfully implemented and orchestrated complex data pipelines using Oozie and Airflow, ensuring the seamless processing and analysis of large datasets. My deep understanding of data manipulation, visualization, and cloud-based solutions has enabled me to deliver high-performance data architectures that drive business insights.Throughout my career, I have leveraged cloud platforms such as Microsoft Azure, AWS, and Google Cloud Platform to develop scalable and secure data solutions. At Freedom Mortgage, I led the development of cloud-based data pipelines using Azure services, integrated Azure Data Lake with Synapse Analytics, and optimized Oracle PL/SQL packages for data transformation. My experience extends to Snowflake data warehousing, where I have implemented solutions that significantly improved query performance and data accessibility. I am also well-versed in utilizing Python for data manipulation and analysis, developing RESTful microservices, and implementing CI/CD pipelines with tools like Jenkins and GitHub.In my previous roles at Charles Schwab and Avanade, I played a crucial role in building hybrid data processing solutions, integrating Python for data analytics, and employing Apache Spark for comprehensive Big Data analytics. I have a strong background in data security, governance, and compliance, having implemented encryption and access controls to meet regulatory standards.

Experience

1. Senior Data Engineer

   Citizens

   May 2024 — Present · Dallas, TX, US

   Implemented data quality measures in Informatica ETL and demonstrated proficiency in Data Warehousing- Integrated data using Informatica (IICS), Snowflake, SQL, and Python to solve complex use cases and meet business requirements- Provided guidance and direction for the design and development of data layers, including reviewing ETL requirements and specifications- Led team design sessions, facilitated decision-making processes, and ensured alignment with project goals- Managed the timely delivery of data solutions, meeting defined requirements and stakeholder expectations- Oversaw metadata asset management, including onboarding data and configuring custom catalog sources and lineage- Created and maintained integration mappings for inbound and outbound data processes, ensuring efficient data transfer- Conducted integration testing of Informatica PowerCenter mappings and workflows to verify that data flows correctly between systems- Monitored the performance of Informatica PowerCenter ETL processes and troubleshoot any performance issues- Used version control tools to manage changes and maintain the integrity of Informatica PowerCenter code and configurations- Designed ETL processes with DBT and enhanced reliability with Python. Migrated legacy systems for better scalability and security- Developed data pipelines with Scala and Spark SDKs, automating tasks with Lambda and Step Functions. Conducted data analysis for actionable insights- Created HIVE UDFs for seamless dataset integration. Used Docker, GitHub, and for efficient development and deployment- Experienced with (Dataproc, GCS, Cloud Functions, BigQuery) and proficient in Data Factory and DataBricks- Automated Kafka Producer tasks with Python and Unix scripting. Improved Hadoop performance with Scala and Spark, migrated MapReduce jobs to Spark, and developed Kafka producers/consumers- Managed relational (Oracle, SQL, PostgreSQL) and NoSQL (HBase) databases.

Education

• Jawaharlal Nehru Technological University, Kakinada

  Bachelor's degree, Computer Science and Engineering

  2011 — 2015