Kamlesh Kumar

WORK HISTORY

EDUCATION

SKILLS

ABOUT KAMLESH KUMAR

A Data Engineer having 12+ years of experience in cloud technologies & BigData.Data Pipeline Development: Specialised in using Python & SQL, with extensive experience in Airflow, dbt & prefect.Data Modelling: Skilled in dbt for data modelling using Dimensional Data modelling techniques.Python Development: Proficient in Python-based development, implementing solutions on cloud platforms.Streaming Technologies: Experienced with Spark/Pyspark on GCP Dataproc and streaming technologies including Kafka, GCP Pub/Sub, and Dataflow.Containerisation: Proficient in using Docker for containerisation and deploying applications to Kubernetes clusters on GKE(GCP) and EKS(AWS).Infrastructure: Used Terraform to manage infrastructure in AWS and GCP.AWS Services: Utilised AWS Lambda functions and SQS for building real-time data pipelines.Monitoring and Notifications: Created dashboards and a notification framework using Datadog and AWS CloudWatch logs.CI/CD and Testing Frameworks: Developed CI/CD pipelines using GitHub Actions and GitLab, used pytest to build test framework.Database Management: Experienced with relational and NoSQL databases, including extensive use of the Hadoop 2.0 (Cloudera) framework.Data Integration: Worked with Informatica Power Center v9.1 to build and manage end-to-end data pipelines, integrating data from various sources (XML, JSON, CSV, DBs) to Oracle DB.#Skills and Competencies* Language : Python, SQL, Scala* Cloud Platform : AWS & GCP* Data Ingestion : Fivetran, Airbyte, Kafka, Python Scripts* Data Transform : dbt, SQL, Databricks Spark/PySpark* Pipeline : Apache Airflow, Prefect, Dagster* Data Modelling : Dimensional Modelling, Facts & Dimensions, SCD1 & SCD2* DataWarehouse : Google Bigquery, AWS Redshift, Snowflake* ETL : Informatica Power Center 9.1* CI/CD : Jenkins, Docker, Kubernetes, Github Actions & Gitlab Pipelines