Lingesh Kumar
Lead Data Engineer @Kyd Analytics
Signup · Get unlimited contacts
WORK HISTORY
Lead Data Engineer @Kyd Analytics
London, GB
Delivered end-to-end technical and architectural solutions for data sourcing, application integration, and operational reporting. Expertly designed and implemented a Financial Crime application for continuous CDD remediation, processing large-scale customer and transaction data. Proficient in Python and PySpark for scalable data pipelines, data transformation, and machine learning. Extensive experience in schema generation, API integration, data analysis, and visualization. Contributed to enhanced data quality through robust cleansing and entity resolution. Demonstrated expertise in Data Lake platforms, cloud services including AWS (Glue, S3, Terraform,Redshift, Snowflake,Matillion) and Azure (Data Factory, Databricks), and mentoring junior data engineers.
EDUCATION
College of Engineering Gitam
Bachelor of Technology (BTech), Electrical and Electronics Engineering
SKILLS
ABOUT LINGESH KUMAR
Provide technical guidance and defining the data architecture for individual projects and the business as a whole. Work with senior management, technical and client teams in order to determine data requirements and best practices for advance data manipulation, storage and analysis in a data warehouse (Databricks, SnowFlake, Redshift,Teradata ), Big data ( Data Lake ) & Cloud environment.AWS Data Analytics Speciality Certified. Microsoft Azure Data Engineer Certified.Quantexa CertifiedQualifications-Able to deliver data management vision, goals, priorities, design principles, and operating policies in support of the business goals of the organisation.Build ETL/ELT processes to ingest, transform, and load large volumes of data from various sources into the data lakehouse using Databricks. Optimize Spark jobs and data transformations for performance and efficiency.Implement data quality checks and monitoring to ensure data accuracy and reliabilityExpert in efficiently and reliably loading diverse data sources into Snowflake (batch, streaming, real-time).Highly skilled in transforming raw data within Snowflake using SQL and other tools to create clean, analytical-ready datasets. Design, implement, and deploy high-performance, custom applications on Data Lake Hadoop Hortonworks/Cloudera or AWS /Azure Cloud platform. Highly skilled in building scalable ETL/ELT pipelines on Databricks using Spark, Python/Scala, and SQL to ingest, transform, and load large datasets into the Lakehouse.Experience working in complete Data science lifecycle from feature engineering to building the models using Scikit LearnImplement best practices and methodologies in Data Integration, Data Migration,Data Analysis, Data reconciliation, Data processing, validation, enrichment, standardisation and transformation.Technology Stack: Spark, Hive, MapReduce, Impala, Hadoop, HDFS, Yarn, Zookeeper, Oozie, Avro, Parquet,Spark Streaming, Kafka, Flume, Sqoop, Azure, AWS glue, Pyspark, S3, AWS Lambda, stepfunction,Kinesis, DyanmoDB, Elastic Search, Snowflake, EMR, AWS managed Containerisation ( Fargate), Terraform, Azure data factory, azure function, Purview, Hdinsight, Azure data lake, Azure Synapse Analytics, azure data bricks. CI/CD with Git/Terraform/Jenkins, Airflow, Databricks, Lakehouse.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.