Senthilnathan Puliyadi Ekanathan
Senior Data Engineer @Cognizant
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @Cognizant
London, GB
SkillsBigdata/SparkExpert in Pyspark,RDD’s,Data frames, SparkSQL,HDI,Hive, Kafka & Zookeeper,SQL,MicroservicesMicrosoft Azure/DatabricksADF, ADLS Gen2,Azure synapse, Azurestorage container, Delta Lake,DatabricksWorkflow Orchestration,GCPAWSSpecialist in AWS – EMR, EC2,S3, Glue ETL, Redshift, Athena,Lambda & Data pipelinesDevOps toolsFamiliar inJenkins,GITHUB,JIRA,Kubernetes,Docker,CI/CD Deployments, PowerBI, PentahoETL,Agile•Designed and implemented high-performance LakehouseArchitecture on cloud environments, utilizing expertise in AWS & Azure for Fusion Data Migration project of Hive Home.• Developed ADF pipelines to source data from Salesforce APIversion into Azure Storage container and Azure Data Bricksjobs using Workflow/Notebooks and Delta Lake tables• Ingested the Data from Finops D365 Source system into theADLS from synapse analytics and loaded into Delta tables• Built and optimized Spark-based data processing workflows,with deep knowledge of Spark runtime internals for fine-tuning job performance• Built a robust stream application to ingest the IOT Devicesevent live Data into Databricks and delivering the stream datainto Data lake and downstream push to Kafka event streams• Collaborating with cross-functional teams to ensure thatcustomer service agents have access to up-to-date customerinformation, facilitating effective customer interactions andretention efforts• Developed Databricks workflow orchestration to transformthe Data into delta lake tables using Pyspark• Managing a portfolio of approximately 120 Gold tables,650Silver tables, 130 Bronze tables and ensuring that all tableshave the latest D-1 data from the data lake enables Data toPBI
EDUCATION
K.L.N.COLLEGE OF ENGINEERING
Engineer's Degree, Computer Science
SPVMS,Madurai
High School
ABOUT SENTHILNATHAN PULIYADI EKANATHAN
Highly motivated Databricks /Azure/AWS Data Engineer with a strong passion for learning new technologies. Total IT Experience: 15 years and 2 months Big Data Experience: 8 yearsCloud Experience: 6 + years Databricks DataEngineer certification Databricks GenAI Certification AWS Certified Solutions Architect Associate(Expires on Jan 2023)-Top Skills : Proficient in Apache Spark withPython, AWS Stack(Glue,EMR Datapipelines,Kinesis Streaming,Redshift/Athena,S3) & Azure synapse,ADF,Azure Databricks(DLT pipelines/Jobs/Workflows/Unity Catalog) Expertise in developing and implementing ingestions through ADF Pipelines and DLT Pipeline using notebooks and jobs in Databricks Lakehouse platform Proficient in implementing CI/CD Pipelines and code changes in Databricks Proficient in implementing Data Lake solution using Apache Kafka Experience working with Kafka Streaming Devops using Jenkins and Pentaho Data Integration -Programming languages : Pyspark ( Intermediate ) Scala ( Intermediate ) Shell Scripting-Tools and Methodologies : Experienced with Git CICD Docker jenkins Experienced with working in Agile Methodologies Experienced with JIRA, Confluence
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.