Spoorthi C

About Spoorthi C

I am a Senior GCP Data Engineer with 10+ years of experience in the field. I have…

Experience

1. Senior Gcp Data Engineer

   Cummins Inc.

   Aug 2021 — Present · IN, US

   Worked on Google Cloud Platform (GCP) in all the big data products big query, Cloud Data Proc, Google Cloud Storage, Composer (Air Flow as a service).• Involved in full Software Development Life Cycle (SDLC)- Business Requirements Analysis, preparation of Technical Design documents, Data Analysis, Logical and Physical database design, Coding, Testing, Implementing, and deploying to business users.• Data engineers are typically responsible for designing, developing, and maintaining data pipelines that ingest data from various sources into Apache Druid.• Transformed and analyzed the data using Pyspark, HIVE, based on ETL mappings.• Used apache airflow in GCP composer environment to build data pipelines and used various airflow operators like bash operators, Hadoop operators and python callable and branching operators.• Used cloud shell SDK in GCP to configure the services Data Proc, Storage, Big query.• Establish data governance policies and practices to maintain data quality, compliance, and data security in Druid• Developed ELT processes from the files from abinitio, google sheets in GCP with compute being dataproc (pyspark) and big query.• Develop dashboards in poker to visualize tech suspicious patterns/activities in real time for business users.• Worked on Building data pipelines in airflow in GCP for ETL related jobs using different airflow operators.• Changing the existing Data Models using Erwin for Enhancements to the existing Datawarehouse projects.• Worked on Google Cloud Platform (GCP) services like compute engine, Git Hub, cloud load balancing, cloud storage and cloud SQL.• Develop and deploy the outcome using spark and Scala code in Hadoop cluster running on GCP.• Built data pipelines in airflow in GCP for ETL related jobs using different airflow operators.• Experience in Google Cloud components, Google container builders and GCP client libraries.

Education

• Jawaharlal Nehru Technological University

  Bachelor of Technology - BTech, Computer Science