Scribd
Upload
7 views2 pages

JD For Senior - Junior - Databricks - Engineer - P3 - P2

The document outlines a job description for a Senior/Junior Databricks/Data Engineer position at NCS in Pune, India. The role involves managing day-to-day operations of the Data Platform in Azure/AWS Databricks, including designing data ingestion pipelines, ensuring data quality, and optimizing performance. Essential qualifications include a Bachelor's degree in IT or related fields, at least three years of experience in data analytics, and proficiency in Azure/AWS Databricks, Scala, or Python.

Uploaded by

Sourabh Ganjave
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views2 pages

JD For Senior - Junior - Databricks - Engineer - P3 - P2

The document outlines a job description for a Senior/Junior Databricks/Data Engineer position at NCS in Pune, India. The role involves managing day-to-day operations of the Data Platform in Azure/AWS Databricks, including designing data ingestion pipelines, ensuring data quality, and optimizing performance. Essential qualifications include a Bachelor's degree in IT or related fields, at least three years of experience in data analytics, and proficiency in Azure/AWS Databricks, Scala, or Python.

Uploaded by

Sourabh Ganjave
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Job Description

Job Title Senior / Junior Databricks / Data Engineer

Business Entity ☐ Singtel ☒ NCS ☐ Optus ☐ Others: Enter entity name

GBU / BU / Division /
NCS - IDC
Department
☒ New Role ☐ Existing Role – Change of Scope
Reason for Submission
☐ Others: Enter reasons
Reporting to Engineering Manager
Working Location Country: India State: Maharashtra - Pune
Job Level / Grade P2 & P3
HR Business Partner Enter name of Biz HR

SECTION A: POSITION SUMMARY


State the objective and purpose of the role.

This role is accountable for running day to day operations of Data Platform in Azure /
AWS Databricks NCS. Data Engineer is accountable for ongoing Development,
Enhancement support and maintenance data availability and data quality, performance
enhancement and stability of the system.

1. Designing and implementing data ingestion pipelines from multiple sources using Azure
Databricks
2. Ensure data pipelines run smoothly and efficiently
3. Adherence to security, regulatory and audit control guidelines
4. Drive optimization, continuous improvement and efficiency

SECTION B: KEY RESPONSIBILITIES AND RESULTS


Indicate key responsibilities and performance indicators of this role.
For existing role, please indicate additional responsibilities in bold.

1
Designing and implementing data ingestion pipelines from multiple sources using Azure
Databricks
2 Ensure data pipelines run smoothly and efficiently
3
Developing scalable and re-usable frameworks for ingesting of data sets
4 Integrating the end to end data pipeline - to take data from source systems to target data
repositories ensuring the quality and consistency of data is maintained at all times
5 Working with event based / streaming technologies to ingest and process data
6 Working with other members of the project team to support delivery of additional project
components (API interfaces, Search)
7 Evaluating the performance and applicability of multiple tools against customer requirements
8 Technically competent in Cloud and Databricks to provide technical advice to the team and
to be involved in issue resolution
9 Provide on-call support and afterhours/weekend support as needed
10 Fulfill Service Requests related to Data Analytics platform
11 Lead and drive optimization and continuous improvement initiatives

_________________________________________________________________________________________
Page 1 of 2
12 Play gate-keeping role and conduct technical review of the changes as part of release
management
13 Understand various data security standards and adhere to the required data security controls
in the platform

SECTION C: QUALIFICATIONS / EXPERIENCE / KNOWLEDGE REQUIRED


Indicate key knowledge and skills required for this role to perform the tasks to a satisfactory level. To
also specify a suitable level of qualification required (i.e. basic, advanced, or professional), where
applicable.

Category Essential for this role Good to have


• Bachelor’s degree in IT, Computer • Relevant accredited
Education and Science, Software Engineering, certifications for Azure, AWS,
Qualifications Business Analytics or equivalent. Cloud Engineering and/or
Databricks
• Minimum three years of experience in • Data Engineering expertise
data analytics field in Big Data Management
• Experience with Azure/AWS using Cloudera, Python,
Databricks Hive, Scala, Data
• Experience in building and optimizing Warehouse, Data Lake,
data pipelines, architectures and data AWS, Azure
sets
• Excellent experience in Scala or
Python, PySpark and SQL
• Ability to troubleshoot and optimize
complex queries on the Spark
platform
Work Experience • Knowledgeable on structured and
unstructured data design / modelling,
data access and data storage
techniques
• Expertise in designing and deploying
data applications on cloud solutions,
such as Azure or AWS
• Hands on experience in performance
tuning and optimizing code running in
Databricks environment
• Demonstrated analytical and
problem-solving skills particularly
those that apply to a big data
environment
• Azure/AWS Databricks
• Python / Scala / Spark / PySpark
Technical / • HIVE / HBase / Impala / Parquet
Professional • Sqoop, Kafka, Flume
Skills • SQL and RDBMS
Please provide at • Airflow
least 3 • Jenkins / Bamboo
• Github / Bitbucket
• Nexus

_________________________________________________________________________________________
Page 2 of 2

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy