Principal Data Engineer

Summary

A principal data engineer leads multi-functional delivery teams to deliver robust data services for their department, other departments and private sector partners.
At this role level, you will:

inspire best practice for data products and services within your teams

build data engineering capability by providing technical leadership and career development for the community

work with other senior team members to identify, plan, develop and deliver data services

Work Activity Components

Title Details
Data pipelines and stores (Level 6)(DENG) Plans and leads data engineering activities for strategic, large and complex programmes.

Technical Skills

Title Details Depth
Business Environment The business environment relating to own sphere of work (own organisation and/or closely associated organisations, such as customers, suppliers, partners and competitors), in particular those aspects of the business that the specialism is to support (i.e. localised organisational awareness from a technical perspective). Proficient in
Cloud/Virtualisation The principles and application of cloud/ virtualisation (including ownership, responsibilities and security implications). Use of tools and systems to manage virtualised environments. Proficient in
Database Software Software that enables the user to capture, create, populate and manipulate data structures and where appropriate unstructured data. Proficient in

Training

Title Details
Data Management Data management concepts, methods, tools and techniques relating to the planning, development, implementation, administration and curation of data.
Coaching Concepts, methods and techniques for providing coaching in subject specialisms to individuals or groups (e.g. GROW model).
Team Dynamics Team dynamics are critical for organisational success. Without positive team dynamics, the organisation cannot fully leverage employee potential and tap into their skills and experience. An understanding of team dynamics facilitates employee productivity and satisfaction while allowing teams to meet business objectives. Team dynamics methods, tools and techniques, such as Belbin, help individuals understand their role within a particular team, help develop strengths and manage weaknesses as a team member and improve overall team contribution and effectiveness..

Professional Development Activity (PDA)

Title Details PDA Group
Negotiating and Influencing Undertaking learning and practice of negotiating with and influencing others. Developing Professional Skills
Gaining Knowledge of Broader IT Issues Increasing and maintaining currency of knowledge of broader IT issues through reading, attending and participating in seminars or conferences, special studies, temporary assignments etc. Increasing Knowledge
Gaining Knowledge of Standards and Legislation Gaining and maintaining knowledge of relevant national and international standards and legislation. Increasing Knowledge
Gaining Strategic Knowledge of Employing Organisation Developing a comprehensive understanding of the business environment in which the employing organisation operates and its position, policies and direction in relation to health and care, country and global issues. Increasing Knowledge
Management Development Undertaking learning and best practice of the skills appropriate to managing all or part of an organisation, including business and financial management, benefits management, people management, management of change and strategic planning. This will require both on and off the job learning and may include participation in an appropriate development programme such as MBA or DMS (Diploma in Management Studies). Developing Professional Skills
Mentoring Acting as a mentor, advising those for whom there is no direct responsibility, on matters to do with their job role, career and professional development. Broadening Activities
Participation in Professional Body Affairs Taking an active part in professional body affairs at branch, specialist group, committee or board level. Participation in Professional Activities
Project Assignments Participating in a project team, working group or task force established to deliver a solution to a specific problem or issue - especially valuable if the group is inter-disciplinary. Broadening Activities

Qualification Components

Title Awarding Bodies
FEDIP Leading Practitioner The Federation for Informatics Professionals

Additional Frameworks

National Competency Framework for Data Professionals in Health and Care

Behaviours

Title Details
Delivering outcomes (B1.1) (Level 5) You are able to work on, and lead, diverse teams from multiple organisations to achieve a shared goal, even where there are conflicts.
Communicating within a hierarchy (B1.2) (Level 5) You only use hierarchical influence where there is no other type of influence that can be used and it is suitable to do so.
Generating consensus (B1.3) (Level 5) You are able to work with a diverse range of people and win their hearts and minds to support your goal.
Logical arguments (B1.4) (Level 5) You are able to debate appropriate complex subjects with a range of experts to agree the balance of probability.
Negotiation (B1.5) (Level 5) You are able to lead and win complex debates, sometimes over a long period of time often with confrontational and adversarial opponents.
Generating support (B1.6) (Level 5) You have created a team of leaders for who their teams consistently go above and beyond
Influence (B1.7) (Level 5) You are often the lead for organisation wide initiatives and have strong relationships across organisations.
Equality (B2.1) (Level 5) You are an ally for underrepresented and marginalised groups and model an open environment by encouraging staff to actively seek out input from these individuals. 
Challenging discrimination (B2.2) (Level 5) You are able to engage in the most sensitive ED&I issues, often in the public eye, and deal with them with the utmost dignity, respect and fairness.
NHS Constitution (B2.3) (Level 5) You are seen as an ambassador of promoting the behaviours and values of the NHS Constitution.
Supporting others (B2.4) (Level 5) You lead by example and are open and honest about health and wellbeing issues. You take adequate time for yourself as you know that must be done before you can create an effective system for others
Open environment (B2.5) (Level Four) You are an ally for underrepresented and marginalised groups and model an open environment by facilitating sessions for these individuals to share their lived experiences with you and your colleagues.
Challenging disrespect (B2.6) (Level Four) You support staff to understand the impact of disrespectful behaviour and support them in challenging it.
Written communication (B3.1) (Level 5) You are capable of having a number of articles published in national or international publications should the opportunity arise.
Discussing complex ideas (B3.2) (Level 5) You are able to engage in complex technical debates with other specialists, from a variety of disciplines, whilst using accessible and accurate language.
Delivering complex ideas (B3.3) (Level 5) Your confidence in your expertise enables others to feel confident and at ease with your contribution, even beyond your specialism.
Understanding new ideas (B3.4) (Level 5) You can easily strip complex problems to the root cause and convey this to people from a variety of backgrounds.
Reading audiences (B3.5) (Level 5) You can impart complex ideas to a diverse audience in simple terms without having to explain a number of different ways.
Problem sharing (B4.1) (Level 5) You promote collaboration opportunities throughout the organisation and beyond.
Seeking opinions (B4.2) (Level 5) You seek out opportunities for cross organisational working for those in your area of work.
Sharing best practice (B4.3) (Level 5) You promote the success of your area in and outside of your organisation.
Embedding best practice (B4.4) (Level 5) You attract examples of what good looks like both regionally and nationally and implement the most relevant to your area.
Patient impact (B5.1) (Level 5) You ensure all work within your area offers the most efficient support to customers and patient outcomes.
Understanding the customer (B5.2) (Level 5) You create opportunities for joint working with the customer to ensure efficient utilisation of the service.
Customer service (B5.3) (Level 5) You pre-empt changes within health and care to ensure the service is always able to deliver the best solutions.
Customer solutions (B5.4) (Level 5) You seek out opportunities for your service to learn cutting edge technologies that could be advantageous to the customer.

Leadership

Title Details
Empathy and understanding (Level Four) You act with care, empathy and understanding and ensure that your team knows you are always available to them.
Pressure (Level Five) You have an acute awareness of organisational and political pressure and are able to influence at the highest levels in order to alleviate them.
EDI (Level Five) You act as a champion for EDI issues and ensure that everyone within your remit shows compassion and understanding to those with differing life experiences and ensure that all opportunities are open and fair to all.
Compassion (Level Five) You show compassion to others who are struggling and create solutions to difficult emotional situations.
Bringing people together (Level Five) You are a leading exponent of bringing people together to work towards joint goals that were not at first visible.
Team support (Level Five) The goals and expectations you set for your unit are both achievable and aspirational.
Positivity (Level Five) You approach discussions on targets with positivity and use challenges as a means to develop innovative solutions.
Innovation (Level Five) Your team often hears you praising their work to others.
Safe to fail (Level Five) You promote the importance of learning through failure and set up safe environments where your team can develop their abilities safely, creating an environment where failure is celebrated as the first indicator of progress.
Fairness (Level Five) You act as an aspirational figure for those working within your business unit.
Opportunities (Level Five) You will maintain a strong overarching knowledge of the data profession in order to highlight opportunities and enhance your team's confidence in you as a leading data professional.
Goals (Level Five) You foresee shifts in staffing requirements and plan personnel and succession accordingly.
Performance (Level Five) You deliver messages around performance in an unambiguous but approachable manner.
Motivation (Level Five) People within your team have a clear understanding of their career path and feel valued and supported in achieving it.
Expectations (Level Five) You have high hopes of your team but make low demands so that everyone knows that what is expected is different to that which is hoped for and that not doing what is expected is not acceptable.
Developing talent (Level Four) You take pride in the talent of your team and go to great lengths to develop their skills and innovations.
Succession planning (Level Three) You are able to readily identify those in your team who have the opportunity to excel at their level and beyond and use this knowledge to begin succession planning.
Managing expenditure (Level Five) You are able to set, monitor and adjust expenditure to ensure your teams have the requisite resources are in place for the year's priorities.
Budget control (Level Five) You are able to set, monitor and adjust annual budgets to ensure individual projects don't overspend unless it is done so with the conscious ssacrifice of other areas of work.
Forecasting (Level Five) You are able to produce accurate budgets based on aspiration and foreseen developments in the responsibilities placed on your department.
Business cases (Level Five) You are able to assess the merits of business cases and options appraisals and balance these in line with business needs and financial capacity.
Headcount (Level Five) You are able to manage a large staffing budget to make sure each team has the requisite number of staff to achieve goals and restructure personnel to achieve objectives.
Recruitment (Level Five) You ensure that funding is in place for the appropriate recruitment and retention of staff whilst leading by example on recruiting to promote equality, diversity and inclusion.
Supporting ambition (Level Five) You take the time to talk to all staff members within your remit and encourage your managers to support their ambitions, providing solutions where these diverge from perceived organisational drivers.
Training opportunities (Level Five) You ensure that your managers enable training on a regular basis within their unit and have ample access to training for their own development. You ensure that everyone has access to the best training, balancing cost with quality.
Professional development (Level Five) You lead by example with your focus on CPD and PDP, even discussing yours with members of your team where appropriate.
Managing external pressures (Level Five) You support your direct reports by showing them how to deal with the political and organisational pressures from outside of the department and give them the tools to evaluate perspectives and push back on work that is not as high a priority.

Data Skills

Title Details
Non-technical audiences (Level Five) Ensure that communication has clear purpose, takes into account people's individual needs and provides the opportunity for open dialogue.
Stakeholder management (DEC1.2) (Level Five) Create an open environment for communication, fostering an honest inclusive approach to communication.
Positive communications (DEC1.3) (Level Five) Define the communication strategy for data engineering, ensuring that corporate messages are cascaded in a straightforward and engaging manner.
Facilitation (Level Five) Own the definition of documentation standards, working with cross-government groups to drive the definition of and adherence to industry wide standards.
Influence (B1.7) (Level 5) You are often the lead for organisation wide initiatives and have strong relationships across organisations.
Data profiling (Level Five) You have advanced expertise across a range of techniques or you may have a recognised deep expertise in a narrower range of specialisms.
Data visualisation (Data Engineer) (Level Five) You have advanced expertise in the relevant applications, systems and platforms in the organisation and share your knowledge with others.
Tools and techniques (Level Five) You exemplify best practice with methods and tools and guide others to a high standard.
Data management (DEC2.4) (Level Five) Derive an overall strategy of master data management that supports data exploitation for analysis, including linking/matching across diverse datasets.
Data requirements (Data Engineer) (Level Three) Investigate data requirements where there is some complexity and ambiguity.
Data transformation (Data Engineer) (Level Three) Provide advice on the transformation of data from one format or medium to another.
Data Modelling (Level Five) Drive the strategy for the data modelling principles within data engineering.
Reverse engineering (DEC3.2) (Level Four) You know how to work across organisational boundaries and recognise opportunities to align with or re-use the data models in different organisations.
Data integration (Level Three) You establish sustainable enterprise-scale data integration procedures and ensure the team adheres to them.
Data services (Level Four) You manage resources to ensure data services work effectively at enterprise level.
Data engineering best practice (Level Five) Maintain excellent knowledge of database and data warehouse concepts, design principles and technologies and work with data engineers to implement database / data warehouse designs that support demands of users.
Data iteration, review and maintenance (Level Three) You manage the iteration, review and maintenance of data requirements and data models.
Ingestion (Level Five) Leads the selection and development of data engineering methods, tools and techniques. Ensures adherence to technical strategies and architectures.
Modernisation tools (Level Five) Develops organisational policies, standards, and guidelines for the development and secure operation of data services and products.
Information governance (DEC4.3)(Level Five) Plans and leads data engineering activities for strategic, large and complex programmes.
Data standards and architectures (Level Five) Develop organisational policies, standards, and guidelines for data engineering.
Design (Level Five) Coordinate work to construct data pipelines & datasets for onward uses drawing on data engineering best practices.
Data governance (Data Engineer) (Level Three) Governance requirements and develop ethical and effective data sharing agreements for data sources.
Quality assurance (Data Engineer) (Level Three) Support processes for complex data quality checking and remediation.
Policies and standards (Level Four) You determine technical strategy, organisational policy, standards and guidelines and ensure alignment with enterprise goals and functional and non-functional requirements.
Metadata repositories (Level Four) You understand how metadata and their repositories support different areas of the business.
Metadata best practice (Level Five) Plan and lead data engineering activities for strategic, large and complex programmes.
Programming (Data Engineer) (Level Five) Take accountability for overall quality and performance of coding developed in area of responsibility.
Development standards (Data Engineer) (Level Five) Set long term strategy, identifying opportunities to build and improve suite of developed products.
Performance analysis (Level Five) Take account of departmental and cross government best practice / guidance, adopting where relevant and influencing direction of these.
Development lifecycle (Level Four) You take responsibility for the lifecycle of the whole code library sponsoring continuous code reviews and refactoring.
Technologies (Level Five) Lead development of a culture that encourages innovation, risk taking and collaboration.
Emerging trends (Level Five) Embed innovation processes throughout business units and link strategy execution with innovation.
Innovation (Level Five) Your team often hears you praising their work to others.
Stakeholder engagement (DEC7.4) (Level Five) Engage at a strategic level with external stakeholders/suppliers to identify opportunity, influencing direction of travel in partner organisations for mutual benefit.
Horizon scanning (Level Five) Take strategic view identifying long term opportunities and ensuring organisation is well placed to take advantage of cutting-edge developments.
Adoption (Data Engineer) (Level Five) Direct the identification, evaluation and adoption of new or existing data technologies to improve business processes.
Strategic alignment (Level Five) Align business strategies, enterprise transformation, and technology strategies.
Geographical Data Mapping (Level Three) You understand when geographical mapping is appropriate and can combine it with other visualisation methods to create greater impact. You can produce dynamic maps based on changing data.
Data Modelling (Level Three) You can build appropriate data models from physical data models and pick the most appropriate infrastructure. You understand data entities, attributes and specific modelling environments, e.g. Oracle, SQL Server, Hadoop etc.
Information Governance (Data)(Level Three) You know when data can be accessed and shared and know who to approach outside the organisation for advice. You understand how data linkage and different types of analysis can re-identify or help anonymise data. You have knowledge of GDPR.
Longitudinal Analysis (Level Three) You understand pragmatic differentiation between independent measures and repeated measures design. You understand how changing populations can affect analysis and choice of techniques used. You understand data attrition.
Technological Specialisms (R, Python, SQL, Tableau etc.) (Level Three) You can produce complex data models and visualisations whilst ensuring accurate linkage and data quality. You use appropriately advanced coding and debugging skills to utilise and contribute to the open source community.
Data Automation (Level Three) You are able to link directly to source data using appropriate tools. You understand the inbuilt functionality of Microsoft and other products to directly link to SQL servers, Azure etc. You have knowledge of APIs and how they may benefit automation.
Influencing (Level Five) Use strong influencing skills to represent data engineering interests with colleagues at all levels within this organisation and with senior external colleagues.
Metadata tools (Level Five) Lead the selection and development of data engineering methods, tools and techniques. Ensure adherence to technical strategies and architectures.
Innovation (Data Engineering) (Level Five) Align organisational and individual objectives, measures and rewards with innovation.
Metadata processes (Level Five) Derive an overall strategy of master data management that supports data exploitation for analysis, including linking/matching across diverse datasets.

Project Skills

Title Details
Business cases (WP1.1)(Level Five) You promote accountability for all digital and data staff involved in project work by modelling the highest professional standards.
Scope (WP1.2) (Level Five) You analyse the portfolio of projects in data and digital and liaise with others to ensure projects of the greatest utility for the business are supported by senior leaders, drawing on lessons learned within the data and digital community.
Reviews (Level Four) You advise on the coherence of programmes in data and digital to maximise the effectiveness of time and available skills within the business.
Quality assurance - Data and digital (Level Four) You lead quality assurance in data and digital, drawing on external expertise where necessary, learning lessons and sharing those with the wider data and digital community for their project work.
Advice and monitoring (WP2.1)(Level Five) You advise on the balance of the portfolio of projects as it affects or is dependent upon the availability of data and digital skills and resources for projects. You advocate for the portfolio to include data and digital projects which will promote the sustainability of the business.
Complexity (Level Two) You understand and can articulate when the complexity of a proposed project requires further professional management or support.
Scheduling (WP2.3)(Level Three) You sequence the activities in data and digital projects logically, effectively and efficiently, incorporating any lessons learned from similar past projects.
Refinement (Level Two) You refine the plan within your work area to take account of any authorised changes communicating actions, progress and results with project managers.
Resource identification (WP3.1)(Level Five) You ensure fully funded resources for projects are in place and optimised across any programmes, convincingly arguing their business value to non-data and digital colleagues and project stakeholders.
Skill acquisition and management (WP3.2)(Level Three) You plan for the recruitment of staff with additional required skill sets, liaising with HR and/or other providers to source skilled staff to fulfil project roles, onboard and manage them and their workloads.
Additional tools and resources (Level Three) You cost and acquire, deploy and contract for the support of additional tools and resources such as hardware, software, training and data sources for the course of the project life cycle.
Resource allocation (Level Two) You plan the allocation of existing resources to project work whilst effectively maintaining business as usual wherever feasible.
Project management (WP4.1)(Level Five) You ensure the team's project delivery activities have sufficient resources to co-exist with business as usual.
Pilots and testing (WP4.2)(Level Five) You oversee the reported risks and issues affecting data and digital roles in projects and programmes.
Implementation (WP4.3)(Level Five) You engage with the wider business, project managers and stakeholders to map out all necessary resources and activities including education in new ways of working and enhancing data-literacy in the business for effective and sustained implementation.
Communications (Level Two) You communicate effectively with others, adapting your style and approach as and when required.
Business change (Level Five) You understand the business and technological drivers for change and communicate that vision with stakeholders within and beyond the technological functions.
Assurance (WP5.2) (Level Five) You bring a holistic perspective, integrating change initiatives across programmes and build lasting solutions which are owned by the whole business.
Evaluation (WP5.3) (Level Three) You ensure appropriate solutions are evaluated and viable alternatives are considered to deliver the intended business benefits.

The Professional Body Responsible for this job family is AphA. This job role profile was created in collaboration with BCS, using Role Model Plus.

Give Feedback

The Occupational Architecture Project is interactive and dynamic


If you would like to provide feedback on this job role, or the job families, please click the button below.

Give Feedback