NCF Category: Data Engineering

You manage stakeholder expectations.

You understand a range of tools for storing and working with metadata.

You provide oversight and advice to more inexperienced members of the team.

You utilise standard tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations.

You collaborate with others to review specifications where appropriate and define test conditions and procedures.

You analyse and report test results.

You perform program performance analysis reporting on a variety of appropriate metrics

You initiate code reviews and promote the use of refactoring techniques to enhance the lifespan of the code library. 

You apply detailed knowledge and understanding of the technical concepts required for the role. 

You understand how these fit into the wider technical landscape. 

You grasp the impact of applying emerging trends in data and analysis tools and techniques on the team.

You understand the concepts and principles of data modelling and can produce relevant data models across multiple subject areas.

Working with senior colleagues you understand how to reverse-engineer data models from existing systems.

You understand industry-recognised data modelling patterns and standards and when to apply them based upon a detailed understanding of requirements.

You design, build and test complex or large-scale data structures and associated components and liaise with colleagues to create data pipelines for services.

You can design, build, test, modify and maintain data pipelines and data stores creating complex or large-scale data products.

You understand how the data engineering standards, tools and technologies fit into the business data architecture.

You understand the Information Governance requirements for the data you handle and can perform administrative tasks to provide accessibility, retrievability, security and protection of data

You understand the ethical implications of your work.

Consideration of data quality issues and remediation.

You create metadata, design appropriate metadata repositories and manage changes to existing metadata repositories.

2

You can effectively communicate to and between technical and non-technical stakeholders and facilitate discussions within a multidisciplinary team, with some potentially difficult dynamics.

You can advocate for the team externally.

You know how to manage different perspectives.

You know how to facilitate difficult discussions within the team or with diverse senior stakeholders.

You develop expertise in data profiling, locally used applications, systems, platforms and reporting tools and share your knowledge with others.

You can apply a range of data visualisation practices and can advise on best practice and guide others to a high standard.

You can determine which tools and techniques to use to explore or solve a variety of business issues and communicate the results of analysis with impact to a range of audiences.

Recognise where external data sources can be integrated with own datasets.

Create new datasets through the manipulation of multiple data sources, including linking or matching data using techniques already established in the work area.

You work with metadata to complete tasks such as data and systems integration impact analysis.

You design, code, test, correct and document simple programs or scripts under the direction of others.

You contribute to code reviews and use refactoring techniques to enhance the sustainability of the code library.

You peer review and test the work of a similar complexity by other team members.

You understand core technical concepts related to the role and can apply them with guidance. 

You seek out opportunities to experiment and innovate with new tools.

You understand the concepts and principles of data modelling and can produce, maintain and update relevant data models for specific business needs. 

You have knowledge of how to reverse-engineer data models from systems. You have knowledge of how to reverse-engineer data models from systems.

You design, build and test repeatable and reusable data products based on data feeds from multiple systems.

You assist in the design, build and testing of data pipelines and products based on feeds from multiple systems using a range of different storage technologies and/or access methods.

You know how to create repeatable and reusable products and you understand how your tools fit into the business data architecture.

You understand the Information Governance requirements for the data you handle and ensure its storage meets current legislation.

You maintain a repository to ensure information remains accurate and up to date.

1

You understand the need to translate technical concepts into non-technical language and can maintain establish communication channels from the data engineering staff to internal and external stakeholders.

You can adopt the most appropriate tool for analytical tasks and communicate results visually and verbally.

You have knowledge of reporting tools, applications and systems used in your organisation and of standard statistical techniques.