Job Description

Senior Data Engineer - Verian

Location: Remote/Hybrid

Company: Verian

Division: Technology

Band: 7

About Verian

Verian

works with clients around the world, providing evidence, insights and advisory services to inspire the next generation of public policy and programmes. We operate at the intersection of data, technology, and human understanding.

With permanent offices in 17 countries and a team of 800 specialist consultants and researchers, we advise government departments, regulators, charities, and public sector bodies to make better decisions and improve people's lives.

About the Role

Our team is looking for a

Senior Data Engineer

to join our Data Platform team and help deliver our product strategy for managing and analysing social research data. You will work primarily with

Azure and Databricks

, supported by technologies like

Power BI

, to design, build, and maintain scalable, reliable, and high-quality data pipelines.

As a key member of the team, you'll collaborate closely with product managers, lead engineers, and cross-functional teams to turn business requirements into technical solutions that drive real-world impact.

Key Responsibilities

Design, develop and maintain data pipelines using

Azure Data Factory (ADF), Databricks, Synapse, Python, R, Spark, Pandas and SQL.

Manage and optimize Azure resources including

Azure Portal, Blob Storage, and Data Lake.

Implement and manage data workflows, ensuring seamless data orchestration and integration. Develop and maintain ETL processes, including

delta loading and change data capture (CDC)

mechanisms. Implement pipelines with

Azure DevOps, Terraform

and/or CI/CD. Ensure

data security, quality, and governance

across all, including access controls and retention policies. Collaborate with stakeholders to translate business requirements into technical specifications. Utilize

Power BI

(PBI) for data visualization and reporting.

Your Skills and Experience

Proven experience implementing

data platforms and products

. Strong experience with

Cloud technologies, particularly Azure

. Proficiency in

Python, R, Spark, Pandas, SQL.

Experience working with

Delta Live Tables.

Familiarity with

Databricks medallion architecture

. Experience with

Agile and DevOps methodologies

. Knowledge of

Unity Catalog

. Experience with CI/CD pipelines and

Terraform

.

Why Join Verian?

Joining Verian means becoming part of a

dynamic and collaborative community

that thrives on innovation and intellectual curiosity.

Play a

highly impactful role

in building a

data platform from the ground up

, right at the heart of our digital transformation. Contribute to the creation of

innovative data products

leveraging

AI and Machine Learning

. Work on

cutting-edge projects

that influence public policy and drive meaningful social change. Enjoy

flexible working arrangements

that support work-life balance. Collaborate with a

passionate, and supportive team

in an environment where wellbeing truly matters.

How to Apply

If you're passionate about turning data into insights that impact society, we want to hear from you! Apply online today, and if your profile matches, we'll invite you for a first exploratory interview.

We encourage applications from all sections of society and provide reasonable accommodations for candidates with disabilities throughout the application and interview process.

Job Types: Full-time, Permanent

Pay: 65,000.00-70,000.00 per year

Benefits:

Company pension Flexitime Life insurance Private medical insurance Referral programme Sick pay Work from home
Work Location: Remote