SC Cleared Databricks Data Engineer - Hybrid - Outside IR35

Newcastle upon Tyne
4 days ago
Create job alert

SC Cleared Databricks Data Engineer - Hybrid - Outside IR35

We are seeking a skilled Databricks Data Engineer to design, build, and optimize scalable data pipelines and analytics platforms. In this role, you will work closely with data scientists, analysts, and product teams to deliver reliable, high-performance data solutions using Databricks and modern cloud technologies.

Key Responsibilities

Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake

Build and optimize ETL/ELT workflows for batch and streaming data

Develop data models to support analytics, reporting, and machine learning use cases

Ensure data quality, reliability, and performance across data platforms

Integrate data from multiple sources including APIs, databases, and cloud storage

Collaborate with data scientists and analysts to enable advanced analytics and ML workloads

Implement monitoring, logging, and cost-optimization best practices

Follow data governance, security, and compliance standards

Document data pipelines, architectures, and best practices

Required Qualifications

3+ years of experience in data engineering or a similar role

Strong experience with Databricks and Apache Spark

Proficiency in Python and SQL

Experience with Delta Lake, data modeling, and performance tuning

Hands-on experience with cloud platforms (AWS, Azure, or GCP)

Familiarity with data orchestration tools (e.g., Airflow, Azure Data Factory)

Solid understanding of data warehousing and big data concepts

Preferred Qualifications

Experience with real-time/streaming data (Spark Structured Streaming, Kafka, etc.)

Knowledge of CI/CD for data pipelines

Experience supporting machine learning pipelines

Databricks or cloud platform certifications

To apply for this role please submit your CV or contact Dillon Blackburn on (phone number removed) or at (url removed).

Tenth Revolution Group are the go-to recruiter for Data & AI roles in the UK offering more opportunities across the country than any other recruitment agency. We're the proud sponsor and supporter of SQLBits, Power Platform World Tour, and the London Fabric User Group. We are the global leaders in Data & AI recruitment

Related Jobs

View all jobs

Principal Software Engineer

Senior Software Engineer

SC / DV Cleared Data Engineers Needed – Consultancy – AWS

Mechanical Design Manager

Product Owner - Outside IR35 - Active SC or BPSS required

Integration Engineer (must be eligible for SC clearance)

Subscribe to Future Tech Insights for the latest jobs & insights, direct to your inbox.

By subscribing, you agree to our privacy policy and terms of service.

Industry Insights

Discover insightful articles, industry insights, expert tips, and curated resources.

What Hiring Managers Look for First in Robotics Job Applications (UK Guide)

Robotics is one of the most dynamic, interdisciplinary fields in technology — blending mechanical systems, embedded software, controls, perception (AI/vision), modelling, simulation and systems integration. Hiring managers in this space are highly selective because robotics teams need people who can solve real-world problems under constraints, work across disciplines, and deliver safe, reliable systems. And here’s the reality: hiring managers do not read every word of your CV. Like in many tech domains, they scan quickly — often forming a judgement in the first 10–20 seconds. In robotics, those first signals are especially important because the work is complex and there’s a wide range of candidate backgrounds. This guide unpacks exactly what hiring managers look for first in robotics applications and how to optimise your CV, portfolio and cover letter so you stand out in the UK market.

The Skills Gap in Robotics Jobs: What Universities Aren’t Teaching

Robotics is no longer confined to science fiction or isolated research labs. Today, robots perform critical tasks across manufacturing, healthcare, logistics, agriculture, defence, hospitality and even education. In the UK, businesses are embracing automation to improve productivity, reduce costs and tackle labour shortages. Yet despite strong interest and a growing number of university programmes in robotics, many employers report a persistent problem: graduates are not job-ready for real-world robotics roles. This is not a question of intelligence or dedication. It is a widening skills gap between what universities teach and what employers actually need in robotics jobs. In this article, we’ll explore that gap in depth — what universities do well, where their programmes often fall short, why the disconnect exists, what employers really want, and how you can bridge the divide to build a thriving career in robotics.

Robotics Jobs for Career Switchers in Their 30s, 40s & 50s (UK Reality Check)

Robotics looks futuristic from the outside. People picture humanoid machines, cutting-edge labs & young engineers writing complex code. In the UK job market, the reality is more practical and more encouraging for career switchers: robotics is already embedded across manufacturing, logistics, healthcare, agriculture, defence, construction & inspection. That means there are real jobs for people in their 30s, 40s & 50s who bring operational experience, delivery skills, quality discipline & the ability to work with real-world systems. This article gives you a clear UK reality check on robotics careers for career switchers: what roles genuinely exist, which paths are most realistic, what skills employers actually hire for, how long retraining tends to take & whether age is a factor.