Data Engineer (Intelligent Augtomation) - Remote - 6 months

JOB TYPE: CONTRACT

JOB ID: 47228

SALARY TYPE: HOURLY

PER DAY PAY: From GBP £0.00 to GBP £0.00

Vacancies: 1

ADDED DATE: 10-04-2025

JOB DESCRIPTION:

SC Cleared - Data Engineer (ETL/ELT Specialist) - Automation and Innovation Project

 

Join a transformative Automation and Innovation project within central Government as a Data Engineer specializing in ETL/ELT processes and data pipeline development. This role offers the opportunity to design and build robust, scalable data infrastructure to support cutting-edge automation initiatives and public service transformation.

 

Key Details:

• Location: Remote-working with occasional visits to London office
• Rate: £500.00pd (Inside IR35)
• Security Clearance: SC Cleared (or eligibility to obtain SC clearance)

 

Responsibilities:

• Design and implement ETL/ELT pipelines using Azure Data Factory, SSIS, and Python for secure and efficient data ingestion into Azure Data Lake Gen2.
• Develop scalable workflows for data transformation and processing using tools like PySpark or Azure Databricks.
• Optimize data storage structures in Azure Data Lake to support analytics and machine learning initiatives.
• Ensure data quality, security, and compliance with government standards through robust governance frameworks.
• Collaborate with cross-functional teams to automate processes and improve data infrastructure performance.

 

Required Skills:

• Expertise in Azure Data Factory and SSIS for ETL/ELT pipeline orchestration.
• Proficiency in Python for automation and complex transformations.
• Experience with Azure Data Lake Gen2 for secure, scalable storage solutions.
• Knowledge of cloud-based data processing tools such as Spark or Azure Synapse Analytics.
• Strong understanding of data governance, security, and GDPR compliance practices.

 

This role offers the chance to work on a high-impact project with significant autonomy, driving innovation in government operations through advanced data engineering solutions.

 

At Lucid, we celebrate difference and value diverse perspectives, underpinned by our values 'Honesty, Integrity and Pragmatism'. We welcome applications from all suitably qualified individuals regardless of personal characteristics. If you require support during the recruitment process, please contact us directly.