Data Engineer
- Duration: Initial 6 months
- Location: Remote
- Rate: £500 - £600 Outside IR35
methodologies:
- Data Architecture Principles: Data lakes, data warehouses, ETL/ELT processes (knowledge Debezium advantageous), data modelling, and data governance.
- Programming Languages: Python (for data manipulation, analysis, and scripting), SQL (for database interaction and data querying), Pyspark, Kafka and potentially Java or Scala for big data processing.
- Database Technologies: Experience with relational databases like MS SQL Server and PostgreSQL, as well as NoSQL databases (e.g. DynamoDB).
- Data Intelligence Platforms: Working knowledge of enterprise DI platforms, specifically
- Databricks, including Data ingestion, streaming, transformation, data integration patterns and tools.
- Data Integration and Pipelines: Building, maintaining, and optimising data pipelines using tools and frameworks designed for efficient data movement and transformation.
- Cloud Data Platforms: Familiarity with cloud services for data engineering, such as AWS (e.g., S3, Redshift, Glue).
- Data Testing and Quality: Implementing data quality checks, data validation, and unit testing for data pipelines to ensure data accuracy and reliability.
- Scripting and Automation: Using Bash, PowerShell, or Python scripts to automate data-related tasks and workflows.
- Version Control: Proficient with Git and GitHub for managing code and data-related configurations.
- Build and CI/CD for Data : Understanding and applying CI/CD principles for data pipelines and deployments, including familiarity with tools like Jenkins, GitLab CI, or similar platforms.
- Data Security: Awareness of data security practices, including access control, data encryption and compliance with data privacy regulations.
- Data for AI products/solutions: Knowledge of the use of AI in the context of data. Practical
- experience of AI and AI tooling is beneficial but not required.
- Strong interpersonal skills with the ability to communicate effectively at all levels
- Analytical thinker with a logical approach to problem-solving and solutions
- Flexible in approach and mindset to adapt to changing priorities and requirements
- Ability to thrive under pressure in a fast-paced environment, able to prioritise tasks and manage your own time appropriately
- Able to work both independently and collaboratively within a team environment to achieve team goals and objectives
Recommended Jobs
CSCS Carpenter
Carpenters needed in Berkshire * 1st & 2nd Fix on Residential Site (Houses & Apartments) * Ongoing work * £23 - £25p/h Depending on experience This is working direct for the Carpentry Subco…
Nursery Deputy Manager
We are looking for a passionate and experienced Deputy Nursery Manager to join our warm, welcoming and nurturing nursery. Our children thrive, our staff feel valued, and we pride ourselves on deliv…
Quantum Engineers - Various Levels
At OQC, we aren’t just theorising about the future; we’re building it. Born from a philosophy of bold innovation, we’ve successfully transitioned quantum computing from an academic dream into a comme…
Fire And Security Service Engineer
Company: MARBLE TALENT GROUP LTD Job Type: Permanent, Full Time Salary: £40000 - £45000/annum door to door travel
Teaching Assistant
TEACHING ASSISTANT Location: Tokers Green, Reading Salary: £12.87ph - £13.75ph (actual annual salary = £22,996 - £24,568 per year) Dependant on experience and qualifications Contract: Pe…
Senior Structural Engineer
Senior Structural Engineer Reading outskirts £55k-£68k DOE plus benefits Ready to step into a role where your expertise truly shapes the built environment? Our client is on an exciting growth journ…
Technical Manager
Technical Project Manager / Technical Delivery Manager Security Tech £55K£65K + Bonus Hybrid (3 Days Remote) Private Healthcare Training Budget Are you ready to lead cutting-edge security projec…
Head of Finance Operations
Head of Finance Operations Salary - dependent on experience, plus a up to 15% discretionary bonus Reading - four days per week on site My client, a leading global business in the Reading are…
Finance Transformation PMO
We’re supporting a public sector programme looking for a proactive Finance Transformation PMO to support delivery across a large-scale ERP transformation. This is a hands-on role — not a Programme Ma…
Science Technician - Independent School in Reading - ASAP...
Our client is seeking a proactive and organised Science Technician to join a high-performing independent school in Reading, known for its academic excellence and well-equipped Science department. •…