Data Engineer

  • Competitive
  • Londres, Angleterre, Royaume-Uni
  • Intérim, Plein-temps
  • Broadgate Search Ltd
  • 15 janv. 18 2018-01-15

See Job Description.

What is the role?

The current data platform is reaching its limits and needs some attention in order to continue its vital role whilst a new platform is being developed. This contract role is to join the existing team in supporting and improving the existing platform as well as starting on the work to build the new platform. 

 

The existing platform consists of ~400 batch jobs written in Python and Shell Script and running in AWS and utilising a number of AWS Redshift warehouses and marts. Qlikview provides the user interface. The future vision is to create a new real time self service data platform capturing and aggregating data across multiple technology platforms from browser events to order tracking events from all group entities. The platform will underpin our test and learn culture supporting the group analytics, BI and machine learning teams.  

What will I be doing?

Maintaining the existing data platform;

Improve and stabilise the existing platform in terms of good software engineering practices such as version control, security as well as moving it to Docker;

Working on the initial build of a new data platform for the group;

Spending time in our Paris office to receive training on the existing platform; 

What will I need to know?

Minimum Degree educated, in a science based subject such as Software Engineering/Statistics/Computing/Physics;

Expert knowledge of relational databases (PostgreSQL, MySQL) ;

Experience of Big Data technologies (e.g. Hadoop, AWS Redshift);

Experience of real-time data processing (e.g. Apache Storm, AWS Kinesis);

Expert knowledge and experience with Python and shell script, with development experience in other languages such as Java, Go, Scala, JavaScript;

you have an appetite for collecting large volumes of data from multiple sources (queues, NoSQL databases, etc.) and for data security;

Ecommerce experience would be an advantage;

Understanding and knowledge of data governance, data management, visualisation, web analytics;

Worked with AWS or other cloud based environment;

Consistently deliver superior technical solutions;

Effective implementation of new ideas/improvements;

Effective interpersonal, written and verbal communication skills is a key requisite;

Ability to work in a collaborative, fast-paced environment and adept at creating practical solutions to daily challenges.