Python Software Engineer (Data Engineering)

Company Description

YouGov is an international online research data and analytics group.

Our mission is to offer unparalleled insight into what the world thinks.

Our innovative solutions help the world’s most recognized brands, media owners and agencies to plan, activate and track their marketing activities better.

At the core of the YouGov platform is an ever-growing source of consumer data that has been amassed over our twenty years of operation. We call it living data. All of our products and services draw upon this detailed understanding of our 24+ million registered panel members to deliver accurate, actionable consumer insights.

As innovators of online market research, we have a strong reputation as a trusted source of accurate data and insights. Testament to this, YouGov data is regularly referenced by the global press, and Meltwater 2022 report concluded that YouGov is the most quoted research company in the world.

With operations in the UK, the Americas, Europe, the Middle East, India and Asia Pacific, we have one of the world's largest research networks.

The Culture

Diversity and inclusion are fundamental to YouGov. We are committed to giving the world a voice by capturing the opinions of all groups, including the ones that are often under-represented in research. We are also committed to making sure that our products and tools are free from any bias, as accuracy is key to what we do. None of the above can be done without having a truly diverse workforce, in an inclusive workplace. We are very keen on attracting and retaining the best talent. And best talent also means a diverse pool of talent, with various backgrounds and perspectives.

Supporting the wellbeing of our staff, including maintaining a good work and life balance, is important to us. We support flexible working arrangements where appropriate for a role, with many locations offering a hybrid office-and-remote working approach.

As an Equal Opportunity Employer, qualified applicants will receive consideration for employment without regard to race, colour, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, veteran status, disability status, or any other characteristic protected by law. All employment decisions are made on the basis of occupational qualifications, merit, and business need.

Job Description

Your mission is to build high-quality, automated data pipelines and infrastructure to serve YouGov’s syndicated data products, including YouGov Profiles — one of the largest market research datasets in the world.

Working inside the Syndicated Data Infrastructure team, you will collaborate with data-minded people to convert vast troves of raw consumer data into meaningful insight by developing automated ETL applications, streaming data processors, RESTful microservices and browser-based user interfaces.

Day to day you will:

• Build new features for our automated data infrastructure and the associated web interfaces, for example to support new sources of data

• Optimise ETL and web applications to increase performance, reliability and visibility

• Support existing data pipelines, ensuring every deliverable reaches its intended destination

• Collaborate with other teams to design appropriate, automated solutions for new data delivery and integration needs

Qualifications

You must:

• Love writing beautiful, idiomatic Python

• Be comfortable in the modern realm of test-driven, version-controlled software development

• Have experience in building substantial ETL pipelines and web applications

• Enjoy solving complex technical problems

• Be eager to develop new skills and expertise

• Be proactive, positive and professional

Experience with the following technologies is desirable:

• Luigi, Flask, SQLAlchemy, Pandas

• Docker, Kubernetes

• JavaScript, HTML, CSS

• SQL, Postgres

• Redis, RabbitMQ

• Amazon S3, EC2, Redshift

• Airflow, dbt

• Snowflake

Additional Information

This role can be based in the UK or any EU country in which we have an entity (Spain, Italy, Poland to name a few) and is 100% remote.

To find out how we collect and use your personal data when you apply for a role at YouGov, please read our privacy notice at https://jobs.yougov.com/privacy