DataPains on YouTube
@DataPains
ML PLATFORM · DATA ENGINEERING · AI · TOOLS THAT MATTER IN PRODUCTION
Master Apache Spark on Kubernetes and Beyond!
Unlocking the Power of ArgoCD: A Game Changer for Kubernetes Users
ML Platform · DataOps · Research Data Engineering
I build the data infrastructure that trains multimodal AI across audio, video and robotics, from Lakehouse and DataOps foundations through to self-serve workflows for ML research. A decade across data and ML, and the creator of DataPains.
10+ years building scalable, governed data systems on AWS and GCP across Lakehouse architecture, DataOps pipelines and production-grade ML infrastructure. Most recently at Odyssey, engineering end-to-end multimodal data for world-model research across audio, video and robotics. Before that, Tech Lead for ML Platform & Data Engineering at Synthesia, serving a 40+ researcher R&D organisation, and Lead Data Engineer at 7bridges.
DataPains is where I share what I've learned. Practical, hands-on content on data engineering, AI tooling and the platforms that earn their place in production. Conference speaker at Big Data London and DataNova 2023. Featured on the Data Team Success podcast.
ML PLATFORM · DATA ENGINEERING · AI · TOOLS THAT MATTER IN PRODUCTION
MEDIUM
A look at whether F3's table format improvements actually address the deeper challenge of unifying vector and structured data in modern AI stacks.
Read on Medium →MEDIUM
How to think about cold, inactive data in a Lakehouse: lifecycle policies, tiering strategies, and the cost of keeping everything hot.
Read on Medium →MEDIUM
A mental model for structuring data transformations and semantic layers: where dbt fits, where it stops, and how to draw the right boundaries.
Read on Medium →Designed and built end-to-end architecture for multimodal and robotics data processing that powers world-model research, spanning audio and video. Led audio data processing and research across diarization, emotion, pitch and IPA sound effects, plus audio tokenisation to speed up data loaders. Built caption strategies with Gemini and self-hosted Qwen Omni, processed as T5 encodings, alongside AV and egocentric multi-camera human data pipelines. Optimised GPU workloads across multi-region and shared-GPU setups, adopted data-locality strategies, and shipped self-serve workflows that sped up researchers.
Led Data Platform and DataOps strategy for multimodal ML training across AWS and GCP, owning the quarterly roadmap and technical direction for a 40+ researcher R&D organisation. Delivered a self-service ML training framework with full data lineage, provenance tracking and multimodal observability. Architected a Lakehouse platform on Delta and Trino with federated query across distributed data domains, and designed the WEKA to S3 migration that cut storage spend by $5,000+/day through lifecycle governance. Built an ML training cost-visibility framework for leadership, standardised inference evaluation pipelines (A/B, MUSHRA) with human annotation, and drove research into semantic layering, RAG/TAG and MCP patterns for dataset and evaluation workflows.
50% reduction in compute and storage costs. 98% query latency reduction via Trino-based Lakehouse architecture on GCP. Championed end-to-end DataOps platform build across batch and streaming. Conference speaker at Big Data London and featured in Starburst's DataNova 2023 success story.
Contracted to provide data platform architecture guidance and hands-on technical leadership to enterprise clients.
Built real-time sports data pipelines processing millions of events across streaming and batch. Designed Lakehouse architecture on Apache Druid. PySpark, Kafka, and AWS-native services powering live sports analytics products.
Improved PySpark/Hive data pipelines into BigQuery. Built the company's data science platform. Introduced Apache Airflow from scratch and established data engineering best practices.
Engineered large-scale data modelling with real-time and batch processing for analytics. Led the data warehouse transition from Postgres to Redshift.
Maintained Redshift with Alembic and worked closely with data analysts to deliver business needs, from ad-hoc work to evolving existing architecture.
Worked closely with OEMs and their delivery partners to develop connectivity for the modern automotive ecosystem. Engineered microservice infrastructure on Ericsson's Apcera-based cloud (similar to Kubernetes).
Where it started. Transitioned into software engineering focused on fleet-management system analytics, GPRS/GPS and CAN hardware integration, and data preparation for early machine-learning solutions.
BSc Computer Science & Engineering, Halmstad University, Sweden (2014–2017).
Exchange, Korea University (2016–2017) · 87.2/100 GPA.
Coursera Machine Learning certification.
Best Thesis scholarship
& Sparbanken Kronan scholarship.
Live conference talk on Lakehouse architecture and DataOps in production, at one of Europe's largest data engineering events.
Watch on YouTube →Building a data analytics platform with a Lakehouse at 7bridges, featured by Starburst as a DataNova success story.
Read the story →In conversation with Ross Webb on building high-performing data teams, ML Platform strategy, DataOps culture, and the realities of end-to-end data engineering at scale.
Listen to the episode →Published in IOP Conference Series: Materials Science and Engineering. Early application of machine learning to real-world sensor data for indirect tyre pressure monitoring.
View on Google Scholar →