"Entity resolution is the hard part of esports data"

Bottom line: Fetching esports match data is a solved problem. Deciding that NAVI, Natus Vincere and Na'Vi Junior are two organisations and not one — and keeping that decision stable across six years and 1,483 team entities — is where the work actually is. This is how our pipeline handles it, and where it still fails.

Every "build an esports data pipeline" post spends its length on the fetching. Fetching is the easy half. The moment you hold records describing the same match from more than one vantage point, you own an entity-resolution problem, and it does not go away.

The problem, concretely

A professional Counter-Strike organisation is not a stable identifier. Over the six years our corpus spans, teams have:

• renamed (sponsorship changes, rebrands);
• been acquired, absorbing an existing roster wholesale;
• fielded academy sides whose names are the parent's name plus a suffix;
• appeared under regional or transliterated variants of the same name;
• disbanded and re-formed with an overlapping roster and a new brand.

Only the third and fifth of those should create a new entity. The others should not. Get it wrong in one direction and a team's history fractures across three rows; get it wrong in the other and you have merged an academy side's results into its parent's.

The same problem exists for players, in miniature: nicknames change, and the same person appears across a decade of rosters under two or three handles.

What we actually do

Three mechanisms, in order of how much work they do:

1. A stable internal ID, assigned once. Every team, player and tournament gets a UUID at first sight, and that ID never changes. Names are attributes of an entity, not the identity of one. This sounds obvious. It is the single decision that makes everything downstream tractable, and it is the one most quick pipelines skip because a name is right there and looks unique.

2. An alias table, not a rename. When a team rebrands, we do not overwrite the name — we record the new one and keep the old as an alias pointing at the same UUID. A historical match from 2021 still resolves. Queries against either name land on the same entity. Renames are additive.

3. Idempotent, convergent writes. Ingestion never assumes an empty table and never assumes it is running for the first time. Every upsert is keyed on a natural identifier and produces the same end state whether it runs once or twenty times. This matters more than it sounds: it means a re-run after a partial failure is safe, which means we can re-run freely, which means backfills are cheap.

Where it still fails

Honesty is more useful than a clean narrative, so:

• Academy sides remain the worst case. A suffix convention is not universal, and some organisations field a secondary roster under a wholly unrelated brand. These need a human decision, and we make them by hand.
• Disband-and-reform is genuinely ambiguous. When four of five players carry to a new organisation, whether that is continuity or a new entity is a judgement, not a fact. We treat the organisation as the entity, which means roster continuity does not merge two brands. Reasonable people would choose differently.
• Reconciling depth across events. Per-round detail exists for the events that were covered closely and not for the rest. We do not synthesise it. A gap stays a gap, because an interpolated round is indistinguishable from a real one once it is in the table, and that is a lie you cannot take back.

The lesson

The interesting engineering in a data product is almost never the ingestion. It is the schema decisions that let you keep the data coherent as reality churns underneath it: stable IDs, additive aliases, idempotent writes, and a refusal to invent rows you do not have.

The result of all that is 22,559 completed matches, 4,599 players and 1,483 teams that stay stable across queries — described in more depth in what the corpus actually looks like.

If you want to skip building this, the CS2 data API hands you the resolved version, and the guides show you how to page through it.