When Does Decentralization Pay Off? An Empirical and Simulation Study of Data Mesh Operating Models

Question

Data mesh proposes to replace a central data team with domain teams that publish their data as products. Advocates say one team that must understand every source becomes a bottleneck as use cases multiply. Critics say decentralisation multiplies the interfaces between teams. Both arguments rest on one quantity that nobody had measured: how data dependencies between domains are distributed, and how they grow as domains are added.

The paper makes the adoption question falsifiable: under dependency structures measured in a real organisation, when does a federated mesh deliver new analytical use cases faster than a central team at equal headcount, and which conditions decide the outcome? RQ1 covers the structure and growth of dependencies, RQ2 the comparison of operating models across domain count, cross-domain demand and central-team overhead, and RQ3 which factors matter most.

Five hypotheses were written down before the evidence existed: H1a and H1b before the history was mined, and H2a, H2b and H3, with the parameters and analysis plan, in the public repository before any comparative simulation ran. Deviations are reported in the paper.

Method

Repository mining. NAV, Norway's labour and welfare administration, declares its Kafka topics as Kubernetes resources. Each manifest names the owning team and the teams allowed to read or write the topic; a read grant for team C on a topic owned by team O is a directed edge from O to C. We retrieved every such manifest in NAV's public GitHub organisation, rendered the templated ones, and built the graph. For growth, we replayed 3,892 historical versions of every manifest path, including deleted ones, into month-end snapshots. Hub concentration is tested against 2,000 random graphs of the same size; growth is fitted on log E against log N with a block bootstrap.

An analytical queueing model. Each team is a first-come-first-served multi-server queue. A team of $h$ people loses a fraction $\eta$ of each member's time for every additional colleague, so its effective capacity in person-weeks per week is

This capacity is bounded: $\kappa(h)<1/\eta$ for every $h$. Four operating models share the same people and differ only in organisation: a central pool, a mesh of domain teams, a "theater" mesh in which domains exist in name only, and a hub-and-spoke hybrid with a pooled core for provider-side work. From the bound, the central model is unstable for every headcount once

where $\lambda$ is the rate of new use cases per domain, $\bar w_c$ the consumer-side effort and $m_\pi$ the platform multiplier. Two further results are proved: mesh stability depends on the most connected domains, not on how many there are, and a "theater" mesh can never deliver a use case faster than the central model under the same inputs.

The capacity function in the released package is one line:

def capacity(h: int, eta: float) -> float:
    """Effective capacity of a team of h people, in person-weeks per week."""
    return h / (1 + eta * (h - 1))

A discrete-event simulation. The four models run on identical random workloads, so comparisons are paired. Each run has a 26-week warm-up, 104 measured weeks and a 104-week drain. The simulator is verified against Erlang C, the analytical utilisations and thresholds, and the "theater" proposition. The pre-registered global sensitivity analysis was not run for this version; RQ3 is answered from one- and two-factor sweeps.

Data

The snapshot holds 566 Topic manifests in 129 repositories, retrieved on 23 September 2026. Of the templated topics, 105 were rendered with a production variable file, 60 kept their literal access lists, and 70 whose environment could not be resolved are excluded from the primary graph. That graph contains production topics only: 70 teams and 180 directed dependencies. A second graph restricted to catalogued topics (24 teams, 39 edges) stands in for deliberately published data products. The growth series has 65 month-end snapshots from May 2021 to September 2026.

All mined data are public infrastructure configuration that NAV publishes openly. Only team-level aggregates are analysed; no personal data are collected.

Results

Every figure below is a measured result of the study. The section of the paper that reports it is given in brackets, and the PDF is the record.

Structure (RQ1). Of the 4,830 ordered team pairs an all-pairs assumption would count, one in 27 carries a declared dependency: a density of 3.7%. One team consumes data from 31 others, another supplies 23, and the top 10% of teams touch 66% of all edges. Against 2,000 random graphs of the same size, the degree Gini (0.53 against 0.24), the maximum in-degree (31 against at most 9) and the top-decile edge share (66% against 33%) all lie outside the null distribution, with p below 0.001 for each [paper, Sec. VIII-B].

Growth (RQ1). The fitted exponent is a = 2.03 (95% CI [1.99, 2.31]): edges grew quadratically in the number of teams, at a density stable between 3.0% and 4.5%. The all-pairs assumption has the right form but overstates the number of dependencies by a factor of about 27 at every size. The catalogued subset grew sub-quadratically, with a = 1.37 [1.29, 1.50] [paper, Sec. VIII-B].

Operating models at NAV's size (RQ2). At 70 domains on the observed graph, the mesh delivers a use case in 1.17 weeks on average against 1.89 for the central model, a paired difference of -0.72 weeks (95% CI [-0.73, -0.72]). The hybrid takes 1.32 weeks and the "theater" mesh 2.15. The mesh's busiest hub runs at a predicted utilisation of 0.53, above the central pool at 0.49: the mesh is faster because its small teams avoid the pool's overhead and context costs, not because it is less loaded. On the all-pairs graph the central model becomes unstable (utilisation 1.19, mean lead time 16.8 weeks) while the mesh stays at 1.6 weeks; the difference-in-differences is -14.6 weeks (95% CI [-15.4, -13.7]). The "theater" mesh was slower than the central model in all 80 paired runs, and in verification for all 720,955 use cases with 0 violations [paper, Sec. VIII-A and VIII-C].

Scaling and decision maps (RQ2). On graphs that follow the measured growth law the mesh is stable up to 160 domains with a nearly flat lead time, while the central model's lead time rises with size. On the all-pairs graph the "theater" and central models become unstable from 55 and 70 domains; the mesh saturates only near 151. Without communication overhead in the central team, the two models deliver within 8% of each other at every size. Raising the cross-domain share of use cases from 0.05 to 0.6 shrinks the mesh's advantage by 30% to 61% in log-ratio terms without reversing it [paper, Sec. VIII-D].

Which factors decide (RQ3). Central-team overhead dominates; domain count amplifies it; cross-domain demand is second-order; dependency structure matters little in the plausible range; reuse has the smallest effect, moving the log-ratio by at most 0.08 [paper, Sec. VIII-E and VIII-F].

Limitations

One public-sector organisation in one country is studied. The graph records declared sharing through Kafka access lists; products on other platforms and in private repositories are invisible, and template resolution is heuristic. Growth over time conflates organisational growth with adoption of the manifest mechanism.

The model abstracts teams as queues with fixed capacity. It ignores learning, turnover and the cost of reorganising, and measures delivery time only. The most influential parameter, communication overhead in large central teams, is not measured in the data, so results are reported as maps along it. Simulation estimates are precise but conditional on the model, and the global sensitivity analysis that would reveal interactions was not run.

What this means in practice

The paper replaces a yes-or-no answer with conditions an organisation can estimate before deciding. How much does the central team's per-capita throughput fall as it grows? How much context must it acquire per domain? How concentrated are the data dependencies, and are the hub domains staffed for their load? Where the diseconomies of a large central team are small, a central team is not slower. Where they are large, the mesh's advantage grows with every domain added.

Two findings are unconditional. Renaming teams as domains without moving ownership adds hand-offs to the same pool and can only slow delivery. And meshes fail locally before they fail globally: the domains everyone depends on saturate first, which is the reason to staff hubs or pool their provider-side work.

The same discipline applies to AI systems that read from several teams' data: the dependency map is something to measure, not assume, and the capacity of whoever maintains the pipelines is what decides.

Code and data

Everything needed to reproduce the study is in the public repository: the mining pipeline, the graph construction and null models, the analytical formulas, the SimPy-based simulator, the experiments and the figures, as a Python package with 23 tests. The analysis emits the LaTeX macros from which every number in the paper is typeset, so text and results cannot diverge. The mined graphs, history and raw simulation outputs are committed. Code is released under the MIT licence.

Citation

Cite the PDF, version 1.0.0, released on 23 September 2026. The BibTeX entry is below the article; the repository's CITATION.cff carries the same record.