Research Gaps in System Design: What We Still Do Not Know

System design is taught with confident rules: split by domain, cache aggressively, prefer asynchronous messaging. Behind many of them lies less evidence than the confidence suggests. Large parts of everyday architecture rest on experience reports, vendor guidance and folklore. Recognising where the evidence is thin is useful, because it tells you where to rely on your own experiments and where to hold decisions loosely.

Why gaps persist

Data access. The most informative data, such as incident timelines, cost structures and architectural changes over years, sits inside companies and is rarely shared.
Hard-to-control variables. You cannot randomly assign teams to monolith or microservices and hold skill, domain and budget constant.
Long feedback loops. Architectural decisions pay off or fail over years, longer than most studies.
Context dependence. A result for a ten-person startup may not hold for a thousand-engineer organisation.
Publication incentives. Novel algorithms are easier to publish than careful evaluations of unglamorous practices.

Areas where evidence is thin

Architecture style outcomes

Claims that microservices improve team autonomy or that monoliths are simpler to operate are plausible and widely repeated, but rigorous comparative evidence is limited. Most of what we have is case reports, which suffer from selection bias. The honest position: the effects depend heavily on organisational context and operational maturity.

Architectural technical debt

We lack agreed ways to measure architectural erosion and its cost. Code-level metrics exist, but linking structural decay to delivery speed or incident rates remains difficult to do convincingly.

Cost as a design input

Papers rarely include dollar cost. Yet in a cloud, cost per request is a first-class quality attribute. A design that doubles throughput at three times the cost may be a loss. Published evaluations seldom let you make that call.

Operational complexity

Distributed system research tends to measure performance and correctness. Cognitive load, debuggability and on-call burden are rarely measured, even though they often dominate real total cost of ownership.

Gray failures and partial degradation

Formal models mostly assume crash or clean network partitions. Production failures are often partial: slow disks, flaky links, overloaded dependencies. Models of retry storms, metastable failures and cascading overload are still maturing.

Evolution over time

Most studies look at a design at a point in time. How a system's architecture should change as team size, load and requirements grow is far less studied than how to design it initially.

Deciding when evidence is missing

When the literature does not answer your question, treat the decision as an experiment.

Name the uncertainty. "We do not know whether splitting billing into its own service will reduce release conflicts."
Prefer reversible options. Extract a module boundary inside the monolith first, then decide on a separate deployment.
Define a measurable outcome. Lead time for changes, change failure rate, or cost per transaction, measured before and after.
Set a review date. Decide in advance when you will look at the data and what would make you reverse.
Record the reasoning in an architecture decision record so the assumption is visible.

Turning gaps into contributions

If you operate real systems, you hold evidence researchers lack. Practical ways to contribute:

Publish detailed postmortems with timelines, contributing factors and measurements.
Share anonymised before-and-after metrics for architectural changes.
Collaborate with university groups on studies using your data under suitable agreements.
Open-source test harnesses and workload generators.

Even internal write-ups using a consistent template build an organisational memory that beats folklore.

Key takeaways

Much of system design guidance rests on experience and plausibility more than controlled evidence.
Gaps are largest around architecture style outcomes, architectural debt, cost, operational complexity, partial failures and long-term evolution.
When evidence is missing, prefer reversible decisions and run them as measured experiments.
State uncertainties and review dates explicitly in decision records.
Practitioners can close gaps by sharing rigorous, detailed data about their own systems.