Doing a Literature Review in Software Architecture

Before committing to an expensive architectural direction, such as a new data platform or a multi-region strategy, it pays to learn what is already known. A literature review does this systematically. Practitioners do not need the full rigour of an academic systematic review, but borrowing its structure keeps you from reading only what confirms your hunch.

Define the question first

A good review begins with a narrow, answerable question. Compare:

• Too broad: "What is the best way to build scalable systems?"
• Workable: "What approaches exist for maintaining read-your-writes consistency with asynchronous replicas, and what do they cost?"

Write the question, the decision it informs, and a time box, for example three days. Add inclusion criteria: topic, time window, and types of source you will accept.

Choose your sources

Different sources serve different purposes.

| Source | Good for | Watch out for | |---|---|---| | ACM Digital Library, IEEE Xplore | Peer-reviewed studies | Paywalls, dense methods | | arXiv and author sites | Recent work, free access | Not peer reviewed | | Conference proceedings such as SOSP, OSDI, EuroSys, ICSE | Systems and engineering research | Prototype-scale evaluation | | Google Scholar | Citation tracking | Quality varies widely | | Engineering blogs and postmortems | Operational reality | Selection bias toward successes | | Books and surveys | Orientation, vocabulary | Dated material |

Start with a recent survey or a respected textbook chapter to learn the terminology. Terminology is the biggest obstacle: you cannot search for what you cannot name.

Build a search strategy

Write down search strings and record them so the review is repeatable.

("read-your-writes" OR "session consistency")
AND ("replication" OR "replica")
AND (evaluation OR experience OR "case study")

Then use snowballing: look at the references of the best papers (backward) and at papers that cite them (forward). Two or three rounds of snowballing usually surface the core of a topic.

Screen and prioritise

You will find far more than you can read. Screen in layers:

1. Title and abstract. Does it address your question?
2. Introduction and conclusion. What was claimed and with what evidence?
3. Evaluation and limitations. Under what conditions does it hold?
4. Full read. Only for the handful that survive.

Keep a log of what you excluded and why. It prevents re-reading the same dead ends later.

Extract and synthesise

For each included source, capture a consistent set of fields in a spreadsheet or document:

Citation:
Type: empirical study / design paper / experience report / survey
Claim:
Method and workload:
Key assumptions:
Results:
Limitations:
Relevance to our question: high / medium / low

Then synthesise by theme rather than by paper. Group approaches, such as sticky routing, version tokens and synchronous replication for sensitive reads. For each, note the trade-offs reported and where sources agree or conflict. Disagreements are valuable: they usually point to different assumptions that matter.

Report findings usefully

The output should be short and decision-oriented:

• The question and the scope.
• A comparison table of approaches with costs and conditions.
• Confidence level for each conclusion, and why.
• Gaps: what the literature does not tell you, which become candidates for your own experiments.
• Recommendation and the evidence behind it.

Store it where the team can find it, ideally linked from the related architecture decision record.

Pitfalls

• Confirmation bias. Searching only for support for a favoured option. Include terms for criticism and failure, such as "limitations", "lessons learned" or "postmortem".
• Recency bias. Foundational work, like Lamport's papers on time and ordering, remains relevant long after publication.
• Authority bias. Judging by the name of the venue or company rather than the method.
• Never finishing. Without a time box, reviews expand indefinitely.

Key takeaways

• Begin with a narrow question, a decision it supports, and a time box.
• Use a survey to learn vocabulary, then search with recorded strings and snowball through citations.
• Screen in layers and log exclusions.
• Extract consistent fields and synthesise by theme, paying attention to disagreements.
• Deliver a short report with confidence levels and explicit gaps.
• Actively guard against confirmation, recency and authority bias.