Writing up findings
A template for writing up a study so others can judge it, how to anonymise it, and how to contribute a finding to this guide by citing the rule it tests.
A finding is useful to somebody else only if they can see how it was reached. Write down the question, the method, who took part and how many, what happened and how sure you are. Then say which rule in this guide it supports or contradicts.
This section has no published BSV findings yet. Each one written up this way helps fill that gap.
The template
Copy this for each finding. One study usually produces several; write each up separately, so each can be argued with on its own.
FINDING: [one sentence, stated as what happened, not as advice]
Question What you set out to learn.
Method Interview / usability test / card sort / tree test /
first-click / comprehension check / other. Moderated or not.
Remote or in person. Link to the plan or guide you used.
Who How many people, and how they were recruited. Which groups
(newcomers, people already holding digital money, a persona).
Anybody excluded, and why.
What happened What participants did and said, with counts:
"[n] of [n] newcomers [did what], at [which step]."
Quote participants by code (P03), never by name.
Confidence High / medium / low, and why. Say what would change your mind.
High: seen in most participants of a group, in more than one
session. Low: one participant, or a task wording that may
have led them. Small samples are fine; say how small.
Guide rule The rule this supports or contradicts, by its permanent link:
/guide/rule/<id>/ (supports / contradicts / complicates)
Product What was tested: fictional or anonymised name, or the guide
reference design used as stimulus; version; date.
Limits Test money or real? Who was missing from the sample?
What the task wording might have given away.Do
State each finding as what was observed, with a count, before stating what you think it means.
Do
Report the number of participants and how they were recruited beside every finding, however small the study.
Do
Say how confident you are and why, including what result would change your mind.
Don't
Report a percentage from a small study without the counts behind it. "4 of 7" is honest; "57%" suggests a precision you do not have.
Don't
Leave out a result because it contradicts the guide or your design. A contradicted rule is the most useful thing a study can report.
Citing a rule
Every do and don't on every page has a permanent address of the form /guide/rule/<id>/, where the id is made from the rule's own text. The checklist shows each rule with a button that copies its link, and each page's own checklist does the same.
For example, /guide/rule/4399f2f0dc/ is the address of the rule on Charging per use against renewing an allowance that has ended without asking. A study of allowance renewals would cite it by that link.
The address survives the page moving and edits elsewhere on the page. It stops pointing at the rule only if the rule itself is rewritten, which is the point: a finding should not point at a sentence that no longer exists.
Do
Cite the rule a finding tests by its permanent link, copied from the checklist, rather than by page title or a paraphrase.
Anonymising
The material is financial, and some of it may be traceable on a public ledger long after your notes are deleted.
Do
Replace every participant's name with a code, and keep the key that links codes to people separately from the write-up, then delete it when the study closes.
Don't
Include a handle, address, transaction or balance from any participant's real wallet in a write-up, even partly masked.
Do
Generalise details that could identify somebody in combination: "a shop owner in a small town", not the town and the trade.
Don't
Publish a screenshot or recording frame without checking it for real names, notifications, and other apps' content in the background.
If a quote could identify its speaker to people who know them, paraphrase it and say that you have.
Contributing a finding to the guide
There are two routes, depending on how much you have.
A finding that bears on one rule. Propose a change to the page that states the rule, quoting the rule's permanent link and your write-up. See Contribute and Propose a change. A rule that a study contradicts may be rewritten, turned into an open question, or kept with a note, and the reasoning goes in the change.
A study of a product. Write it up as a case study. The case study brief lists the gaps the guide most wants filled, and several (a permission screen, pay-per-use, recovery measured, a handle rollout) match the usability test plans and vocabulary methods in this section.
Recommendation
Submit a finding from a small study rather than waiting for a large one. Say how small it was; the template already asks you to.
Open questions
- How many independent studies should agree before a finding changes a rule, rather than adding an open question beside it, has not been decided.
Related
Written from: Case studies brief, Contribute, Published research, the guide's rule citation addresses.
Checklist
The rules on this page, in a form you can work through.
11 rules