A lead researcher explaining the benefits of Cognition.run

Counterbalancing and Balanced Condition Assignment in jsPsych Without Writing a Server

The problem: jsPsych has no memory between participants

A jsPsych experiment runs entirely in the participant's browser. When the page loads, the script starts from scratch: it does not know how many people have already taken part, which conditions they were assigned to, or what happened in any other tab or device. This is exactly why between-subject counterbalancing is one of the most recurrent questions in the jsPsych community; see, for example, GitHub discussions #1532, #1719 and #2262.

The jsPsych.randomization module is excellent at randomizing within a participant: randomize_order: true on a timeline, jsPsych.randomization.shuffle(), sampleWithoutReplacement(), factorial() and friends. None of them can balance between participants, because balancing requires a shared counter that survives across sessions, and that counter has to live somewhere outside the browser.

When simple random assignment is enough

The simplest approach is to draw a condition at random on every load:

const condition = jsPsych.randomization.sampleWithoutReplacement([1, 2], 1)[0];
jsPsych.data.addProperties({ condition });

With hundreds of participants this converges to an even split and is perfectly defensible. With small samples it does not. With 40 participants and two conditions, a split of 26/14 or more lopsided happens about 8% of the time (two-tailed, Binomial(40, 0.5)). One study in twelve ending up with one group almost twice the size of the other is not a rare accident, and it costs you statistical power and, for order effects, interpretability. If your N is small or your conditions are expensive to fill, you want balanced assignment rather than independent coin flips.

Server-free workarounds

a) Assign by URL parameter

Read the condition from the query string and hand out different links:

const params = new URLSearchParams(window.location.search);
const condition = Number(params.get("condition")) || 1;
jsPsych.data.addProperties({ condition });

You then control the balance yourself: on Prolific, publish one study per condition with the same number of places, each pointing to ?condition=1, ?condition=2, and so on. On Cognition, single-use links carry a sequential ?link=1..N parameter, so link % 2 (or % 4) gives you a strict alternation of conditions across the links you distribute, and the query parameters are stored as columns in the data automatically.

b) Derive it from the participant ID

If every participant arrives with an ID (for example PROLIFIC_PID), hash it and take the remainder:

function hashString(s) {
  let h = 0;
  for (const c of s) h = (h * 31 + c.charCodeAt(0)) | 0;
  return Math.abs(h);
}
const pid = new URLSearchParams(window.location.search).get("PROLIFIC_PID") ?? "anonymous";
const condition = (hashString(pid) % 2) + 1;

This is deterministic and reproducible: the same participant always lands in the same condition, even if they reload. It is still random assignment in disguise, though, so it gives no guarantee of balance. See our guide on integrating jsPsych with Prolific, SONA and MTurk for how these IDs reach your code.

c) Time of day or date

Assigning by the minute or the weekday is tempting but confounds condition with recruitment time: people who respond to a posting in the first hour differ from those who trickle in later, and platforms release participants in bursts. Avoid it.

The server-side solution: balanced assignment on Cognition

When a server hosts the experiment it can keep the counter for you. On Cognition you set the number of between-subject conditions, N, in the task settings. Each time a participant starts a run, the server counts the runs already assigned to each condition (finished runs plus runs currently in progress; declined consents, timeouts and dropped runs are excluded), picks the condition with the lowest count, and breaks ties at random. The result is exposed as window.CONDITION, an integer from 1 to N, before your code executes, and stored in the condition column of every run, visible in the runs table and in every CSV or JSON row you download.

const condition = window.CONDITION ?? 1; // 1..N, assigned by the server
jsPsych.data.addProperties({ condition });
const stimulusOrder = condition === 1 ? ["A", "B"] : ["B", "A"];

The ?? 1 fallback keeps the code runnable in the editor preview or on your own machine. You can also force a condition with ?condition=2 in the URL, which is handy for testing each branch and for topping up a condition at the end of data collection. It combines naturally with "stop after N participants", so a study configured with 4 conditions and a cap of 80 runs ends close to 20 per cell. Note that there are no per-condition quotas or stratification by demographic variables: balance is on counts only.

A Latin square with four conditions maps cleanly onto CONDITION:

const latinSquare = [
  ["A", "B", "C", "D"],
  ["B", "C", "D", "A"],
  ["C", "D", "A", "B"],
  ["D", "A", "B", "C"],
];
const condition = window.CONDITION ?? 1;
const blockOrder = latinSquare[condition - 1];
jsPsych.data.addProperties({ condition, block_order: blockOrder.join("") });

const timeline = [];
for (const block of blockOrder) {
  timeline.push({
    timeline: [trial],
    timeline_variables: stimuliByBlock[block],
    randomize_order: true,
    data: { block },
  });
}
jsPsych.run(timeline);

Mapping common designs to N conditions

  • Two groups (treatment vs. control): N = 2; condition === 1 selects the manipulation.
  • 2 × 2 between-subject factorial: N = 4. Decode with const factorA = (condition - 1) % 2; const factorB = Math.floor((condition - 1) / 2); and store both factors as data properties.
  • Block order (ABBA): N = 2, condition 1 runs A-B-B-A and condition 2 runs B-A-A-B, so each block appears equally often in each position across participants.
  • Response-key counterbalancing: N = 2, with const keys = condition === 1 ? { yes: "f", no: "j" } : { yes: "j", no: "f" }; and choices: [keys.yes, keys.no]. Multiply N by 2 if you combine it with another factor.

Common mistakes

  1. Editing the condition logic mid-collection. If you change what condition 2 means after 30 people have run, the server's counts no longer describe what participants actually saw. If a cell needs filling, leave the code alone and force it with ?condition= instead.
  2. Counting your own test runs. Every normal visit creates a run and moves the counters. Test with ?demo=true, which runs the task without creating a run, or delete the test runs before recruiting.
  3. Not saving the condition in the data. Cognition stores it in the condition column, but call jsPsych.data.addProperties({ condition }) anyway so it appears on every trial and survives any export pipeline.
  4. Confusing within- and between-subject randomization. randomize_order shuffles trials for one person; it does nothing for group balance.

Summary

jsPsych gives you everything you need to randomize within a session, and with URL parameters or hashed IDs you can get reasonable between-subject assignment without any backend. When you need the groups to actually come out even, let the host do the counting: paste your code into Cognition, set the number of conditions, and read window.CONDITION. Create a free account to try it, or read the documentation first.