First-step analysisAbout 6 min · Untimed

Discover

One step, then the same problem

For a random walk with absorbing boundaries, condition on the first move to turn absorption probabilities and expected durations into small linear systems. With u_i as the chance of ever reaching the top starting from i, the first step gives u_i = p * u_{i+1} + q * u_{i-1}. On a fair walk across {0, 1, 2, 3} starting from 2, this yields u_2 = 2/3 (about 0.67).

Words you need

State
Where the process is right now, which is all that matters for what happens next.
Absorbing state
A state you never leave once you reach it, such as going broke or reaching the target.
Random walk
A position that moves up or down by random steps, like your money after repeated coin-flip bets.
First-step analysis
Write the answer from a state as one step plus the average answer from wherever that step lands, then solve the equations.
Expected value (EV)
The average result if you could repeat the bet many times: each outcome times its probability, added up.e.g. A die pays its face in dollars: EV = 3.5.
Why it works

The first move partitions all paths into disjoint cases, so total probability and total expectation split exactly across the neighboring states.

u_i = p * u_{i+1} + q * u_{i-1}, with u_0 = 0 and u_N = 1

Spot it: A walk absorbs at two boundaries and you need absorption odds from an interior start.

Technique reference and sources

After the first coin flip you stand one step higher or lower, facing the identical question with a new starting point. Write the unknown in terms of its neighbors, pin the two boundaries, and solve the chain.

When this method fits

  • A walk absorbs at two boundaries and you need absorption odds from an interior start.
  • You need expected time to absorption from a small state space.
  • An interviewer asks for ruin odds, extinction odds, or hitting probabilities.
  • The chain has 6 or fewer states so the linear system solves by hand.
  1. Name the unknowns. Let u_i be the absorption probability (or expected remaining steps E_i) starting from state i.
  2. Condition on one move. From i you move up with p or down with q: u_i = p * u_{i+1} + q * u_{i-1}. For durations add 1 for the step taken: E_i = 1 + p * E_{i+1} + q * E_{i-1}.
  3. Pin the boundaries. Absorption probability is 0 at the bottom (u_0 = 0) and 1 at the top (u_N = 1). Remaining steps are 0 at any absorbing state.
  4. Solve the chain. Substitute along the line. On a fair walk the probabilities interpolate linearly: u_i = i / N.
Fair walk interpolation: On a fair walk across {0..N}, absorption probability from i is u_i = i / N and expected duration is E_i = i * (N - i).
  • N = 3 from 2: u = 2 / 3 = 0.67 to 2 places
  • N = 4 from 2: duration 2 * 2 = 4
  • N = 4 from 1: u = 1 / 4 = 0.25
One-step plug-in: A single first-step equation for expected terminal payoff evaluates directly once neighbors are known: V_i = p * upper + q * lower.
  • p = 0.6, terminal payoffs 10 and 5: 0.6 * 10 + 0.4 * 5 = 8
  • p = 0.5, absorption neighbors 1 and 0: 0.5 * 1 + 0.5 * 0 = 0.5

Common traps

Writing u_i = p * u_{i+1} + q * u_{i-1} for expected durations without the +1 step cost. Durations consume one step per move: E_i = 1 + p * E_{i+1} + q * E_{i-1}. Dropping the 1 undercounts every state.

Setting interior values as boundary conditions instead of u_0 = 0 and u_N = 1. Only the absorbing states are pinned. Interior values come from solving the system, never from assumption.

Using fair-walk interpolation u_i = i / N on a biased coin. Linear interpolation needs p = 0.5 exactly. Biased walks concentrate geometrically and need the (q/p) ratio formula from the gambler's ruin lesson.

Another example: Biased mini chain {0, 1, 2} with p = 2/3
  1. Only one interior state: u_1 = (2/3) * u_2 + (1/3) * u_0
  2. Boundaries: u_2 = 1 and u_0 = 0
  3. u_1 = (2/3) * 1 + (1/3) * 0 = 2/3 = 0.67 to 2 places

Source notes

These explanations and questions are written for QuantArc. Links show the underlying mathematics and learning design.

Harvard Stat 110: Probability (Joe Blitzstein)
Free course site with videos and the open companion book Introduction to Probability; the topic spine for counting, Bayes, and linearity. Lesson examples and questions are original.

Grinstead and Snell, Introduction to Probability (free PDF)
Free textbook PDF confirmed live: counting methods, poker-style dealing models, and the random-walk chapter behind gambler's ruin and first-step equations. Lesson examples and questions are original.

IES: Organizing Instruction and Study to Improve Student Learning
Supports worked examples, retrieval, explanations and spaced study. Does not establish QuantArc's exact lesson length, mastery threshold or review intervals.