Measure Reach on the Log Scale: 738 Weeks Became 30 on the Same 182 Posts
· 6 min read · by Michael Kaminski
My LinkedIn scheduler moves a slot on eight observations, so I priced the plan to fix it with volume. On raw impressions (182 posts, mean 223, SD 983, one post carrying 31.9% of all impressions) a two-arm test for a 20% lift needs 7,747 posts per arm, 738 weeks at 21 a week. On log impressions the same posts need 314 per arm, 30 weeks. Reach is multiplicative, so the log scale asks the question the data was generated by. The admitted cost: a log-scale test measures the geometric mean and is partly blind to the 12,959-impression outlier that carried the account.
My LinkedIn scheduler moves a posting slot on eight data points. I ran the power calculation on the plan to fix that with volume, and on raw impressions it needs 738 weeks. On log impressions, the same 182 posts need 30.
The conclusion first: for any metric that multiplies, and reach is one, the scale you measure on decides whether the experiment can ever finish. Impressions, revenue per post, tokens per conversation, latency tails. Test them on the log scale or accept that the test is decorative.
This is the second time I have found a decorative test in my own system. The first was a 14-day window that could only detect a shift of 0.97 standard deviations. That time the fix was more days. This time more days was the trap.
The optimizer decides on eight observations
The scheduler is a Thompson sampler in a 1,171-line Python file. Each arm is a 30-minute bucket inside a daypart, with a Beta(2, 8) prior updated by the engagement rate of every post that landed there. Three dayparts, one arm chosen per daypart, once a week.
Run 2026-09-20, it proposed moving the evening slot from 17:30 to 18:30 on a sampled posterior of 0.266 from an arm with eight observations. The morning slot, twelve. Midday, thirteen.
Eight is a posterior, not a result. The sampler is doing exactly what Thompson sampling does with thin arms, which is draw wide and move often. The retune output prints an estimates-versus-actuals block for the last 14 days, and for the evening slot it reads model estimate 0.0201, actual median 0.0000.
The plan was volume. Twenty-one posts a week, let the arms fill, let the posterior narrow. Before committing a year to that, I priced it.
On raw impressions, a 20% lift takes 738 weeks to detect
The store is one file, posts.jsonl, 278 rows, 182 of them with impressions logged. The calculation is 14 lines of Python on the standard library: json, math, statistics.
The 182 posts: mean 223 impressions, standard deviation 983, median 88.5. The largest post did 12,959. The second largest did 2,164. One post is 31.9% of every impression I have ever logged, and 161 of the 182 posts sit below the mean. Sample skewness is 12.1.
For a two-arm test at 80% power and a two-sided alpha of 0.05, the sample per arm is close to 16σ²/δ². A 20% lift on a mean of 223 is δ = 44.65.
16 × 983² / 44.65² = 7,747 posts per arm. Two arms, 15,494 posts. At 21 a week, 738 weeks. Fourteen years, for one comparison, against a threshold I would actually act on.
The obvious objection is that the 12,959 post is doing all of that. It is doing most of it. Drop it and the standard deviation falls to 254, the requirement to 1,107 per arm, 2,214 posts, 105 weeks. Two years, for one comparison, and I had to throw away the only post that mattered to get there.
On log impressions, the same posts need 30 weeks
Reach is multiplicative. A post that works does not add 200 impressions to a baseline; it multiplies the baseline. The algorithm shows it to a batch, the batch engages or does not, the next batch is sized on the first. That is a compounding process, and compounding processes produce log-normal outcomes.
So take the natural log of every impression count and run the identical calculation. Same 182 posts, same 14 lines, one flag flipped from raw to log.
Log mean 4.66, which back-transforms to a geometric mean of 106 impressions. Log standard deviation 0.81. A 20% lift is now a constant, ln(1.2) = 0.182, because on the log scale a multiplicative lift is an additive shift.
16 × 0.81² / 0.182² = 314 posts per arm. Two arms, 628 posts. At 21 a week, 30 weeks.
Nothing about the data changed. The raw-scale test was asking a question the distribution could not answer, because the variance of a log-normal is dominated by its tail. The log-scale test asks the question the distribution was generated by.
Sanity check on the fit: 82.4% of the posts fall within one log-standard-deviation of the log mean, against 68% for a true normal. Skewness drops from 12.1 to 2.1. Not a clean log-normal. Close enough that the arithmetic is honest, and far closer than the raw scale was.
What the log scale costs
The log-scale test measures the geometric mean. It answers: does this change multiply the reach of a typical post? It does not answer: does this change produce more 12,959-impression posts?
Those are different objectives, and on this account the second one is where the value has been. One post carried 31.9% of all impressions. On the raw scale it sits 13 standard deviations from the mean. On the log scale, 5.9. The log transform pulls it in, which is what makes the test tractable and is also what makes the test partly blind to it.
I do not have a clean way to optimize the median and the tail at once. Extreme-value methods exist for the tail, and they need a tail sample I do not have; there are three posts over 2,000. For now the scheduler optimizes the typical post and I read the outliers by hand.
The second cost is that 30 weeks assumes 21 posts a week, and the weekly scorecard for 2026-09-20 shows 13 of 21 shipped. At 13 a week, 628 posts is 48 weeks. The log scale bought a 25-fold improvement; it did not buy a fast experiment. A two-arm test on this account still takes most of a year, which is why the weekly review adopts at most one change and labels most weeks directional rather than conclusive.
What changed in the engine, and what did not
The power script sits next to the scheduler and takes one argument, raw or log. Before any experiment on this account, it prints the posts per arm and the weeks at current cadence. That number now exists at the moment I am deciding whether to run the experiment, which it did not before.
The scheduler itself did not change. Its reward is engagement rate, clipped at 15% and normalized to [0, 1], and engagement rate is a bounded ratio of two counts. The log-normal problem lives in the impressions denominator and in any test that treats impressions as the outcome. The 1,171-line file does not read the power script, and the retune still prints an arm of eight as if eight were enough.
So the honest state is: the test is fixed, and the optimizer still moves on thin arms every Monday. The warm-up floor is 20 observations per daypart, the floor for a bucket to be a candidate is 3. Raising the bucket floor to something the power calculation would respect is the next edit, and it will make the sampler move less, which is the point.
If you run experiments on a multiplicative metric
First, look at the ratio of the largest observation to the median before you look at anything else. Here it is 146 to 1. Anything past 10 to 1 on a metric that cannot go below zero is a log-normal until proven otherwise, and a t-test on the raw values will need a sample you do not have.
Second, make the lift a multiplier. A 20% lift is ln(1.2) on the log scale regardless of the baseline, which means one power table serves every arm. On the raw scale the same 20% is a different δ for every baseline, and the table has to be rebuilt each time the mean moves.
Third, report what the test is blind to. A log-scale result is a statement about the geometric mean. If the business runs on the tail, say so in the same sentence, or someone will read a significant result as permission to stop looking for the outlier.
The 738-week number was not a bug in the optimizer. The optimizer was fine. The measurement was on the wrong scale, and it had been for 182 posts before I checked.