Feedback Loops, Position Bias and Exploration
On this page
A video platform retrains its recommender every day on yesterday's clicks. After a few months, the home page shows the same few hundred popular creators to almost everyone. New creators complain that nobody sees their work. Offline metrics look fine, because the test data comes from the same loop. The model has been grading its own homework.
Every model that decides what users see also decides what data it will learn from next. That loop is the most common source of hidden bias in recommendation, search, ads and trust and safety systems, and senior interviewers almost always ask about it.
How it works
The loop

- The model ranks item A above item B.
- Users see A far more often, so A collects most of the clicks.
- Training data says A is clicked a lot and B almost never.
- The next model ranks A even higher.
B may be just as good; it was simply never shown. The model can't learn from outcomes it never observes. The same structure appears everywhere:
- Recommendations and feeds: popularity bias, filter bubbles, new items starved of impressions.
- Search: results on page 2 rarely get clicks, so they rarely move up.
- Fraud and moderation: blocked payments and removed posts never show whether they were truly bad, so the model can't learn from its own false positives.
- Pricing and credit: you only see whether people pay at the prices you offered, or repay the loans you approved.
Position bias

A special case: items at the top of a list get clicked more because they are at the top. If you train on raw clicks, the model learns "whatever was ranked first is good", which reinforces the previous model's choices. Fixes:
- Position as a feature. Include the displayed position in training, so the model can attribute part of each click to position. At serving time, set it to the same value (say 1) for every candidate, so items are compared on their merit alone.
- Inverse propensity weighting. Estimate how likely an item at position k is to be seen at all (from experiments that shuffle results), and weight each click by 1 / that probability.
- Labels less tied to position. Long watch time, purchases or saves depend less on placement than a click does.
Exploration
The only way to learn about items the model doesn't show is to show them sometimes.
- Epsilon-greedy: on a small share of requests (for example 1–5%), or in one or two slots per page, show items chosen at random from a broader candidate pool, or new items.
- Boosting new items: give new items a temporary bonus or a guaranteed number of impressions, so they collect enough data to be judged (see Cold Start).
- Bandit methods: Thompson sampling or upper-confidence-bound methods show uncertain items more when their potential upside is high, and less as evidence accumulates. They balance exploring with exploiting automatically.
- Holdouts for trust and safety: approve a tiny random sample of payments the model would block, or send a random sample of all posts to human review, to measure what the model misses and to get unbiased training labels.
Exploration data is also the cleanest data for offline evaluation, because it isn't filtered by the current model.
Trade-offs and costs
- Exploration costs engagement now in exchange for better models later. Keep it small and targeted: new items, uncertain items, a fixed slot.
- Holdouts cost money or safety in fraud and moderation. Use small samples, cap the amount at risk, and never apply them to the most severe categories.
- Debiasing adds variance. Inverse propensity weights can be very large for low positions; cap them.
- Diversity rules in re-ranking reduce the loop's grip but can hurt short-term metrics. Judge them with long-term holdouts.
When to reach for it
In the monitoring and deep-dive part of any ranking, recommendation or trust and safety design. Typical interviewer prompts: "won't the model just keep recommending the same things?", "how do new items get a chance?", "your training data comes from your own model; is that a problem?", "how do you know what your fraud model blocks wrongly?"
Example
Prompt: "Design a news feed ranking system."
- Loop risk: posts from close friends and big pages get the top slots, collect the engagement, and the model learns to show them even more.
- Position: log the slot of every impression; train with position as a feature and serve with it fixed.
- Exploration: in about 2% of feed loads, one slot in the top 10 is filled from a wider pool (newer posts, smaller pages, friends the user rarely sees).
- Labels: combine clicks with longer dwell, comments and shares, and "see less of this" as a negative.
- Evaluation: offline metrics are computed on the exploration traffic as well as normal traffic; a long-term holdout tracks whether the feed's variety and users' return rate change over months.
Common mistakes
- Ignoring the loop entirely, and evaluating on data generated by the model being replaced.
- Training on raw clicks without logging position.
- No exploration, so new items never get impressions.
- Assuming blocked means bad in fraud or moderation, with no holdout to check.
- Large random exploration that visibly hurts the product. Keep it small and targeted.
Say this in the interview
"This system trains on data it generated, so it will keep favouring what it already shows. I'd log the position of every impression and train with position as a feature, fixed at serving. I'd reserve one slot in the top ten on a small share of requests for exploration, boost new items for their first few thousand impressions, and use that exploration traffic for offline evaluation. A long-term holdout checks whether variety and retention change over time."