Credibility scoring — how it works.

Two credibility layers

The pipeline tracks credibility at two distinct levels. They serve different purposes and are stored in separate tables.

How aggregators are handled

Sources like Google News are RSS aggregators — they link to articles published elsewhere. When an article arrives via a Google News feed, the pipeline extracts the article's actual destination URL (the publisher's page), then derives outlet_domain from that URL. So a Reuters story republished via Google News gets Reuters' credibility score (0.92), not Google News'.

This is why the "cred." column on the Sources admin page shows 0.5 for Google News feeds — that number is the legacy feed score, which is inert. The badge marked agg on those rows signals that article credibility resolves per-outlet, not per-feed.

Seeded outlet scores

Initial scores were seeded in migration 0047 based on editorial reputation and primary-source status. The seed values are starting points, not permanent judgments:

Domains not in this table default to 0.5. Unknown outlets aren't penalized; they're treated as neutral until enough evidence accumulates to adjust.

EMA feedback loop

Outlet scores drift over time based on admin coding actions. When an admin submits a coding that includes a code with a non-zero reliability_weight, the source's credibility score nudges via an exponential moving average (EMA):

next_score = prev_score + alpha × (target - prev_score)

where:
  target = 1.0  if weight_sum > 0  (reliable code applied)
  target = 0.0  if weight_sum < 0  (unreliable code applied)
  alpha  = learning rate (typically 0.1)

Two seeded codebook codes trigger this automatically:

• high-quality-source — nudges score toward 1.0 (reliability_weight = +1)
• source-unreliable — nudges score toward 0.0 (reliability_weight = -1)

Every nudge is written to source_credibility_events with the before/after scores, alpha, applied codes, and the admin who triggered it. The audit log is append-only and never modified.

How scores affect the pipeline

Credibility scores affect two steps in the scoring pipeline:

1. Topic consolidation. When multiple articles cover the same story, the pipeline picks one representative per cluster. The selection criterion is the credibility_weighted_score:

   credibility_weighted_score = relevance_score × credibility_score

   The article with the higher weighted score wins the cluster. A highly relevant article from a low-credibility outlet can be outcompeted by a slightly less relevant article from a higher-credibility outlet.
2. Relevance routing. The average credibility of representative articles in a category's pool is computed and passed as context. Categories with a low average credibility score may be down-ranked or skipped entirely at the routing threshold — untrusted source pools fall out of future scoring runs without manual intervention.

Score interpretation

What changes here

This page is updated as the methodology evolves. Significant changes — new scoring dimensions, changes to the EMA alpha, additions to the seed table, or changes to how credibility affects routing — will be noted here with a date. Purely additive changes (new outlets seeded at 0.5) are not noted.

Last updated: 2026-05-07 — initial methodology documentation published.