The Daily Diff

Google Research: Recall Is the Bottleneck for LLM Factuality

Thu, 20 Aug 2026 11:06:25 GMT

When an LLM gets a fact wrong, is the fact missing (an "empty shelf") or present-but-unretrievable ("lost keys")? Jordan and Riley unpack Google Research's argument that standard accuracy metrics collapse these two cases — even though one calls for scaling model/data and the other for cheaper post-training/inference-time fixes. They walk through "knowledge profiling" (analyze the state of each fact, not each question) and its five profiles, the three behavioral notions (encoding via priming, knowledge across phrasings incl. the reversal curse, and recall — direct vs. inference-without-encoding), and WikiProfile, a 2,150-fact benchmark (10 tasks each) built with a Gemini-2.5-Pro + search-grounded pipeline. Headline finding on frontier models (Gemini 3, GPT-5): many factual errors are recall failures, not encoding failures — the knowledge is often in there, the model just can't reach it on demand. Practical takeaway: before assuming the model doesn't know something, test whether it can recognize it or reach it with chain-of-thought; if so, the fix is on your side of the API, not in the weights. Source: Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality — Google Research Blog, Aug 12 2026 (paper: arXiv:2602.14080) — https://research.google/blog/empty-shelves-or-lost-keys-recall-is-the-bottleneck-for-parametric-factuality/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Anthropic: How We Contain Claude Across Products

Thu, 20 Aug 2026 11:06:27 GMT

A rare, honest look at agent security from a lab shipping agents at scale. Jordan and Riley get into the central idea — "blast radius" (likelihood of failure x damage per failure): safeguards keep pushing likelihood down, but the worst-case damage only grows as agents gain capability and access, so the engineering job becomes bounding the damage, not preventing every failure. They cover why human-in-the-loop degrades (Anthropic's telemetry: users approved ~93% of Claude Code permission prompts -> approval fatigue), the shift to containment (sandboxes, VMs, egress controls), three risk types (user misuse, model misbehavior, external attackers), and the three containment shapes for claude.ai / Claude Code / Cowork matched to how much oversight each user can give. Best lessons: "the software you build yourself is the weakest" (their custom allowlist proxy failed while stock hypervisor/gVisor/seccomp held); the egress incidents where the model layer had nothing anomalous to catch; VM isolation locking EDR out too; tool/MCP output as a prompt-injection surface; and forward risks like persistent memory poisoning (CLAUDE.md, agent state dirs), multi-agent trust escalation, and agent identity. Source: How we contain Claude across products — Anthropic Engineering Blog, 2026, by Max McGuinness, Mikaela Grace, Jiri De Jonghe, Jake Eaton & Abel Ribbink — https://www.anthropic.com/engineering/how-we-contain-claude This is commentary/summary in the hosts' own words, not a reproduction of the article.

Cloudflare: Tracking RFC 9234 Adoption (Stopping BGP Route Leaks)

Wed, 19 Aug 2026 11:37:30 GMT

Route leaks — when a BGP announcement propagates beyond where it should, often black-holing traffic — have historically been guarded by hand-written operator filters. Jordan and Riley walk through how RFC 9234 moves that protection into the protocol itself, using the BGP Role capability (neighbors negotiate provider/customer/peer up front) and the Only-to-Customer (OTC) attribute (routes get stamped so compliant routers auto-reject leaked ones), enforcing "valley-free" routing automatically. The catch from Cloudflare's real-world measurement: some large Tier-1 networks strip the OTC attribute as they forward routes, undermining protection for everyone downstream — a classic internet collective-action problem where the safety property only fully holds if the backbone cooperates. Source: BGP Role model: tracking the adoption of RFC 9234 — The Cloudflare Blog, Aug 18 2026 — https://blog.cloudflare.com/rfc9234-bgp-role-model/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

AWS: Consistency Is the New Latency (AI at the Data Layer)

Wed, 19 Aug 2026 11:37:33 GMT

As AI shifts from chatbots to autonomous agents, AWS argues data-layer consistency matters as much as speed — because an agent reads a stale replica as ground truth and acts on it (their example: a 2-second lag makes an inventory agent halt a sale despite 500 units in stock). Jordan and Riley get into "silent poison" and "hallucination debt" (agents writing wrong conclusions back as permanent memory), and the three patterns for matching a consistency model to each workload: Aurora Global Database / DSQL for high-stakes data, DynamoDB Global Tables with conditional writes for shared agent memory, and Keyspaces for high-velocity intake. Source: Consistency is the new latency: AI at the data layer — AWS Architecture Blog, Aug 18 2026 — https://aws.amazon.com/blogs/architecture/consistency-is-the-new-latency-ai-at-the-data-layer/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Stripe: Reducing FX Costs

Tue, 18 Aug 2026 12:06:12 GMT

Stripe's new currency features aimed at the "double FX trap" — paying conversion fees twice when you convert inbound payments to your home currency, then convert back to pay international expenses. Jordan and Riley walk through the two connected capabilities: expanded multicurrency settlement (settle in the currency the customer paid in — up to 18 currencies across 37 markets by end of 2026) and instant currency conversion (real-time across 15 currencies, transparent pricing, 24/7, with API access). They also flag it's a product-announcement post, not a deep systems piece — the engineering win is removing an unnecessary conversion step, not a novel algorithm. Source: Reduce FX costs with Stripe — Stripe, Aug 17 2026 — https://stripe.com/blog/reduce-fx-costs-with-stripe This is commentary/summary in the hosts' own words, not a reproduction of the article.

AWS + Bosch: Serverless Vehicle Tracking at Scale (L.OS)

Tue, 18 Aug 2026 12:05:02 GMT

How Bosch's L.OS uses a serverless "horizontal integration layer" on AWS to unify India's fragmented logistics telematics — dozens of incompatible providers, formats, and APIs. Jordan and Riley cover the three standardized workflows (discovery, consent-based tracking, termination) and the architecture: a Central Tracking Connector on ECS Fargate, per-provider Lambda adapters that quarantine each vendor's quirks, MSK as the async event bus, DynamoDB for rules, API Gateway as the unified entry point. Numbers: ~35,000 trips/day, provider onboarding cut from 2–4 weeks to ~3 days. Takeaway: standardize in the middle, push vendor/region-specific chaos to thin, independently scaling serverless adapters. Source: Serverless vehicle tracking at scale: Bosch L.OS on AWS — AWS Architecture Blog, Aug 14 2026 — https://aws.amazon.com/blogs/architecture/serverless-vehicle-tracking-at-scale-bosch-l-os-on-aws/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Cloudflare: Detecting & Securing MCP Traffic

Tue, 18 Aug 2026 12:04:59 GMT

As AI agents use the Model Context Protocol to call tools, orgs lose visibility into which MCP servers those agents reach — and MCP traffic is cryptographically indistinguishable from ordinary HTTPS. Jordan and Riley dig into the resulting "shadow MCP" and "portal bypass" problems, and how Cloudflare detects MCP at the network boundary using spec HTTP headers and the JSON-RPC envelope (surfaced as experimental.is_mcp == true) instead of hostname guessing — plus a new MCP dashboard and Gateway block policies. Governance loop: discover → validate → migrate behind portals → enforce portal-only. Source: How Cloudflare detects MCP traffic and helps secure it — The Cloudflare Blog, Aug 14 2026 — https://blog.cloudflare.com/mcp-security-updates/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Shopify: Raising Mobile E2E Test Stability to 98%

Tue, 18 Aug 2026 12:02:21 GMT

Shopify's mobile end-to-end suite had decayed to ~50% stability — blocking more good PRs than it caught real bugs. Jordan and Riley cover how the team rebuilt it: an opinionated wrapper around Appium with a builder-style API that forces an assertion at every step (doubling as a condition-based wait, killing hardcoded sleeps), plus computer-vision element finding (PaddleOCR for text, OpenCV matching icons against Polaris design-system SVGs) so tests locate elements the way a user would. Test IDs became opt-in fallbacks with an "UNSAFE_" prefix. Result: 98% stability. Takeaway: frameworks that expose low-level controls without guardrails accumulate anti-patterns — make best practices the path of least resistance. Source: How we raised mobile end-to-end test stability to 98% — Shopify Engineering, Aug 12 2026 — https://shopify.engineering/mobile-e2e-testing This is commentary/summary in the hosts' own words, not a reproduction of the article.

Uber: Cost-Efficient Export Workloads

Tue, 18 Aug 2026 12:01:02 GMT

How Uber slashed the cost of highly selective "export" queries — like Data Subject Access Requests that pull a handful of records out of massive historical datasets. Jordan and Riley get into why these queries trigger full-partition scans that keep cold data "hot" and defeat GCS auto-tiering, why Uber passed on Hudi's Secondary Index, and the combo that actually worked: Hudi column stats for file pruning plus sorting tables by the predicate column so a user's records cluster into a few files. Reported ~24.8% disk reduction on a tested partition. Takeaway: access patterns and file layout drive storage cost more than raw data volume. Source: Running Cost-Efficient Export Workloads at Uber — Uber Engineering, Aug 12 2026 — https://www.uber.com/us/en/blog/running-cost-efficient-export/ This is commentary/summary in the hosts' own words, not a reproduction of the article.

Netflix's Real-Time Distributed Graph, Part 3 — Querying with gRPC

Mon, 17 Aug 2026 14:30:30 GMT

Netflix's serving layer for its Real-Time Distributed Graph, built to answer tens of thousands of queries/sec in under 100ms. Jordan and Riley dig into the two query shapes (shallow-wide vs. deep-narrow), why they chose breadth-first batched traversal over depth-first, the async-first threading model over thread-per-request, and packing multi-hop traversals into a single gRPC call to kill per-hop round-trips.

Source: How and Why Netflix Built a Real-Time Distributed Graph, Part 3: Querying the Graph with gRPC — Netflix Tech Blog, Aug 7 2026 — https://netflixtechblog.com/how-and-why-netflix-built-a-real-time-distributed-graph-part-3-querying-the-graph-with-grpc-0f3468349607

This is commentary/summary in the hosts' own words, not a reproduction of the article.

Netflix's Real-Time Distributed Graph, Part 2 — Building a Scalable Storage Layer

Mon, 17 Aug 2026 14:28:18 GMT

Why Netflix skipped native graph databases (Neo4j, Neptune) for its Real-Time Distributed Graph and built on KVDAL — a key-value abstraction over Apache Cassandra — instead. Jordan and Riley cover the record/item mapping that makes a key-value store hold a graph (nodes as property records, edges as adjacency lists), the per-type namespace model for independent scaling, and the scale: 8 billion nodes, 150+ billion edges, 2M reads/sec and 6M writes/sec at single-digit-millisecond latency across 2,400 EC2 instances.

Source: How and Why Netflix Built a Real-Time Distributed Graph, Part 2: Building a Scalable Storage Layer — Netflix Tech Blog, Nov 14 2025 — https://netflixtechblog.medium.com/how-and-why-netflix-built-a-real-time-distributed-graph-part-2-building-a-scalable-storage-layer-ff4a8dbd3d1f

This is commentary/summary in the hosts' own words, not a reproduction of the article.

Netflix's Real-Time Distributed Graph, Part 1 — Ingesting and Processing Data Streams

Mon, 17 Aug 2026 14:26:38 GMT

The origin story of Netflix's Real-Time Distributed Graph: why microservices left them with data silos across ads, live events, and games, and how member actions become graph nodes and edges. Jordan and Riley cover the API Gateway to Kafka to Flink pipeline (~1M msgs/sec per topic, 5M+ records/sec published downstream), and the two scaling calls Netflix made — one Flink job per Kafka topic, and a separate topic per node/edge type — trading operational overhead for independent scaling.

Source: How and Why Netflix Built a Real-Time Distributed Graph, Part 1: Ingesting and Processing Data Streams at Internet Scale — Netflix Tech Blog, Oct 17 2025 — https://netflixtechblog.com/how-and-why-netflix-built-a-real-time-distributed-graph-part-1-ingesting-and-processing-data-80113e124acc

This is commentary/summary in the hosts' own words, not a reproduction of the article.

Pilot Ep 1 — Cloudflare's Town Lake: A Data Platform, and the AI Agent Built on It

Sun, 16 Aug 2026 13:24:56 GMT

In this episode, Jordan and Riley break down a recent Cloudflare Engineering post on Town Lake, their internal data lakehouse, and Skipper, the AI agent built on top of it. They cover why Cloudflare needed it (a billion+ events/sec scattered across Postgres, ClickHouse, Kafka, and more), how Trino and Iceberg tie it together into one queryable system, the "default-closed" access model for sensitive data, and how Skipper turns natural-language questions into safe, grounded SQL — including a neat trick that cut its tool-calling round-trips from five down to one.

Source: How we built Cloudflare's data platform and an AI agent on top of it — The Cloudflare Blog, May 28, 2026. This episode is commentary and discussion, not a reproduction of the original post — read the full article at the link above.

Google Research: Cardiometabolic Risk from Smartphone Photos (PhotoScan)

Thu, 20 Aug 2026 11:06:23 GMT

A health-AI piece with a clever engineering core. Jordan and Riley cover PhotoScan, a deep-learning framework that estimates 3D body-composition metrics — body fat %, android-to-gynoid (apple vs. pear) fat ratio, and visceral-to-subcutaneous fat ratio — from ordinary 2D smartphone photos, to flag insulin resistance (which precedes type 2 diabetes by years and is poorly captured by BMI). The standout trick solves a data problem: pre-train a ResNet-50 (ImageNet-init) on UK Biobank (N=35,323) using 2D projections rendered from 3D MRI with DXA as ground truth, fuse image features with sex/height/weight/BMI, and output probability density functions (uncertainty, not point guesses); then fine-tune on real smartphone photos (PhotoBIA, N=677, with landmark detection picking best frames from 360-degree video) and validate on an independent cohort (N=132), hitting near-DXA agreement and beating smartwatch impedance sensors — while unlocking ratios impedance can't measure. Transferable lessons: bootstrap a cheap deployment modality from a data-rich hard-to-collect one via projection; fuse image + tabular; predict distributions when stakes are clinical. Caveat: investigational, not a cleared medical device. Source: Seeing beyond BMI: Estimating cardiometabolic risk with smartphone imagery — Google Research Blog, Aug 17 2026 (paper: arXiv:2603.27017) — https://research.google/blog/seeing-beyond-bmi-estimating-cardiometabolic-risk-with-smartphone-imagery/ This is commentary/summary in the hosts' own words, not a reproduction of the article.