Accountability: Named Owner or Enforced Mechanism?
A side-by-side comparison of the two accountability models engineers actually rely on: a named human owner per outcome, and system-enforced traces, budgets, and isolation.
Loading…
Topic
Decision-Making
12 posts about Decision-Making. Every summary links to the original.
- 12 posts
- 8 companies
- 9 explainers
See the idea
Decision-Making explainers
A/B Testing: The Confidence Budget
A/B testing trades traffic, time, and exposure risk for evidence. This explainer maps the benefit of a controlled comparison, the real cost of running one, and the condition that decides when measuring beats shipping on judgment.
The BI Trade-Off: Answer Speed vs. Time to Trust
Business Intelligence turns operational data into decisions, but every speed-up trades against trust. This diagram maps the benefit, cost, and condition behind governed metrics, shared context, and provenance.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Jev: Bounded semantic judgments for messy text
Jev is a managed API from TypeSafe that performs bounded semantic judgments on messy text, returning probabilities instead of binary answers or generated text. It supports three question types: Choice selects from developer-defined options, Noul provides a probability for yes/no queries, and Score rates on an ordered scale such as frustration level. Each decision uses application state as input and returns confidence values, allowing code to apply business thresholds rather than treating the model’s output as permission. Jev is not self-hostable; it runs as a closed API in TypeSafe’s US West Coast data center, introducing network latency and data transfer concerns for teams outside that region. The article uses a coffee shop metaphor to illustrate Jev’s role as an order taker making bounded judgments, contrasting it with generative models and exact computation, and noting alternatives like open models on vLLM.
The 6 Stages of an Effective, Quick Decision-making Process
According to research by Wal McConnell and Lynsey Duncan, quick decision-making is a startup’s biggest competitive advantage, but it is often dismissed as reckless. The post proposes a six-stage process that addresses two key challenges: balancing intuition, experience, and data as inputs, and creating a feedback loop that studies the decision itself rather than just the outcome. The stages begin with defining the dilemma and establishing a default decision—what you would choose immediately—then ask what evidence would change that choice, research that evidence, identify a set of meaningful alternatives, and make a risk-versus-reward decision with documented rationale. Finally, the process requires reviewing both the outcome and the decision quality, independent of the outcome, to extract learnings and improve future decisions. The approach advocates breaking large decisions into smaller experiments, continuously reviewing assumptions, and iterating quickly to maintain agility and momentum.
Why Spotify Is Not Using Bayesian A/B Testing
Spotify argues that Bayesian A/B testing is not a single method but a family of configurations defined by a prior, likelihood, and stopping rule. It contrasts flat-prior posterior-probability stopping, which can reproduce frequentist peeking false-positive rates, with Bayes-factor stopping, which controls false positives under optional stopping, and calibrated empirical Bayes priors, which can shrink estimates and control false discovery rates. The post frames experiment design around program goals such as limiting harmful or ineffective launches, achieving estimation precision, and minimizing costs rather than selecting a statistical framework first. It says Bayesian and frequentist approaches overlap substantially, while Bayesian advantages such as richer probability statements require additional prior complexity. Spotify therefore continues using frequentist tooling because supporting two modes would add planning, monitoring, interpretation, and configuration costs without a sufficient benefit.
Enterprise Data Strategy Roadmap for Business Outcomes
An enterprise data strategy connects organizational data assets to measurable business outcomes, while fragmented architectures can leave data investments uncoordinated and limit real-time analysis and action. The roadmap starts with purpose, scope, executive sponsorship, measurable objectives, KPI mapping, and use-case prioritization based on business impact, feasibility, time to value, and organizational readiness. It then organizes governance, lifecycle management, data quality, target-state architecture, integration, analytics, team structure, compliance, and measurement as interdependent capabilities, emphasizing owners, stewards, decision rights, executable quality rules, and automated cleansing. Implementation proceeds through a time-boxed cross-functional pilot, documented learnings, and incremental scaling, with steering-committee oversight and governance that evolves through feedback. The text gives indicative timelines of 60 to 90 days for a focused pilot, 12 to 18 months for a foundational platform across multiple business units, and multiple years for a mature data-driven culture.
From emissions reporting to decarbonization decisions
Energy companies have built sophisticated infrastructure for Scope 1, 2, and 3 emissions reporting across complex asset portfolios, but that infrastructure is often oriented toward historical disclosure rather than forward-looking action. Sustainability leaders still need timely answers about which assets warrant intervention, what operational choices drive carbon intensity, and whether current performance is tracking toward commitments. Databricks Genie addresses this gap by letting users query emissions, operational, and financial data in natural language, linking results to dispatch decisions, fuel purchases, and asset utilization. It supports multi-scope analysis, traceability to source data, and scenario questions such as retiring an asset or increasing renewable PPA volume. The stated outcome is a shift from compliance reporting toward data-driven decarbonization decisions, with Genie available today for energy-sector use.
Lessons from Building a First-Pass AI PRD Reviewer at Uber
Uber built an AI-powered PRD Evaluator to help product managers strengthen drafts before they enter structured checkpoint and approval forums. Starting with a PRD, it gathers linked documents, decks, meeting notes, prior experiments, cross-functional artifacts, and Uber-specific context, then classifies the proposal to calibrate review depth. It assesses opportunity and hypothesis, product scope, user experience and impact, and metric and data rigor, producing a launch-readiness scorecard with prioritized gaps, evidence, replacement-text suggestions, and a “start here” fix. Early internal use by dozens of PMs helped surface blind spots, test unsupported headroom assumptions, identify adjacent-system effects, and improve experiences within existing scope. The team’s lessons emphasize decision frameworks, broader context, hard boundaries, prioritization, and preserving human judgment for final approval.
Operating room utilization is hiding in your scheduling data
Operating room utilization measures in-room surgical minutes against allocated block-time minutes, yet most US health systems reportedly run at 65–75% versus an 80% industry target. Daily performance reports arrive the next morning, after schedules are set and opportunities to release unused blocks, redeploy staff, or backfill add-on cases may have passed. The post presents Databricks Genie as a natural-language interface for querying scheduling, utilization, and outcomes data without a data analyst request. Its proposed analytical environment combines scheduling data, actual case logs, block-release records, contribution margin, and staffing costs, with breakdowns by surgeon, service line, facility, and day of week. Genie surfaces specific intervention targets, although the post says it provides data access rather than automating OR management.
When predicting the next hit requires more than intuition
Content investment decisions in entertainment commit substantial funds despite limited information, because leaders often combine executive instinct, competitive benchmarks, and historical performance data. Although companies hold viewing and subscriber signals such as episode completion, skip patterns, genre performance by demographic, acquisition, and retention, those insights are frequently buried in dashboards and spreadsheets. The post presents Databricks Genie as a natural-language interface that lets content leaders query governed performance data directly, including comparisons linking content types with 90-day retention, subscriber demographics, lifetime value, acquisition source, and historical greenlights. It argues that current, conversational access can complement creative judgment by reducing dependence on delayed analyst recommendations, while external market data can add competitive context.
How conversational analytics removes the BI bottleneck
The interview presents conversational analytics as a way to move beyond fixed dashboards and turn governed data into decisions and workflows. Databricks’ Genie lets nontechnical users ask open-ended questions in plain language across Lakebase, Lakehouse, and federated sources, while Unity Catalog applies shared definitions and access controls. Lakebase provides transactional storage, instant provisioning, environment forking without copying underlying data, and scale described as billions or trillions of records; examples include real-time matchmaking, routing, and purchase decisions. The discussion stresses that human oversight, refreshed data, business context, and controls against hallucination remain necessary, citing Fox Sports’ chatbot deployment as an example. It concludes that organizations combining trusted semantics with operational data systems can make faster decisions and automate work, while the competitive gap may compound for slower adopters.
Trust Calibration for AI Software Builders
Trust calibration means aligning users’ trust in an AI product with its actual capabilities, avoiding both dangerous over-reliance and unnecessary rejection. Drawing on a 2023 review of more than 1,000 papers on trust in human/automated systems, the article distinguishes cooperative tools, where users accept or reject suggestions, from delegative systems, which replace human action and need stronger boundaries, fallbacks, and capability communication. It recommends calibrating expectations before and during interaction through capability-focused onboarding, context-sensitive feedback, visible suggestion boundaries, confirmation friction for high-stakes actions, and layered performance and process information; post-interaction reflection can still support learning. The article concludes that adaptive calibration is more effective than static approaches, anthropomorphic design can create unwarranted trust, explanations can overload users, and builders need measurements beyond satisfaction or adoption.
Make Better Decisions by Embracing Uncertainty
Ramp uses multi-armed bandits to choose among third-party bank-linking providers when provider performance varies by financial institution and available data is noisy, incomplete, or stale. Instead of maintaining separate, static A/B tests, it applies Thompson Sampling, modeling each provider’s success probability with a Beta distribution and repeatedly sampling, routing traffic to the highest sampled value, then updating the model from the result. Simulations examined changing success-rate trends, noise, volume, rolling-window size, trend type, and initial conditions; the algorithm generally exploited clearly better providers, split traffic more evenly when rates were close, and recovered eventually from incorrect starting conditions. In a sequential quasi-experiment, Thompson Sampling increased financial-institution linking success by 10% and reduced customers with manually uploaded bank statements by 25%.
Make Great Decisions Quickly with TOMASP
Technical leaders and managers face high-impact decisions about hiring, technology, architecture, and priorities, but cognitive biases, short-term emotion, and overconfidence can distort judgment. TOMASP is a framework for making decisions timely, considering alternatives, defining decision criteria, examining uncertainty, and preparing to be wrong. It recommends timeboxing the decision, generating at least three options, making a meta-decision about optimization criteria, analyzing options, stepping back from emotion, and preparing to be wrong through prototypes and tripwires. In the Flutter rewrite example, Michelle compares alternatives against cross-platform consistency, performance, and development speed, then sets a two-week prototype and blind test. The framework treats decision quality as a function of how a choice is made and implemented, not simply its eventual outcomes.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.