AI privacy budgets: Ask for the calculation, not the claim
Your vendor claims epsilon 3.0 privacy. Ask them to prove it—most can't connect the number to the training run that produced it.
Why it matters
Enterprise AI buyers are accepting privacy budget numbers without verification. This deep dive explains what questions to ask vendors about federated learning privacy claims, and why the gap between contractual promises and operational evidence is where real exposure lives.
The key facts
13 to knowPrivacy budgets require specification of: privacy unit, accounting method, noise/clipping parameters, round cap, and cumulative loss—not just a number
Same epsilon value (e.g., 3.0) can be achieved with different parameter combinations that do NOT describe the same training run
Composition example: noise multiplier of 1.1 = epsilon ~7.5; 1.5 = epsilon ~4.4; 2.0 = epsilon ~2.9 over 100 rounds with 25 of 250 tenants per round
Proposed artifact: 'privacy release record' tying privacy unit, target budget, accountant method, sampling, noise/clipping, actual rounds completed, cumulative loss, and release decision to a single model version hash
Secure aggregation does not prevent malicious coordinator from selecting Sybil participants to reconstruct federated learning (established attack class)
Federated unlearning remains research area; deletion requests do not automatically remove a tenant's influence on already-released models
Accountant should gate release pipeline (control plane), not just report to dashboard after training completes
CIO contributor: identity governance not designed for AI agents; same gap exists one layer below in participant-set governance
Privacy budgets require five components to be meaningful: privacy unit (record/user/tenant), accounting method, noise/clipping parameters, round count, and release decision—not a single epsilon value
Example: same noise multiplier change (1.1 to 2.0) shifts epsilon from ~7.5 to ~2.9 in a 100-round tenant-level run; a stated budget of 3.0 cannot describe both scenarios
Secure aggregation does not prevent Sybil attacks or malicious coordinator selection of training cohorts—a known attack class not yet translated into procurement evidence
Federated unlearning is now research-active because deleting a tenant's data does not remove its influence on already-deployed models; legal and engineering positions must be locked before deletion requests arrive
Proposed artifact: a 'privacy release record' linking privacy unit, target epsilon/delta, accountant method, sampling model, noise/clipping, actual rounds completed, cumulative bounds, release decision, and model hash—generated from training state, not assembled later
Go to the source
CIOcio.com
Publisher excerpt: Writing a specification requires a precision that a contract does not. This year, I focused on an internet draft for privacy-preserving federated learning in multi-tenant agent systems. The privacy budget could not go in as a number. It had to be presented with the privacy unit, the accounting…
