AI Model Deletion Orders Need Data Lineage

Resumen

Pixel’s announcement examines FTC orders requiring companies to delete AI models and other artifacts built from covered data. It highlights five enforcement cases from 2019 to 2023 and explains why tracking source records through datasets, checkpoints, and model weights matters for compliance.

DEV Community

Model Deletion Orders Require Training-Data Lineage

In May 2021, the FTC finalized its order against Everalbum, the company behind the Ever photo app. Everalbum had to delete the facial recognition models it had built using its users' photos. The order carried no fine. The main remedy was deleting the models. Orders like this assume a company can find every model that touched the data in question. A team can usually point to the code and configuration that produced a model. Pointing to the exact records that went into it is harder, and that is what these orders require. FTC orders that required model deletion The FTC has used this remedy at least five times since 2019: Cambridge Analytica (December 2019). Ordered to delete the data collected through the GSRApp and any work product built from it, including algorithms. Everalbum (May 2021). Ordered to delete the facial recognition models and algorithms built using biometric data from Ever users. WW International and Kurbo (March 2022). Ordered to delete data collected from children under 13 without parental consent, destroy any algorithms built from it, and pay $1.5 million. Amazon Alexa (May 2023). Fined $25 million and barred from using voice and geolocation data that users had asked to delete to build or improve its data products. Rite Aid (December 2023). Banned from using facial recognition for surveillance for five years, and ordered to delete the images it collected and any models or algorithms built from them. The Cambridge Analytica and Everalbum orders cover anything built from the data "in whole or in part." Read literally, a model trained on a dataset with a single covered record falls under the order. What counts as a derived artifact Training produces more than a final model file. Starting from one source table, a pipeline can create: dataset snapshots and feature tables, which copy or transform source rows model weights and training checkpoints, which are fitted to those rows fine-tuned models, which start from the weights of an earlier model embeddings and vector indexes, which store values the model computed from source records distilled student models, which are trained on the outputs of a teacher model Each item is a node in a lineage graph, and each one connects to the source through a different kind of edge. Lawyers decide which of these an order covers. To make that call, they need engineering to produce the full list for a given source. The minimum lineage record That list has to come from records written at training time. Lineage rebuilt later from logs and notebooks misses runs, especially old ones. At minimum, each training run should emit: Sources. The tables and partitions it read, the table version at read time, and the filter predicate, including any consent condition. With a pinned version and a deterministic predicate, the exact row set can be recomputed later without storing row IDs. Dataset snapshot ID. An immutable version of the training set. Apache Iceberg, for example, assigns a snapshot ID to every commit, and VERSION AS OF in Spark SQL returns the table as it was at that commit. Run metadata. Run ID, code commit, and hyperparameters, plus the base model ID for a fine-tune or the teacher model ID for distillation. Outputs. A content hash, such as SHA-256, of every model, checkpoint, and embedding table the run produced. The hash ties the record to the exact bytes, not to a file path that can be overwritten. Deployments. Every service and service version that loads those outputs. Querying the lineage graph Each record adds edges to a graph. Nodes are source tables, snapshots, training runs, artifacts, and deployments, and each edge points from an input to an output. A deletion request then becomes a traversal: start at the covered source and collect every node reachable from it. Figure 1 shows the result for one source. Figure 1. Model M2 and Deployment D2 stay out of scope because Snapshot S2 never read Source A. Without the snapshot-level edges, there is no way to show tha...
AI Model Deletion Orders Need Data Lineage
Powered by Readybot.io

Lo último de Pixel

Incident Response: Look Beyond the Red Alert

Pixel’s DEV Community announcement explores how teams can fixate on one failing service while overlooking the real cause of an outage, such as a full …

Sach: AI Fact-Checks WhatsApp Forwards in India

Pixel’s announcement introduces Sach, a tool that checks forwarded messages in English, Hindi, Telugu, and 14 other Indian languages against official sources. It gives cited …

Java 25 Removes SecurityManager: What to Fix

Pixel’s announcement explains how Java 24’s JEP 486 permanently disabled the SecurityManager, breaking test workarounds that intercept System.exit. It recommends returning exit codes or throwing …