Refers to instances when an individual is incorrectly classified in a dataset, despite the existence of an accurate (representative) data category.

For example, an algorithmic system that uses racial proxy analysis by assigning a racial category to an individual based on the analysis of an individual’s skin tone in an image may classify someone as “White” due to the perceived “light” skin tone of an individual who racially identifies as “Asian.”

Refers to a system composed of one or more algorithms, or an automated procedure used to perform a computation, and includes systems using machine learning or following a pre-programmed set of rules.

For example, search engines, traffic signals, and facial recognition software all rely on algorithmic systems to function.

Refers to the closeness (accuracy) between the representation of a value or data point and the true value or data point.

For example, high analytical accuracy is achieved when the distribution of gender categories (% of a population in each gender category) in a dataset matches the gender distribution of the measured population.

Refers to an incident where sensitive data or confidential information is stolen or otherwise accessed without the authorization of the system’s owner.

For example, the physical theft of a hard drive containing users’ personal information or a ransomware cyberattack that prevents a company from accessing its own customer data unless a ransom is paid are both considered data breaches.

Refers to the conversion of various aspects of human life into quantitative data which then allows for quantitative analysis of social and individual behaviors.

For example, peoples’ communication, images, and speech are all converted into data points via various technological platforms by messaging platforms (e.g., text from conversations), social media sites (e.g., text- and image-based posts), and digital assistants (e.g., audio recordings of voice commands given to the device), respectively.

Refers to instances when the demographic categories applied in a dataset do not adequately or accurately represent the identity of the individual being counted.

For example, if a survey only provides the options to select “man” or “woman,” an individual who identifies as gender non-binary (as neither a woman or a man) will be represented inaccurately in the dataset.

Refers to situations where the identity of a person or organization is discoverable even though the individual or organization’s name is not available or purposely removed, typically by matching the anonymized dataset with publicly available data or auxiliary data.

For example, an anonymized dataset containing private health information can be re-identified if the identification number used to distinguish individuals from one another is their social security number and a separate list with the individuals’ names and social security number is used.

Refers to a set of techniques used by data analysts to fill in, as accurately as possible, missing or unidentified demographic traits by analyzing other available data.

For example, an individual’s name can be used to guess their gender identity by analyzing how often that name is associated with individuals who identify as a woman versus how often it is associated with individuals who identify as a man.

Refers to the often-used legal interpretation of “fairness” where the emphasis is on determining whether one group experiences different outcomes or treatment (unfair), including when differences emerge unintentionally.

Adverse impact specifically refers to instances of disparate impact when a group disproportionately (with greater frequency or intensity) experiences a negative outcome or treatment.

The “80% rule” refers to one specific, and simple, way to “test” for instances of adverse impact, originally designed by the State of California Fair Employment Practice Commission, by calculating whether the selection rate for a minority group (the group with the lowest selection rate) is less than 80% of the rate for the group with the highest selection rate (typically the majority group).

Refers to an approach of user data analysis that does not transfer data from individual devices to a central server for analysis, but instead runs local computation on the individual devices and sends the results to a central server where composite results are made available. Federated learning most often refers to an approach used to train machine learning models on individual devices, while federated statistics most often refers to basic statistical analysis conducted on individual devices.

Refers to a form of consent (permission for something to happen) where individuals fully understand the scope and implications of their participation (including disclosure of their information) and have the ability to refuse or withdraw their participation at any time.

For example, a person is able to provide knowledgeable consent for the use of their blood sample in a scientific experiment because the individual understands the various instances when their sample may be used — and what may happen due to the use of their sample — and is given frequent opportunities throughout the entire duration (when their sample is in possession of the scientists conducting the experiment) to withdraw their sample from use.

Refers to a vulnerability of federated learning (due its distributed nature) where the model (or data analysis) being trained through federated learning is attacked through the insertion of “bad” (inaccurate or irrelevant) data into an AI model’s training dataset, leading the algorithmic model to complete unintended learning and creating undesirable variability in the model’s outputs.

Refers to a cryptographic protocol (a system of rules related to how should be structured and represented and how algorithms should be used specifically to prevent third parties from accessing or revealing information) that securely computes the aggregation (analysis or compilation) of its inputs in such a way that the inputs are kept hidden and the central server where the aggregation takes place or is stored cannot learn the value of the individual inputs.

Refers to a technique designed to cryptographically protect information by allowing multiple, distinct parties — each holding their own private data — to conduct a collaborative computation (combine and analyze information together) without revealing the specific details or value of that data to the other parties, thereby protecting the privacy of the data while not endangering the security of the data.

Refers to an approach in which social structures and technical systems are understood to co-inform one another. Assessing just technical components of a system obscures the human components that are embedded within them, thereby misrepresenting the consequences and impacts of the system.

For example, a sociotechnical analysis of an algorithmic system would include assessment of the various social components influencing the design, production, and deployment of the system as well as the social impacts of the system.

Refers to a group within a society that is smaller in size (fewer number of people) than another group. In the case of demographic groups, this may overlap with social minorities, which are defined as groups that experience systematic discrimination, prejudice, and harm on the basis of a demographic trait.

For example, people who identify as transgender (someone whose gender identity differs from the one that is typically associated with the sex assigned at birth) is a social minority in the US due to the discrimination and harm experienced and a statistical minority due to the relatively smaller size of the population compared to people who identify as cisgender (someone whose gender identity corresponds with the sex assigned at birth). On the other hand, women are considered a social minority due to the systematic discrimination the group experiences but are not currently a statistical minority.

Refers to a commonly used definition of fairness in machine learning related to the legal doctrine of “disparate impact” where a model is considered to be operating fairly if each group is expected to have the same probability of experiencing the positive, favorable outcome.

For example, statistical parity in a machine learning model used to recommend promotions at a workplace would require that men and women in the dataset have the same likelihood of receiving a recommendation for promotion.

Refers to data-driven tools embedded in the built and/or digital environment that allow for the monitoring of individual behaviors and actions.

Surveillance infrastructure can include tools like traffic light cameras or wearable devices. Due to systematic inequalities like racism and xenophobia which consider specific groups of people as more dangerous or socially deviant (and therefore require constant monitoring), marginalized communities are often disproportionately subject to surveillance infrastructure.