Andrew Smith, Director of the Federal Trade Commission Bureau of Consumer Protection, recently offered businesses guidance for the use of artificial intelligence (AI) and related algorithms. At the outset, Smith noted that AI has “enormous potential” to improve welfare and productivity, while nevertheless observing that AI also presents risks, such as the potential for unfair or discriminatory outcomes or the perpetuation of existing socioeconomic disparities.
Smith pointed out that the FTC has extensive experience dealing with the challenges presented by the use of data and algorithms to make decisions about consumers. The FTC has investigated and brought many cases alleging legal violations involving AI and automated decision-making.
In 2016, the FTC issued a report titled Big Data: A Tool for Inclusion or Exclusion? which advised companies using big data analytics and machine learning to reduce the opportunity for bias. In November 2018, the FTC held a hearing to explore AI, algorithms, and predictive analytics.
The FTC’s law enforcement actions, studies, and guidance emphasize that the use of AI tools should be transparent, explainable, fair, and empirically sound, while fostering accountability.
Smith offers a number of suggestions to users of AI in an article written on the Federal Trade Commission website. Featured in part below, you can find it in its entirety here: https://www.ftc.gov/news-events/blogs/business-blog/2020/04/using-artificial-intelligence-algorithms.
Don’t deceive consumers about how you use automated tools. Oftentimes, AI operates in the background, somewhat removed from the consumer experience. But, when using AI tools to interact with customers (think chatbots), be careful not to mislead consumers about the nature of the interaction. The Ashley Madison complaint alleged that the adultery-oriented dating website deceived consumers by using fake “engager profiles” of attractive mates to induce potential customers to sign up for the dating service. And the Devumi complaint alleged that the company sold fake followers, phony subscribers, and bogus “likes” to companies and individuals that wanted to boost their social media presence. The upshot? If a company’s use of doppelgängers—whether a fake dating profile, phony follower, deepfakes, or an AI chatbot—misleads consumers, that company could face an FTC enforcement action.
Be transparent when collecting sensitive data. Generally, the bigger the data set, the better the algorithm, and the better the product for consumers. However, be careful about how you get that data set, says Smith. Secretly collecting audio or visual data—or any sensitive data—to feed an algorithm could also give rise to an FTC action. Last year, the FTC alleged that Facebook misled consumers when it told them they could opt into facial recognition—even though the setting was on by default. As the Facebook case shows, how a business gets the data may matter a great deal.
If you make automated decisions based on information from a third-party vendor, you may be required to provide the consumer with an “adverse action” notice. Under the Fair Credit Reporting Act, a vendor that assembles consumer information to automate decision-making about eligibility for credit, employment, insurance, housing, or similar benefits and transactions, may be a “consumer reporting agency.” That triggers duties for the user of that information. Specifically, the user must provide consumers with certain notices under the FCRA. Smith provides an example. Say you purchase a report or score from a background check company that uses AI tools to generate a score predicting whether a consumer will be a good tenant. The AI model uses a broad range of inputs about consumers, including public record information, criminal records, credit history, and maybe even data about social media usage, shopping history, or publicly available photos and videos. If you use the report or score as a basis to deny someone an apartment, or charge them higher rent, you must provide that consumer with an adverse action notice. The adverse action notice tells the consumer about the right to see the information reported about them and to correct inaccurate information.
EXPLAIN YOUR DECISION TO THE CONSUMER.
If you deny consumers something of value based on algorithmic decision-making, explain why. According to Smith, some might say that it’s too difficult to explain the multitude of factors that might affect algorithmic decision-making. But, in the credit-granting world, companies are required to disclose to the consumers the principal reasons why they were denied credit. Smith observes that it’s not good enough simply to say, “your score was too low” or “you don’t meet our criteria.” The business needs to be specific (e.g., “you’ve been delinquent on your credit obligations” or “you have an insufficient number of credit references”). This means that the business must know what data is used in its model and how that data is used to arrive at a decision. And the business must be able to explain that to the consumer. “If you are using AI to make decisions about consumers in any context, consider how you would explain your decision to your customer if asked.”
If you use algorithms to assign risk scores to consumers, also disclose the key factors that affected the score, rank ordered for importance. Similar to other algorithmic decision-making, scores are based on myriad factors, some of which may be difficult to explain to consumers. For example, if a credit score is used to deny someone credit, or offer them less favorable terms, the law requires that consumers be given notice, a description of the score (its source, the range of scores under that credit model), and at least four key factors that adversely affected the credit score, listed in the order of their importance based on their effect on the credit score.
If you might change the terms of a deal based on automated tools, make sure to tell consumers. Smith observed that more than a decade ago, the FTC alleged that subprime credit marketer CompuCredit violated the FTC Act by deceptively failing to disclose that it used a behavioral scoring model to reduce consumers’ credit limits. For example, if cardholders used their credit cards for cash advances or to make payments at certain venues, such as bars, nightclubs, and massage parlors, they might have their credit limit reduced. The company never told consumers that these purchases could reduce their credit limit – neither at the time they signed up nor at the time they reduced the credit limit. Says Smith—"that decade-old matter is just as important today. If you’re going to use an algorithm to change the terms of the deal, tell consumers.”
ENSURE THAT YOUR DECISIONS ARE FAIR.
Don’t discriminate based on protected classes. Cavalier use of AI could result in discrimination against a protected class. A number of federal equal opportunity laws, such as the Equal Credit Opportunity Act and Title VII of the Civil Rights Act of 1964, may be relevant to such conduct. The FTC enforces ECOA, which prohibits credit discrimination based on race, color, religion, national origin, sex, marital status, age, or because a person receives public assistance. If, for example, a company made credit decisions based on consumers’ Zip Codes, resulting in a “disparate impact” on particular ethnic groups, the FTC could challenge that practice under ECOA. Smith advises that a business can save itself a lot of problems by rigorously testing its algorithm, both before it is used and periodically afterwards, to make sure it doesn’t create a disparate impact on a protected class.
Focus on inputs, but also on outcomes. When the FTC evaluates an algorithm or other AI tool for illegal discrimination, it looks at the inputs to the model – such as whether the model includes ethnically-based factors, or proxies for such factors, such as census tract. But, regardless of the inputs, the FTC reviews the outcomes. For example, does a model, in fact, discriminate on a prohibited basis? Does a facially neutral model have an illegal disparate impact on protected classes? The agency’s economic analysis looks at outcomes, such as the price consumers pay for credit, to determine whether a model appears to have a disparate impact on people in a protected class. If it does, the FTC then reviews the company’s justification for using that model and considers whether a less discriminatory alternative would achieve the same results. Companies using AI and algorithmic tools should consider whether they should engage in self-testing of AI outcomes, to manage the consumer protection risks inherent in using such models.
Under the FCRA, consumers are entitled to obtain the information on file about them and dispute that information if they believe it to be inaccurate. Moreover, adverse action notices are required to be given to consumers when that information is used to make a decision adverse to the consumer’s interests. That notice must include the source of the information that was used to make the decision and must notify consumers of their access and dispute rights. If a business is using data obtained from others—or even obtained directly from the consumer—to make important decisions about the consumer, it should consider providing a copy of that information to the consumer and allowing the consumer to dispute the accuracy of that information.
ENSURE THAT YOUR DATA AND MODELS ARE ROBUST AND EMPIRICALLY SOUND.
If you provide data about consumers to others to make decisions about consumer access to credit, employment, insurance, housing, government benefits, check-cashing or similar transactions, you may be a consumer reporting agency that must comply with the FCRA, including ensuring that the data is accurate and up to date. If a business compiles and sells consumer information that is used or expected to be used for credit, employment, insurance, housing, or other similar decisions about consumers’ eligibility for certain benefits and transactions, the business may be subject to the FCRA. Among other things, the business has an obligation to implement reasonable procedures to ensure maximum possible accuracy of consumer reports and must provide consumers with access to their own information, along with the ability to correct any errors. Smith notes that RealPage, Inc., a company that deployed software tools to match housing applicants to criminal records in real time or near real time, “learned this the hard way.” The company ended up paying a $3 million penalty for violating the FCRA by failing to take reasonable steps to ensure the accuracy of the information it provided to landlords and property managers.
If you provide data about your customers to others for use in automated decision-making, you may have obligations to ensure that the data is accurate, even if you are not a consumer reporting agency. Companies that provide data about their customers to consumer reporting agencies are referred to as “furnishers” under the FCRA. They may not furnish data that they have reasonable cause to believe may not be accurate. In addition, they must have in place written policies and procedures to ensure that the data they furnish is accurate and has integrity. Furnishers also must investigate disputes from consumers, as well as disputes received from the consumer reporting agency. These requirements are important to ensure that the information used in AI models is as accurate and up to date as it can possibly be. The FTC has brought actions, and obtained big fines, against companies that furnished information to consumer reporting agencies but that failed to maintain the required written policies and procedures to ensure that the information that they report is accurate.
Make sure that your AI models are validated and revalidated to ensure that they work as intended, and do not illegally discriminate. The consumer lending laws encourage the use of AI tools that are “empirically derived, demonstrably and statistically sound.” This means, among other things, that they are based on data derived from an empirical comparison of sample groups, or the population of creditworthy and noncreditworthy applicants who applied for credit within a reasonable preceding period of time; that they are developed and validated using accepted statistical principles and methodology; and that they are periodically revalidated by the use of appropriate statistical principles and methodology, and adjusted as necessary to maintain predictive ability.
HOLD YOURSELF ACCOUNTABLE FOR COMPLIANCE, ETHICS, FAIRNESS, AND NONDISCRIMINATION.
Ask questions before you use the algorithm. In the 2016 Big Data report, the FTC warned companies that big data analytics could result in bias or other harm to consumers. To avoid that outcome, any operator of an algorithm should ask four key questions:
- How representative is your data set?
- Does your data model account for biases?
- How accurate are your predictions based on big data?
- Does your reliance on big data raise ethical or fairness concerns?
Protect your algorithm from unauthorized use. If one is in the business of developing AI to sell to other businesses, think about how these tools could be abused and whether access controls and other technologies can prevent the abuse. For instance, according to Smith, the FTC recently hosted a workshop on voice-cloning technologies. Thanks to machine learning, these technologies enable companies to use a five-second clip of a person’s actual voice to generate a realistic audio of the voice saying anything. This technology promises to help people who have lost the ability to speak, among other things, but could be easily abused if it falls into the hands of people engaged in imposter schemes. One company that is introducing this cloning technology is vetting users and running the technology on its own servers so that it can stop any abuse that it learns about.
Consider your accountability mechanism. Smith says consider how a business holds itself accountable, and whether it would make sense to use independent standards or independent expertise to step back and take stock of your AI. Smith mentions a healthcare algorithm that ended up discriminating against black patients. Well-intentioned employees were trying to use the algorithm to target medical interventions to the sickest patients. Outside, objective observers who independently tested the algorithm were the ones who discovered the problem. Such outside tools and services are increasingly available as AI is used more frequently, and companies may want to consider using them.