April 27, 2023
Stroock Client Alert
By: Shauneida C. Navarrete, Joshua Sohn
On April 6, 2023, the New York City Department of Consumer and Worker Protection (“DCWP”) adopted final rules in connection with a novel law intended to regulate the use of artificial intelligence in hiring and promotions.[1] This law requires employers or employment agencies to conduct bias audits to assess the artificial intelligence (“AI”) tool’s potential for infecting the hiring and selection process with unlawful bias that may result in decisions that have an unlawful disparate impact on certain groups of applicants or employees. Because the AI being used in this process is called an automated employment decision tool or “AEDT," the law is referred to as the “AEDT Law.” The DCWP will begin enforcement of the AEDT Law on July 5, 2023.[2] For more information about the basics of the AEDT Law, please review our previous alert here.[3]
The final rules followed a protracted debate on regulating AEDT in hiring. The New York City Council originally enacted the legislation in November 2021 with an intended effective date of January 1, 2023.[4] After enactment, however, the DCWP received numerous comments from employers, employment agencies, law firms, AEDT developers, and advocacy organizations seeking clarity and revisions to the law. In response to rounds of public comments, the DCWP clarified what constituted an AEDT, a bias audit, and an independent auditor.
1. What is an AEDT?
Generally speaking, AEDT refers to any automated screening or selection processes that were previously done by humans. To eliminate confusion, and add more specificity to the definition, the DCWP explained that an AEDT is any computational process, “derived from machine learning, statistical modeling, data analytics, or artificial intelligence,”[5] which provides a score, classification, or recommendation to the employer that is used to significantly assist or substitute a human’s decision-making process.[6] The DCWP further stated that to substantially assist or replace a natural person’s decision making process means: 1) to rely solely on a simplified output without consideration to other factors; 2) to use a simplified output as a consideration in a list of criteria, but weigh the output more heavily that the other criterion in the set; or 3) to use the output to overrule human decision making conclusions.[7]
2. What is required for a bias audit under the AEDT Law?
a. The Auditor Must be Independent
Through its final rules the DCWP clarified that bias audits must be conducted by truly independent auditors. Under the originally drafted law, there was room for interpretation as to who can properly serve as an independent auditor. For example, it was unclear whether a company that developed an AEDT could have a separate business unit conduct an “independent” audit of that tool. In the final rules, the DCWP clarified that the bias audit must be an impartial evaluation by an independent auditor. And it defined an independent auditor as a person or group that is capable of exercising “objective and impartial judgment on all issues within the scope of a bias audit.”[8] To maintain impartiality, the auditor is not permitted to be involved in using, developing, or distributing the AEDT; have an employment relationship with the employer seeking to use the AEDT during the period they are conducting the bias audit; or have a financial interest in the AEDT’s use during the period for which they are conducting the audit.[9]
b. The Audit must calculate how the AEDT impacts decisions related to people in each EEO protected category
Besides providing guidance on the auditor, the DCWP specified that when the AEDT selects candidates for employment or promotion or classifies candidates into groups, the bias audit must, at minimum:
Notably, the AEDT Law does not address whether a sex categories impact ratio should include non-binary individuals, and if they should be included, how to properly calculate the impact ratio for these individuals. For years New York City has made it unlawful to discriminate against an individual based on a person’s gender identity, self-image, appearance, behavior, or expression. Therefore, it would appear that the impact ratio should incorporate non-binary individuals. It is unclear whether the DCWP will provide clarity on this open issue.
3. What notice is required to candidates and employees?
As previously reported, candidates for a job posting or employees seeking a promotion have to receive notification from the employer that an AEDT will be used as a hiring tool. The final rules provide the logistics for providing notice, including updating the employer’s website at least ten business days before the use of an AEDT, noting the use of an AEDT in the job posting, or informing candidates via U.S. mail or email that an AEDT will be used.[11]
The final regulation also makes clear that to use the AEDT in hiring or promotion, the employer or employment agency must conduct a bias audit within a year of the tool being used.[12] For instance, if an employer wants to use its AEDT to screen resumes and conduct interviews for a vacant position, the employer must ensure that a bias audit was conducted no more than a year before the employer’s planned use of the AEDT. It is inconsequential that the AEDT is being used only to initially screen resumes and not to make a final hiring decision—the regulation still requires the employer to conduct an audit bias. Additionally, before an employer can use an AEDT, the employer must publish on its employment section of its website the date of the most recent bias audit as well as the results of the audit.[13]
* * *
So, with such impactful regulation on the horizon, it is important that employers start examining and adjusting their internal and external HR processes and practices. Below are some high-level takeaways that employers should address now before the law takes effect.
The AEDT Law signals a change in New York City that other states are likely to replicate. With an overwhelming focus on artificial intelligence, it is imperative that employers and human resources personnel stay abreast of these changes.
[1] See New York City Department of Consumer and Worker Protection, Notice of Adoption of Final Rule, available at https://rules.cityofnewyork.us/wp-content/uploads/2023/04/DCWP-NOA-for-Use-of-Automated-Employment-Decisionmaking-Tools-2.pdf.
[2] See New York City Department of Consumer and Worker Protection, New Laws & Rules, available at: https://www.nyc.gov/site/dca/about/new-laws-rules.page#:~:text=December%202022%20Update,page%20for%20updates.
[3] https://www.stroock.com/news-and-insights/are-you-ready-for-nycs-anti-bias-ai-law
[4] See New York City Department of Consumer and Worker Protection, New Laws & Rules, available at: https://www.nyc.gov/site/dca/about/new-laws-rules.page#:~:text=December%202022%20Update,page%20for%20updates.
[5] Additionally, the final law clarifies that machine learning, statistical modeling, data analytics, or artificial intelligence means a group of mathematical, computer-based processes that generate an expected outcome or prediction or that generate an assignment of an observation to a group (i.e., classification), “such as categorizations based on skill sets or aptitude,” and for which the computer at the bare minimum “identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models” to improve the accuracy of the classification.
[6] See New York City Department of Consumer and Worker Protection, Update to Chapter 5 of Title 6 of the Rules of the City of New York, Subchapter T, § 5-300, available at https://rules.cityofnewyork.us/wp-content/uploads/2023/04/DCWP-NOA-for-Use-of-Automated-Employment-Decisionmaking-Tools-2.pdf (“Updated AEDT Law”).
[7] Updated AEDT Law at § 5-300.
[8] Updated AEDT Law at § 5-300.
[9] Updated AEDT Law at § 5-300.
[10] Updated AEDT Law at § 5-301.
[11] Updated AEDT Law at § 5-304(b).
[12] Updated AEDT Law at § 5-301(a).
[13] Updated AEDT Law at § 5-303(a).
April 27, 2023
Stroock Client Alert
By: Shauneida C. Navarrete, Joshua Sohn
On April 6, 2023, the New York City Department of Consumer and Worker Protection (“DCWP”) adopted final rules in connection with a novel law intended to regulate the use of artificial intelligence in hiring and promotions.[1] This law requires employers or employment agencies to conduct bias audits to assess the artificial intelligence (“AI”) tool’s potential for infecting the hiring and selection process with unlawful bias that may result in decisions that have an unlawful disparate impact on certain groups of applicants or employees. Because the AI being used in this process is called an automated employment decision tool or “AEDT," the law is referred to as the “AEDT Law.” The DCWP will begin enforcement of the AEDT Law on July 5, 2023.[2] For more information about the basics of the AEDT Law, please review our previous alert here.[3]
The final rules followed a protracted debate on regulating AEDT in hiring. The New York City Council originally enacted the legislation in November 2021 with an intended effective date of January 1, 2023.[4] After enactment, however, the DCWP received numerous comments from employers, employment agencies, law firms, AEDT developers, and advocacy organizations seeking clarity and revisions to the law. In response to rounds of public comments, the DCWP clarified what constituted an AEDT, a bias audit, and an independent auditor.
1. What is an AEDT?
Generally speaking, AEDT refers to any automated screening or selection processes that were previously done by humans. To eliminate confusion, and add more specificity to the definition, the DCWP explained that an AEDT is any computational process, “derived from machine learning, statistical modeling, data analytics, or artificial intelligence,”[5] which provides a score, classification, or recommendation to the employer that is used to significantly assist or substitute a human’s decision-making process.[6] The DCWP further stated that to substantially assist or replace a natural person’s decision making process means: 1) to rely solely on a simplified output without consideration to other factors; 2) to use a simplified output as a consideration in a list of criteria, but weigh the output more heavily that the other criterion in the set; or 3) to use the output to overrule human decision making conclusions.[7]
2. What is required for a bias audit under the AEDT Law?
a. The Auditor Must be Independent
Through its final rules the DCWP clarified that bias audits must be conducted by truly independent auditors. Under the originally drafted law, there was room for interpretation as to who can properly serve as an independent auditor. For example, it was unclear whether a company that developed an AEDT could have a separate business unit conduct an “independent” audit of that tool. In the final rules, the DCWP clarified that the bias audit must be an impartial evaluation by an independent auditor. And it defined an independent auditor as a person or group that is capable of exercising “objective and impartial judgment on all issues within the scope of a bias audit.”[8] To maintain impartiality, the auditor is not permitted to be involved in using, developing, or distributing the AEDT; have an employment relationship with the employer seeking to use the AEDT during the period they are conducting the bias audit; or have a financial interest in the AEDT’s use during the period for which they are conducting the audit.[9]
b. The Audit must calculate how the AEDT impacts decisions related to people in each EEO protected category
Besides providing guidance on the auditor, the DCWP specified that when the AEDT selects candidates for employment or promotion or classifies candidates into groups, the bias audit must, at minimum:
Notably, the AEDT Law does not address whether a sex categories impact ratio should include non-binary individuals, and if they should be included, how to properly calculate the impact ratio for these individuals. For years New York City has made it unlawful to discriminate against an individual based on a person’s gender identity, self-image, appearance, behavior, or expression. Therefore, it would appear that the impact ratio should incorporate non-binary individuals. It is unclear whether the DCWP will provide clarity on this open issue.
3. What notice is required to candidates and employees?
As previously reported, candidates for a job posting or employees seeking a promotion have to receive notification from the employer that an AEDT will be used as a hiring tool. The final rules provide the logistics for providing notice, including updating the employer’s website at least ten business days before the use of an AEDT, noting the use of an AEDT in the job posting, or informing candidates via U.S. mail or email that an AEDT will be used.[11]
The final regulation also makes clear that to use the AEDT in hiring or promotion, the employer or employment agency must conduct a bias audit within a year of the tool being used.[12] For instance, if an employer wants to use its AEDT to screen resumes and conduct interviews for a vacant position, the employer must ensure that a bias audit was conducted no more than a year before the employer’s planned use of the AEDT. It is inconsequential that the AEDT is being used only to initially screen resumes and not to make a final hiring decision—the regulation still requires the employer to conduct an audit bias. Additionally, before an employer can use an AEDT, the employer must publish on its employment section of its website the date of the most recent bias audit as well as the results of the audit.[13]
* * *
So, with such impactful regulation on the horizon, it is important that employers start examining and adjusting their internal and external HR processes and practices. Below are some high-level takeaways that employers should address now before the law takes effect.
The AEDT Law signals a change in New York City that other states are likely to replicate. With an overwhelming focus on artificial intelligence, it is imperative that employers and human resources personnel stay abreast of these changes.
[1] See New York City Department of Consumer and Worker Protection, Notice of Adoption of Final Rule, available at https://rules.cityofnewyork.us/wp-content/uploads/2023/04/DCWP-NOA-for-Use-of-Automated-Employment-Decisionmaking-Tools-2.pdf.
[2] See New York City Department of Consumer and Worker Protection, New Laws & Rules, available at: https://www.nyc.gov/site/dca/about/new-laws-rules.page#:~:text=December%202022%20Update,page%20for%20updates.
[3] https://www.stroock.com/news-and-insights/are-you-ready-for-nycs-anti-bias-ai-law
[4] See New York City Department of Consumer and Worker Protection, New Laws & Rules, available at: https://www.nyc.gov/site/dca/about/new-laws-rules.page#:~:text=December%202022%20Update,page%20for%20updates.
[5] Additionally, the final law clarifies that machine learning, statistical modeling, data analytics, or artificial intelligence means a group of mathematical, computer-based processes that generate an expected outcome or prediction or that generate an assignment of an observation to a group (i.e., classification), “such as categorizations based on skill sets or aptitude,” and for which the computer at the bare minimum “identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models” to improve the accuracy of the classification.
[6] See New York City Department of Consumer and Worker Protection, Update to Chapter 5 of Title 6 of the Rules of the City of New York, Subchapter T, § 5-300, available at https://rules.cityofnewyork.us/wp-content/uploads/2023/04/DCWP-NOA-for-Use-of-Automated-Employment-Decisionmaking-Tools-2.pdf (“Updated AEDT Law”).
[7] Updated AEDT Law at § 5-300.
[8] Updated AEDT Law at § 5-300.
[9] Updated AEDT Law at § 5-300.
[10] Updated AEDT Law at § 5-301.
[11] Updated AEDT Law at § 5-304(b).
[12] Updated AEDT Law at § 5-301(a).
[13] Updated AEDT Law at § 5-303(a).
