New York City marked the end of 2021 by enacting a law that will make it challenging, if not infeasible, to use a broad swath of algorithmic, computerized tools to review, select, rank or eliminate candidates for employment or promotion. Local Law Int. No. 1894-A, which takes effect on January 1, 2023, regulates the use of “automated employment decision tools” in hiring and promotion decisions within the city. The law, which applies to employers and employment agencies alike, requires that:

• any such tool undergo an annual, independent “bias audit,” with a publicly available summary;
• employers provide each candidate (internal or external) with 10 business days’ notice prior to being subject to the tool;
• the notice list the “job qualifications and characteristics” used by the tool to make its assessment;
• the sources and types of data used by the tool, as well as the applicable data-retention policy, be made available publicly (or upon written request from the candidate); and
• candidates be able to opt out and request an alternative selection process or accommodation.

The law establishes civil penalties of $500 to $1,500 for each violation of any of these requirements. It empowers New York City’s corporation counsel (or its designee) to enforce the provisions of this law by allowing it to file suit in any court of competent jurisdiction. In addition, claims can be filed with the Office of Administrative Trials and Hearings (and other authorized agencies) to seek the recovery of these civil penalties. In addition, the law provides applicants and employees a private right of action to enforce its provisions.

What is an automated employment decision tool?

The law applies only to employers that use “automated employment decision tools” in screening candidates for hiring or promotion. The scope of this term is broad and encompasses any process that uses “machine learning, statistical modeling, data analytics, or artificial intelligence” to generate a “score, classification or recommendation” that is used to at least substantially assist (if not replace) human discretion in screening candidates for employment or promotion within New York City. This likely covers any computerized tool or algorithm-based software program used to identify, select, evaluate, or recruit candidates for any employment position. It may thus include any data-driven tools used to review résumés, conduct skills testing, rank applicants, “chat” with applicants, conduct behavioral analysis of prospective hires, assess employee performance and productivity, monitor field-based or remote employees, or determine compensation and promotions. Tools that are used to identify employees for termination or reductions in force, however, are not explicitly within the defined scope.

What is a “bias audit”?

In order to make use of an “automated employment decision tool” employers must retain an “independent auditor” to assess whether the tool’s selection criteria result in disparate impact based on race, ethnicity or sex. The law provides no details on what this “bias audit” is supposed to examine, how it is expected to account for all potential jobs and job classes for which an organization might hire in the upcoming year, how its findings are to be utilized, or whether the tool must “pass” such an audit (and if so, what the passing criteria are). The only requirement is that such an audit be conducted annually by a third party and a summary of its results be published on the employer’s or employment agency’s website along with the “distribution date” of the tool. It is also unclear whether employers that use third parties to screen candidates will be required to post those contractors’ “bias audits” on the employer’s own website.

Who must receive notice?

The law requires that each resident of New York City “who has applied” for a position receive notice if their application will be subject to an automated tool as part of the decision-making process. It is unclear whether this excludes passively identified candidates who have taken no volitional step to be considered for a specific position; if not, this would be especially thorny for organizations that engage in identifying potential candidates from publicly available information (or based on an individual’s prior application for a different position).

It appears that non-residents of New York City are not required to receive this notice even when applying to a city-based position. Organizations will therefore need to ensure that they are capturing adequate information on the residence of each applicant, while ensuring that their attempt to seek this information is not itself perceived as leading to discriminatory outcomes.

When must notice issue?

Local Law Int. 1894-A requires that notice be issued on an individual basis at least 10 business days before the use of a tool. It is unclear whether this means notice must be issued before each use of an automated tool with respect to an individual’s application, whether the use of multiple automated tools requires a separate notice process with respect to each tool, or what counts as effective service of notice (including whether an acknowledgement of receipt is necessary to set the 10-day clock ticking).

What notice must be provided?

In addition to informing recipients that their application will be subject to an automated tool, the notice must identify the specific job qualifications and characteristics that the tool will use in making its assessment. This component is troublesome because the sophisticated computerized programs and other tools in this space, particularly those using machine learning and other forms of artificial intelligence, are by design constantly evolving, making it impractical to know what criteria the tool is using at any given point in time to assess a candidate. Given the complexity of this continuous “learning” process, these programs typically do not retain a snapshot or other record of each step in its evolution. Nor can their programmers fill the gap, because these tools are designed to evolve in a self-determined manner without the need for human intervention.

The notice must also inform candidates of their right to request an alternative selection process or accommodation. The law is silent, however, on key aspects of this requirement: whether each opt-out request must be granted or whether organizations have any discretion in the matter; what suffices as an alternative selection process or accommodation; who selects the alternative process or accommodation to be used; when or how quickly the alternative process is provided; what consideration may be given to business or job-related necessity for the use of specific computerized tools such as testing tools; and how disputes with respect to the opt-out process will be resolved. Note also that this requirement will likely slow down the hiring process, as each opt-out request is addressed and completed, and the impact on operational efficiency, often the chief driver of the adoption and use of these computerized tools, could be significant.

Finally, the law requires that employers (and employment agencies) retain “information about the type of data collected” for the tool, the source of that data, and the organization’s “data retention policy.” The law requires employers to either disclose this information on the organization’s website or provide the information to applicants within 30 days of a written request. (Employers do not need to disclose this information if disclosure would violate a local, state, or federal law or interfere with a law enforcement investigation.)

What are the penalties for non-compliance?

Any “person” who violates this law is “liable for a civil penalty” of “not more than $500 for a first violation and each additional violation occurring on the first day of the first violation.” Each subsequent violation occurring after the first day will result in a civil penalty of “at least $500 and not more than $1,500.” Each failure to meet applicant notice requirements constitutes a separate violation. Failure to meet the “bias audit” requirements results in separate, daily violations. While it is unclear from the text of the law, we expect that “person” is strictly limited to organizational entities as opposed to natural persons.

Who has standing to enforce this law?

As noted above, the city’s corporation counsel has the power to enforce the law by filing a claim in court. Moreover, since the provision amends the New York City Human Rights Law, it appears that the New York City Division of Human Rights (even though it is not mentioned in the body of the law) has the power to file claims before the city’s Office of Administrative Trials and Hearings to seek the recovery of these civil penalties. In addition, applicants and employees can bring suit themselves in any court of competent jurisdiction.

Next Steps

It is likely that the New York City Division of Human Rights will promulgate regulations to provide clarification on employers’ obligations under this law. It is not clear when those regulations will be published, however, nor is it clear how long covered employers and employment agencies will have to provide comments to those regulations. In the meanwhile, organizations may consider developing a taxonomy of AI-driven tools used for hiring and promotion decisions, and working with their vendors and technology stakeholders to develop means for independent audits that are sufficiently tied to the jobs and job classes for which the organization anticipates hiring. Talent acquisition functions may seek to prioritize development of a notice mechanism and opt-out protocol and prepare for the drag that this additional process layer will create on the hiring flow. To avoid complications relating to the residency requirement, organizations should consider issuing notice to all applicants for New York City openings and promotions. Finally, organizations may want to update their data-retention policies to account for the additional retention requirements and transparency expectation built into Local Law Int. 1894-A.