The short answer
A bias audit is an impartial evaluation of a hiring tool by an independent auditor that tests whether the tool's results differ across demographic groups. Under New York City's Local Law 144, it must at minimum calculate selection or scoring rates and impact ratios across sex, race/ethnicity and intersectional categories. Employers need one because they cannot use an automated employment decision tool in the city if more than a year has passed since its last bias audit.
The AI bias audit explained
The audit starts with data: candidates the tool assessed, their outcomes, and their sex and race/ethnicity. The auditor then works out, for each category, either the selection rate (for tools that advance or classify people) or the scoring rate (for tools that output a score: the share scoring above the sample's median). Each rate is divided by the top group's rate to get an impact ratio.
New York City is specific about who can run it. An auditor is not independent if they were involved in using, building or distributing the tool, or if during the audit they work for or have a financial interest in the employer or the vendor.
The data must normally be historical data from real use of the tool. One audit can pool several employers' data, but an employer can rely on it only if it contributed its own data or has never used the tool. Test data is allowed only when there isn't enough historical data, and the summary must explain why and how the test data was made.
What the published summary must include
The employer must post this on the employment section of its website before use and keep it up for at least six months after last using the tool.
What a bias audit can't tell you
An audit shows whether outcomes differed by group in the data reviewed. It doesn't show that the tool measures job skills well, and the law requires no specific action based on the results, though anti-discrimination laws still apply. Read the lowest ratio, the group sizes and the date before trusting a summary. Our full guide on how to read an AI bias audit goes deeper, and you can compare vendors in AI hiring tools with published audits. For a real example, NTRVSTA posts its own independent Local Law 144 audit, conducted by Asenion and finalized October 20, 2026, on our compliance page.
Related terms
- Automated employment decision tool: the kind of tool NYC requires to be audited.
- Impact ratio: the main number in every audit table.
- Selection rate: the share of a group a tool moves forward.
- Adverse impact: the disparity an audit is designed to surface.
- Four-fifths rule: the 0.80 benchmark readers often apply to impact ratios.
Questions
Who can conduct a bias audit under Local Law 144?
An independent auditor: a person or group able to judge objectively who was not involved in using, developing or distributing the tool, and who during the audit has no employment relationship with, and no direct or material indirect financial interest in, the employer or the vendor.
How often is a bias audit required in New York City?
An employer or employment agency may not use or keep using an automated employment decision tool if more than one year has passed since its most recent bias audit, so in practice at least once a year.
Can an employer rely on the vendor's bias audit?
Yes, in some cases. An audit can use historical data from several employers, but an employer may rely on it only if it provided its own data from using the tool or has never used the tool. The legal duty still sits with the employer or employment agency.
Want to walk through a published audit with the team that commissioned it? Book a 20-minute demo, or try a sample AI interview first.
This is not legal advice.
