Protecting personal data in the age of AI

As AI reshapes the candidate journey, data privacy becomes increasingly important.

When organizations discuss AI and data privacy, the conversation often begins with regulation – and for good reason. As AI becomes embedded in hiring, organizations must navigate an increasingly complex landscape of privacy, employment, and AI governance requirements.

‍

But regulation answers only part of the question.

‍

The other part is human.

‍

Before organizations collect, process, or govern personal data, people first choose to share it. AI is changing those choices in ways we are only beginning to understand. Candidates are increasingly putting deeply personal information into AI tools, from career histories and skill gaps to salary expectations, perceived weaknesses, and accommodation needs. They're sharing this data as they use AI to prepare for interviews, rewrite resumes, and strengthen their applications, creating a new data trail long before an employer ever receives an application.

‍

As organizations work to comply with evolving privacy laws, they must also understand how AI is reshaping the relationship between individual candidates and their own personal data.

As candidates turn to AI, their personal data goes with them

Increasingly, candidates are turning to AI to prepare for job searches. Candidates are using AI coaches to practice interviews, tailoring resumes for individual opportunities, and asking AI to help them understand what employers are looking for.

‍

As AI becomes more accessible, it is quickly becoming part of the job search itself. But there is another consequence that receives far less attention: these interactions often ask candidates to disclose something personal.

‍

Career histories. Employment gaps. Salary expectations. Self-assessed weaknesses. Accommodation needs. Behavioral assessments. Relocation preferences.

‍

These are no longer shared only with prospective employers. They are increasingly shared with AI systems that candidates neither designed nor control. The result is a subtle but profound shift. The privacy conversation no longer begins when an employer receives an application. It begins much earlier, at the moment a candidate turns to AI for help.

A new governance question

Alongside this evolution, there should also be a shift in the questions organizations are asking.

‍

Much of today's discussion understandably focuses on what AI systems are capable of doing. Equally important is understanding the relationship those systems have with the people whose information they process.

‍

Many talent platforms aggregate publicly available professional information from multiple sources before organizing it into searchable candidate profiles. This leaves candidates with what can be termed a visibility gap.

‍

Candidates rarely see how the systems interpret their experience. A platform may identify strengths, highlight “gaps,” and score a profile against a particular search long before a human reviews it. Yet a gap is not necessarily missing experience. It is often experience expressed in language that does not align with what that particular search is evaluating. Candidates almost never see these scorecards, understand which criteria mattered most, or have an opportunity to reframe how their experience is interpreted before it enters the hiring process.

The regulatory lens: Where AI meets the law

These shifts in how candidate information is shared, represented, and used are not happening in a vacuum. In fact, many of the concerns they raise map directly to existing and emerging areas of law.

‍

Three questions from the candidate perspective are particularly important: What is my data being used for? Is the information being used about me in the employment context accurate? Could this information be used in a way that discriminates against me?

‍

The first is fundamentally a privacy question. Privacy laws are designed to give people notice about how their information is used and, in certain circumstances, the ability to access it or opt out. Data broker laws address a particularly important version of this problem: information collected behind the scenes by a company with which an individual has no direct relationship. There is an important nuance when that information is publicly available. United States (US) privacy laws generally exclude publicly available information from the definition of personal information. But in California, public information used to make sensitive inferences about someone, such as their income, race, or gender, can still raise data broker considerations.

‍

The second question is accuracy. Does the information being used actually reflect a candidate's full skills and experience? The Fair Credit Reporting Act (FCRA) addresses certain situations in which information is assembled or evaluated behind the scenes for employment purposes. When that information contributes to someone being denied employment, the law can provide rights to access it, receive an adverse action notice, and seek to correct it.

‍

Thirdly, there is discrimination. Existing anti-discrimination laws continue to apply to AI, while US states, local jurisdictions, and the European Union (EU) are also developing requirements to ensure that consumer concerns are addressed.

‍

For candidates, these laws provide different forms of protection around how their information is used, whether it is accurate, and whether it is used fairly. For organizations developing or deploying AI, they create a different challenge: understanding which existing laws already apply while navigating a new generation of requirements specifically focused on AI and automated decision-making.

‍

Established laws, including the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), data broker laws, the FCRA, and existing anti-discrimination laws, can all apply to AI systems. At the same time, newer AI-specific requirements are adding another layer to the regulatory landscape.

‍

In the United States, that emerging landscape is particularly fragmented. The federal government has stepped back toward a lighter-touch approach to AI, while states and local governments have become more active. New York City was an early mover with bias-audit and candidate-notice requirements; California and Colorado have taken different approaches of their own.

‍

Two questions are particularly important when navigating these different requirements: what triggers them, and who is responsible for complying with them?

‍

The first is about what the technology actually does. There is a meaningful difference between technology that schedules an interview or organizes candidate information and technology that scores, ranks, or recommends candidates. Different laws draw that line differently. New York and the CCPA focus on systems that replace or substantially assist human decision-making; Colorado uses a “materially influence” standard for consequential decisions; and California's Fair Employment and Housing Act (FEHA) regulations extend to systems that facilitate human decision-making in employment.

‍

The second question is who bears responsibility. Under many of these requirements, primary obligations such as conducting bias audits or providing candidate notices fall to employers. But California's FEHA framework can extend potential liability to the AI company as well.

‍

That question of responsibility is also playing out in litigation, which has increasingly become a leading edge in determining how existing laws apply to AI. The Mobley v. Workday case is an important example. Workday argued that discrimination claims should be directed at the employers using its technology. The court allowed claims to proceed on the theory that Workday could be acting as an employer's agent. It is a meaningful development for AI providers, which cannot necessarily assume that responsibility for employment outcomes stops with the employer deploying their technology.

‍

These questions are not unique to the US. The EU AI Act takes a more comprehensive approach, but it requires similar line-drawing around the role a system plays. Employment and recruitment systems can be considered high risk when they analyze, filter, score, rank, or generate individual evaluations that are relied upon. Systems performing purely procedural or preparatory tasks, such as scheduling interviews, may be treated differently.

‍

Across jurisdictions, these distinctions make clear why organizations need to understand not simply whether or not to use AI, but how they use it. That is where internal governance becomes important.

‍

There is no single US law requiring every organization to establish a broad AI governance policy. But having one can mitigate regulatory risk, establish accountability, create credibility with customers, and help employees recognize and manage risk as AI uses evolve.

‍

The building blocks are familiar from privacy and cybersecurity: clear roles and responsibilities, risk assessments, vendor management, employee training, review and approval of new or changed AI uses, and ongoing monitoring. And because the regulatory landscape is changing, those practices need to be continually benchmarked against the laws that apply.

Where governance and trust meet

The ways candidates use AI will continue to evolve, as will the regulations governing it. Responsible AI governance requires understanding both.

‍

Candidates need confidence in how their information is used and interpreted. Organizations need clarity about where AI is influencing decisions, what regulatory obligations those uses create, and who is accountable for them.

These are not separate challenges. Together, they define what responsible stewardship of personal data increasingly requires in an AI-enabled world.

‍

As AI becomes an increasingly important intermediary between people and opportunity, perhaps the question that connects both sides is this: What will it take to remain worthy of the trust people place in these systems?