In the European Union (EU), managerial decision-making with AI and automation is governed by existing data protection law, and by new rules that are being rolled out to deal with the issue head-on. Together, these rules have the potential to profoundly affect working conditions for a large group of workers, as digital technologies that augment or automate managerial decisions spread from the platform economy to the traditional labor market. What is at stake, in the broadest interpretation of the rules, is the employer’s prerogative to use AI and automation in decision-making.  

Three EU instruments are central. The General Data Protection Regulation (GDPR), in force since 2018, is the general framework for processing personal data. It constrains employer processing of employee data, with or without AI, and gives individuals a number of rights, including a right not to be subject to solely automated decisions that have “significant” effects.

The Artificial Intelligence Regulation (AI Act) and the Platform Work Directive (PFD), both adopted in 2024 but not yet fully applicable, add new layers.

The AI Act sets the same rules for AI systems in all EU countries, and applies to companies outside the EU that want to place AI systems on the EU market. It follows the EU’s product safety model, used for e.g. toys, machinery and electronics, where compliance is checked before a product reaches the market. The Act sorts AI uses by risk, from minimal (e.g. spam filters) to unacceptable (prohibited, e.g. social scoring of individuals). Uses that may affect people’s health, safety or fundamental rights are considered high-risk. AI systems used to carry out employer functions are generally classified as such, triggering requirements on risk management, data quality, documentation, transparency, human oversight, accuracy and cybersecurity before use.

The PFD aims to establish the correct employment status of platform workers and safeguard their working conditions. Its chapter on algorithmic management gives workers far-reaching rights to information, explanation, and human review of automated decisions.

In warehouses and factories as well as in call centers and law firms, algorithms increasingly assume the managerial role. This is an evolution of earlier management rather than something completely new: using technology to organize, monitor and discipline workers has roots going back centuries, as in Taylor’s scientific management where work was broken into measured, standardized tasks.

Contemporary technology nevertheless makes it possible to collect many forms of data continuously and simultaneously, and AI enables multidimensional analysis of granular data. Decision-making with AI is opaque: it is difficult to understand why a certain input leads to a certain output. Herein lies one of the main challenges for regulation in this field. For workers, the power imbalance inherent in the employment relationship increases the algorithmic opacity. Employees may not know what inputs the employer uses in the system, let alone understand the grounds for particular decisions.

All algorithmic management requires processing employee data, such as the number of deliveries completed during a shift, the pace at which a person picks goods from a pallet, or the tone and words a person serving a hamburger uses with customers. The GDPR therefore applies; the employer must observe its principles such as lawfulness, transparency, purpose limitation and data minimization.

Regarding the GDPR prohibition on automated decision-making, the Court of Justice of the European Union (CJEU) in 2023 established that a decision may be automated even if a human takes part in the procedure, provided that the human has no qualitative impact on the outcome. The case concerned a bank loan application; according to the CJEU, the decision was made when the automated credit score was generated, not when bank personnel decided on the loan. The notion of a fully automated decision is thus to be interpreted broadly. To be prohibited, however, the decision must also have legal or similarly significant effects. So-called algo-firing is an example of such “robot says no” decisions incompatible with EU law (in contrast to the US, where Amazon workers have been dismissed without any explanation of why the algorithm terminated them).

For other automated managerial decisions, legality under GDPR is harder to assess. Consider an employer that uses an algorithmic system to sort employees for promotion. In light of the CJEU ruling, the employer may lack the right to do so if he or she does no more than effectively “rubber stamp” the algorithm’s suggestion.

Pending further CJEU guidance, national enforcement decisions illustrate how the GDPR can protect workers. In 2021, the Italian data protection authority fined Foodinho and Deliveroo approximately EUR 2.5 million for algorithmic systems that scored riders and allocated work without sufficient transparency or opportunities to challenge the outcomes. In 2023, the Amsterdam Court of Appeal found that Uber’s human review of driver deactivations was merely symbolic.

Under the AI Act, AI used to carry out employer functions is high-risk, unless the system does not significantly impact the human task, for instance when it only prepares it. The high-risk requirements fall mainly on providers of the systems, usually AI companies rather than the employers. However, how a system is designed, for instance how transparent it is, affects how well employers can meet their own obligations under both the AI Act and employment law, which remains fully applicable (in most EU countries, for instance, an employer needs a valid ground for dismissal).

The requirement of human oversight provides that AI systems must be capable of being “effectively overseen by natural persons” through “appropriate human-machine interface tools”. It is not yet clear how this is to be fulfilled.

Under the AI Act, an employer who wants to use AI in algorithmic management must inform workers’ representatives and affected workers that they will be subject to the system. Furthermore, if an individual considers that their health, safety, or fundamental rights have been infringed in high-risk AI decision-making, the deployer of the AI system must provide a “clear and meaningful explanation of the role of the AI system in the decision-making procedure” and of the “main elements of the decision taken.” This may give workers a basis for challenging decisions on scheduling, compensation or other working conditions, where rights such as equal treatment or data protections may have been infringed.

The PFD goes further: anyone performing work subject to a decision taken or supported by an automated system has a right to an explanation, human review and rectification. This means stronger protection against algo-firing, including a right to contact a human decision-maker. Under the directive, workers’ private conversations with colleagues or representatives may not be monitored, and analysis of their tone of voice may not be used to assess their performance.

Like the GDPR, both the AI Act and the PFD carry threat of heavy fines. 

As AI takes hold in the workplace, urgent questions arise. Will employers refrain from using algorithmic management for fear of heavy sanctions, given the difficulty of explaining how systems function and of determining what constitutes meaningful human intervention? Will workers strengthen their positions as the rules open spheres traditionally within managerial prerogative? Or, will technical complexity deepen the power imbalance that the AI Act and the PFD were meant to address, leaving workers with rights on paper but little influence? It is too early to say, but the stakes are high.