AI Act: What Organizations Must Do for Responsible AI Governance

The AI Act: New Obligations for Users of AI Systems

The European AI Act imposes obligations not only on developers and providers of artificial intelligence, but also on organizations that use AI systems. Whether it involves medical decision-making, chatbots, CV screening, or automated invoice processing, users are faced with concrete responsibilities.

This article outlines what these obligations entail and how organizations can prepare.

Human oversight is Mandatory

Organizations that use AI must take active measures. Passive use is no longer sufficient. Policies must be established, oversight procedures implemented, employees trained in AI literacy, and responsibilities must be clearly defined. These obligations apply not only to high risk systems but also to applications that have indirect effects on people or processes. For example, a chatbot that automatically handles customer inquiries also requires control and transparency from the outset.

Users are Responsible for Data Quality

The quality of AI systems highly depends on the data being used for training. Users must ensure that data is sufficiently representative and relevant. This means verifying that the data aligns with the intended purpose and that different perspectives and groups are adequately represented. For example, datasets should not consist solely of one specific age group, gender, or cultural background to prevent biased or one sided outcomes.

Users must also assess data quality based on the goals for which the system is used. Determine in advance which outcomes the system should support, and verify whether the data is suitable, complete, and up to date.

Mandatory Incident Reporting

Incidents in which an AI system fails, causes harm, or produces unexpected outcomes must be reported by the user to both the provider and the competent supervisory authority. Depending on the AI system, different authorities may be responsible, so they must be identified in advance. Timely reporting helps prevent further damage, reduces legal liability, and protects the organization's reputation. Therefore, organizations need a reporting protocol that identifies the relevant authority for each AI system, assigns responsibilities, and trains employees to recognize incidents.

Registration in the European Database

For certain AI systems, users must register the actual use of the system in an EU managed database. This increases market oversight and provides transparency about where and how AI is used within the Union.

If your organization uses a GPT or Copilot solution, registration is usually not required for simple applications such as text suggestions or email assistance. However, when AI is used in decision-making that affects people, such as recruitment or performance assessment, registration becomes mandatory. It is therefore essential to analyse your AI usage and determine whether your application falls under this requirement.

Awareness as the First Step

The AI Act marks a turning point in how organizations handle artificial intelligence. Responsibility does not lie solely with developers but also with users, who must ensure safe, fair, and transparent AI deployment. Implementing measures, training staff, ensuring human oversight, and complying with reporting and registration requirements are not optional; they form the foundation for future proof AI use within the European framework.

A proactive approach not only protects end users but also strengthens trust in technology. Do not wait until the law is fully in force. Contact us today to discuss how your organization can comply with the AI Act.

Contact us