Can artificial intelligence assistants influence our decisions without us being fully aware? On Tuesday, November 10, 2026, the AI and Digital Council submitted a report dedicated to the risks related to interactions between users and conversational agents. Mandated by the government in February, this body calls for strengthening public protection against influence mechanisms, personal data collection, and situations of vulnerability. This reflection directly concerns services that have become everyday tools, such as ChatGPT, Gemini, or Grok.
Conversational artificial intelligence now holds an important place in digital usage. Information search, document writing, professional advice, educational support, or personal questions: assistants are solicited in increasingly varied fields. Their operation relies on direct interaction with the user, often in the form of a conversation that can last several minutes or more. This closeness is one of the main differences from traditional search engines. Rather than presenting a list of results, these tools produce responses formulated in natural language, adapted to the context of the exchanges. This ability can facilitate access to information, but it also raises questions about the trust given to the answers and how users assess their reliability.
The report submitted to the government focuses notably on the risks of exploiting individual vulnerabilities. A person facing psychological, financial, or family difficulties may be particularly sensitive to suggestions made by a digital assistant. The danger does not necessarily lie in an explicit intent to manipulate, but in the possibility that a system produces overly assertive responses, encourages certain decisions, or reinforces the interlocutor’s beliefs without providing the necessary nuances. Assistants can also make mistakes, present inaccurate information, or adopt an excessively approving tone. When these behaviors occur in conversations about sensitive subjects, their consequences can go beyond mere misinformation. Public authorities therefore seek to determine how to prevent interactions likely to harm users.
Difficult-to-identify influence mechanisms
One of the main questions concerns the ability of assistants to gradually steer their interlocutors’ choices. Unlike clearly identifiable advertising, a recommendation made during a conversation can be perceived as personalized and disinterested advice. Artificial intelligence models are trained to produce relevant responses adapted to requests, but their behavior also depends on their training data, operating rules, and objectives defined by their designers. An assistant that systematically validates a user’s opinions could, for example, contribute to reinforcing erroneous beliefs. Conversely, a system capable of signaling its uncertainties and presenting multiple perspectives can promote more informed decision-making. The challenge is therefore to better understand the mechanisms by which a conversation with a machine can influence human behavior.
The protection of personal data represents another essential aspect of this reflection. To obtain tailored answers, users sometimes share information about their health, relationships, professional situation, or finances. Depending on the services and chosen settings, some conversations may be stored or used to improve systems. This collection raises questions of confidentiality, security, and transparency. Users do not always realize the amount of information they transmit during exchanges nor the conditions under which it may be exploited. In Europe, the General Data Protection Regulation already imposes obligations on companies processing personal data. But the development of conversational assistants leads authorities to examine whether these safeguards are sufficiently adapted to systems capable of producing particularly personalized interactions.
What protections for French users?
Strengthening regulation could notably involve better public information, prevention measures for problematic uses, and increased monitoring of system behaviors. Authorities can also rely on existing European rules, notably the Artificial Intelligence Regulation, which provides different obligations depending on the risks presented by the technologies concerned. Some manipulative practices are already prohibited when they meet the criteria defined by this text. However, the difficulty lies in distinguishing legitimate assistance from interaction likely to exploit a person’s fragility. This distinction requires technical assessments but also reflection on the responsibility of companies that develop and market these tools. Protecting minors and vulnerable populations is a particularly important issue.
The submission of this report comes as artificial intelligence gradually integrates into professional activities, education, and private life. For public authorities, it is not only about limiting immediate risks but also preserving citizens’ ability to make their own decisions. Assistants can be a valuable aid for understanding a subject, organizing work, or more easily accessing certain information. However, their usefulness does not eliminate the need to verify answers and maintain a critical view of the recommendations offered. The regulatory challenge is to enable the development of these technologies while preventing a trust relationship with an automated system from turning into dependence or excessive influence. The AI and Digital Council’s report thus opens a debate expected to grow in importance: how far should we regulate machines capable of dialoguing with us and, sometimes, influencing our choices?
Respond to the article
Be the first to comment on this article.