One of the hottest topics in artificial intelligence research right now is agentic AI. On top of large language models (LLMs), artificial intelligence (AI) agents are a programming layer that enables them to strive toward predetermined objectives. This additional software layer has the ability to gather information, decide what to do, act, and modify its behavior in response to the outcomes. Agents are able to use logic, communicate with other systems, and follow the rules and priorities that you, as the principal, have established.
For instance, businesses like Salesforce have already implemented agents that can autonomously respond to consumer inquiries across a variety of sectors and applications and identify when human assistance is necessary.
However, the most promising future of agentic AI may be found in personal agents that can act independently on your behalf. These agents will serve as your personal assistant, managing your calendar, conducting focused research and analysis, locating, negotiating, and acquiring goods and services, curating information, taking over basic communications, and ultimately learning and improving themselves.
The concept of personal AI agents has been around for decades, but it seems that the technology is finally ready for widespread use. Prominent businesses are already presenting prototype personal AI agents to their suppliers, customers, and other stakeholders, posing difficult technological and business issues. The most important question is: Can we trust AI agents to behave in our best interests? Will they only be devoted to us, or will they also be loyal to developers, advertisers, service providers, and users? How will we know?
Whether and how soon people adopt personal AI agents, as well as whether their broad use will improve or harm business relationships and brand value, will depend on the answers to these concerns.
What Could Go Wrong?
Consider a personal AI agent as someone you could hire as a real-world agent, contractor, or employee. You must determine whether a person or organization is trustworthy, capable, honest, and legally obligated to protect you before assigning them responsibilities. For instance, if a human agent had the authority to commit your money and other assets, you would very probably do a background check, get insurance, and, in some situations, ask them to post a bond.
Digital equivalents will be necessary for these and other controls, depending on the tasks performed by your own AI agents. This is due to the fact that personal AI agents are also susceptible to the dangers of unreliable workers and contractors. Users will need to be even more certain that their personal AI agents are reliable before handing over the keys to their most important assets, especially considering the potential breadth and speed of agentic AI. The most significant hazards that need to be addressed are as follows:
Exposure to Criminals
A worst-case scenario is that hackers may program (or reprogramme) personal AI agents to work against you, just like a criminal employee or identity thief would embezzle funds.
The U.S. National Institute of Standards and technologies and private Internet security companies have been regularly testing top LLMs and their agent technologies for potential security vulnerabilities, albeit it is too soon for widespread claims of personal AI agents being hijacked. These simulated hacks show that even the most secure models of today can be readily fooled into carrying out harmful tasks, such as sending phishing emails, disclosing proprietary software, and exposing passwords.
Retail Fraud by Paid Influencers and Marketers
Personal AI agents in retail could be purposefully created with skewed marketing preferences to encourage purchases from their developers or business associates.
Take online purchasing, for example. It’s already flooded with paid promotion and deceptive advertising, most of which isn’t revealed. Strong incentives exist for consumer marketers to prevent AI agents from buying in a setting where they are completely autonomous. “Free” agents might direct customers to particular companies or merchants; worse, users might not even be aware of the predetermined bias in suggestions and purchases.
Because marketers might employ software to affect or even change the LLMs that personal AI agents rely on, AI agents may be subject to the same kind of exploitation as humans, who can be duped into buying and selling from people who unfairly or even unlawfully modify information. You think your agent is getting you the greatest bargain, but changes to the inputs and reasoning it utilizes may subtly or not so subtly affect its analysis, decision-making, and learning.
Preference for Sponsors and Advertisers
Special preference for particular types of content or opinions can also be considered manipulation. Personal AI agents might, for example, be biased to favor digital content or advertise a service provider’s sponsor in the news, entertainment, and social media sectors rather than providing consumers with the information that best suits their requirements or preferences.
This is particularly likely to happen if the implementation of personal AI agents adopts the strategy used by current digital services, which provide users with free or heavily discounted access to content, leaving platform operators to profit from product placement, advertising, and other content-related indirect sources. Similar to the days of ad-supported television and radio, this business model heavily favors the interests of sponsors above those of users, allowing for both direct and indirect influence on content to best serve the interests of advertisers and their brands.
Take Spotify, a music provider that just introduced a function that lets users listen to music selected by an automatic DJ—”a personalized AI guide that knows you and your music taste so well that it can choose what to play for you.” Additionally, Spotify offers a feature known as “Discovery Mode” that permits musicians to have their work included in certain user recommendation algorithms in return for a decrease in royalties. As of right now, Spotify has stated that their AI DJ does not work with Discover Mode.
Exposure to False Information
Misinformation could purposefully or inadvertently bias the decision-making of personal AI agents, a challenge that both human principals and agents currently confront. This is the biggest risk, but it’s also maybe the most widespread. For instance, human beings and personal AI agents can both be duped by phony videos, which are sometimes used to extort or blackmail victims.
Since ChatGPT and other early AI programs were initially released, there have been frequent reports of LLMs responding to user inquiries with inaccurate or purposefully incorrect information, sometimes providing harmful health recommendations. Developers have previously been held accountable by several courts when AI chatbots provide inaccurate responses or recommendations: A passenger on Air Canada, for instance, was promised a discount that was not truly available.
The danger that personal AI agents would unintentionally utilize such data to make important decisions for their users is hard to predict because the people who spread false information have a variety of goals, such as political, criminal, financial, or just plain malevolent ones.
Combining Technical, Legal, and Market Solutions
As with their human counterparts, strict oversight, auditing, and limiting autonomy by setting levels of approval based on the possible scope and expense of delegated decisions are some ways to ensure that AI agents are honest. But putting in place such intricate control over AI agents would negate the time-saving advantages of allowing them to operate on our behalf in the first place.
Rather, we think that a mix of insurance, specialized hardware and software, and public and private regulation can reduce the need for boring micromanagement of AI agents by their users. To guarantee reliable personal AI agents, some of which are now under development, follow these three crucial steps:
1. Treat AI Agents as Fiduciaries
Attorneys, legal guardians, trustees, financial advisers, board members, and other representatives who look after their clients’ assets are referred to as fiduciaries because they are subject to a higher standard of care. Depending on the situation, a fiduciary’s legal obligations to a client may include disclosure, secrecy, accountability, loyalty, obedience, and using reasonable care and diligence to manage the client’s affairs.
The Securities and Exchange Commission and the Department of Labor, which are in charge of licensing, reporting, and disciplinary procedures, are two public organizations that strictly regulate certain fiduciaries in the United States. The National Association of Realtors, the Certified Financial Planner Board, and bar associations are examples of private self-regulatory organizations that have the authority to directly or indirectly enforce fiduciary duties. Personal AI agents will require similar systems, possibly managed by a separate organization founded by business users and AI developers.
2. Promote AI Agent Independence Market Enforcement
Company executives who stand to gain from providing their stakeholders with personal AI agents ought to collaborate with service providers, private regulators, and entrepreneurs to advance the safety and trustworthiness of agentic AI technology. This includes providing insurance and incorporating it into the use of personal AI agents.
For instance, a rapidly expanding, multibillion dollar identity theft protection industry emerged in response to the explosion in use of retail and banking applications, shielding consumers against financial fiduciaries’ illicit use of digital information. Particularly, insurers are highly motivated to monitor data managers’ actions, advocate for stricter legislation, and use private enforcement methods, such as class action lawsuits, as necessary.
Personal AI agents could be added to the offerings of other service providers who now assist consumers in managing their virtual interactions with fiduciaries. For instance, credit bureaus allow customers to freeze their financial history, making it impossible for criminals and other unauthorized users to open new credit lines or manage credit history without express consent, in addition to monitoring a variety of transactions and sending alerts based on user-defined criteria. (Some of these tools have been required to be provided to U.S. consumers at no cost since 2018.)
Likewise, regardless of who develops and runs the software, people using personal AI agents should push insurers and other service providers to allow users to watch, manage, and audit the actions of their agents. AI “credit bureaus” may provide tools to impose user-defined restrictions on the autonomy of AI agents, such as limiting the quantity or scope of important choices the agent is permitted to make in a given time frame.
3. Rely on Local Decisions
Many trust-related problems can be prevented before they start by carefully planning and implementing agentic AI technologies. Limiting personal data disclosure is a good strategy to prevent commercial or illegal manipulation of personal AI agents. Agentic AI technologies are being developed by a number of device and operating system developers, such as Google and Microsoft, to keep all sensitive data and agent decision-making localized to the user’s computer, tablet, or phone. This lowers the possibility that malicious software may steal and use sensitive data by impersonating an authorized agent and minimizes the chance that outsiders could tamper with the agent.
Additionally, the majority of agent activity will be restricted to a user’s device by Apple Intelligence, Apple’s AI architecture. The business uses what it refers to as Private Cloud Compute (PCC), which uses Apple hardware and robust encryption to access larger LLMs and processing resources when additional processing power is needed. According to the firm, personal information won’t be saved when utilizing PCC. Additionally, the business has promised to let independent privacy and security researchers check the system’s integrity whenever they choose.
Strong encryption for both internal and external processing, reliable business partners, and rigorous localization of individual user data are all elements that all organizations providing personal AI agents to its stakeholders should take into account in order to guarantee a quick rollout of these agents. It’s also critical that the behavior of the agent can be verified, and that sponsorships, paid promotions, and advertising interactions with personal AI agents are fully disclosed.
While certainly not infallible, technical solutions such as this significantly reduce the number of possible sources of failure, lowering the likelihood that fiduciary duties won’t be met.
Getting Started
Agentic AI technology has enormous potential to improve and ease people’s lives, both for businesses and for individuals. However, until consumers are certain that the technology is reliable, that agent behavior is being monitored both publicly and privately, and that there are suitable tools for monitoring, reporting, and customization that are not controlled by the agents’ creators, they will not accept AI agents.
A strong market for insurance and other third-party protection and enforcement instruments, along with a clear assignment of legal rights and responsibilities, are necessary to ensure that it is done correctly, as is the case with any fiduciary relationship. For this important technology to be adopted more quickly, industry associations, tech developers, consumer services firms, entrepreneurs, users, consumer advocates, and legislators must unite.
