New Delhi: From politics to films and even war, the year 2023 has demonstrated that not everything one sees or hears on the internet may be real.

With the ever-evolving Artificial Intelligence technology rapidly becoming a part of people’s lives, a sharp rise in deepfakes has raised concerns in the country about its potential to influence electoral politics, especially during the Lok Sabha polls slated to be held next year.

According to the ‘2023 State of Deepfakes Report’ by ‘Home Security Heroes’ — a US-based web security services company — deepfake videos saw a five-times increase since 2019.

In 2023, India witnessed scores of troubling instances related to deepfake videos, like the one involving actor Rashmika Mandanna, whose face was superimposed on that of a British-Indian social media influencer.

The incident sparked a nationwide debate around the implications of deepfakes and raised alarms about the invasion of privacy and the potential to cause harm. Four people were arrested for making and uploading the deepfake video on social media.

It was not just her, other film stars such as Alia Bhatt, Kajol, Aishwarya Rai and Katrina Kaif were also targeted with deepfake videos.

A couple of months ago, Prime Minister Narendra Modi flagged the use of artificial intelligence for creating deepfakes, saying this can precipitate a big crisis. He urged the media to spread awareness about its misuse and impact.

Modi said he recently suggested to ChatGPT professionals that as products like cigarettes come with health warnings, deepfakes too should carry disclosures.

Supreme Court Advocate and chairman of the International Commission on Cyber Security Law Pavan Duggal said, “It is a new emerging technology but is very quickly becoming a part of people’s lives. More and more people are going to use deepfakes, not just by cybercriminals but also in all kinds of human activities, including election processes.” “You don’t need to be a Rashmika Mandana, Katrina Kaif or Alia Bhatt. We will soon start realising that there will be deepfakes on normal users of the internet,” he said.

Duggal expressed fear that since one can now create deepfakes at the drop of a hat using online tools, a lot of which are available for free, “we are going to see far more poisoning of the data stream in the entire cyber eco-system”.

“Already, we are seeing so many deepfake videos put on pornographic websites. It is going to become a huge challenge,” he added.

Globally, deepfakes have been used to shape narratives around conflicts such as those in Ukraine and Gaza. In India, the worrying factor is their potential usage in elections.

During the recently concluded Assembly elections, deepfake videos targeted not only individual political leaders but also attempted to influence public narratives. Deepfakes targeting public figures like YS Sharmila and former Madhya Pradesh chief minister Shivraj Singh Chouhan emerged as potential threats to electoral politics.

With the general elections scheduled next year, the gravity of the situation has captured the attention of lawmakers and technology experts.

“For a country like ours, the largest democracy in the world, deepfakes and the misinformation they represent is certainly a very problematic issue to the conducting of safe, free, and fair elections,” said Rajeev Chandrashekhar, the Minister of State for Electronics and Information Technology.

Amid growing concerns over deepfakes, the government directed all social media platforms to comply with IT rules, as companies have been mandated to inform users in clear terms about prohibited content, and cautioned that violations will attract legal consequences.

The IT Ministry will closely observe the compliance of intermediaries (social media and digital platforms) in the coming weeks and decide on further amendments to the IT Rules or the law if and when needed, an official release said.

The missive underlines the hardening of the government’s stance on the issue, amid growing concerns around misinformation powered by AI-Deepfakes.

However, experts feel there is perhaps a need for the government to do more to ensure the impact of the evolution of such technologies is regulated.

“The Indian government realised that there was a need for a data privacy act. This act exists. It now has to be extended to include potential misuse of AI and deepfake to be able to protect the citizens and corporates alike from such misuse across the world,” said Pankit Desai, the cofounder and CEO of cybersecurity company Sequretek.