IT experts concerned ChatGPT could be exploited by cybercriminals

ChatGPT exploded into the public domain in November last year, and since then has continued to dominate column inches globally. Like all technology, despite the many benefits there does appear to be with ChatGPT, some concerns have emerged from those in the known within the IT ecosystem. CNME heard the thoughts of Colin Crowley, CX Advisor at Freshworks and Steve Povolny, Principal Engineer and Director at Trellix in a bid to find out more about ChatGPT.

OpenAI that announced ChatGPT, the viral AI-powered chatbot that can write essays, emails, poems and even computer code is making waves in the news.

In the same context, Colin Crowley – CX Advisor for Freshworks Inc, a software company empowering the people who power businesses has offered some exclusive thoughts on how chatbots is going to change the business landscape-

Chatbots and Their benefits for businesses going forward? 

The main benefit will be in the enhanced natural language processing (NLP) of chatbots where chatbots would be able to understand the nuances of customer questions and provide extremely tactical and specific answers.

A bot that operates on such a robust level could potentially revolutionize help centers/knowledge bases by removing them all together and reducing all or most knowledge-searching into bot conversations.

All of this could lead to an extensive increase in self-service capabilities among customers and the further trend towards only very complex issues being left for increasingly smaller groups of agents.

Key Concerns in Using Automated Tools

There are definitely two main concerns.

Downside #1 to ChatGPT is that its advanced AI-based logic could make business leaders overconfident in what chatbots can deliver and result in an over-reliance on chatbots. (Right now, it is easier to understand where chatbots don’t add value because they aren’t as advanced as the ChatGPT version – but ChatGPT could make the limits of chatbots much less obvious.)

To counterbalance this, businesses will likely have to get a lot better at understanding. – which is to say metricizing and monetizing – the value of the human component in customer interactions.

Currently, most businesses only guess at this or define it anecdotally – but more discipline would be needed in putting data behind it and strategically guiding where to inject human beings into engagements. Simply having a human connection in some cases may be a branding opportunity for business or may be tied to higher long-term lifetime customer value or retention.

Downside #2 to ChatGPT is that, if they become more widespread, one wonders how much they will cost – especially considering that current chatbots, with all their faults, are still very pricey for companies – potentially hurting small business.

ChatGPT could create a stark, two-tier system of chatbots, with wealthy companies adopting ChatGPT and smaller companies stuck with more traditional bots.

This is still an issue now, but less so, since there are still many common benefits both small and large companies can gain from chatbots, even with current chatbot limitations.
Impact on the broader chatbot/conversational AI industry

I believe this type of technology will help get more companies on board with chatbot adoption, where adoption now can be significantly challenged by ambiguities about how to calculate ROI, concerns about poor chatbot experiences, and a lack of institutional knowledge of how to build chatbots (especially where we still live in a menu-based world where chatbots need to be built).

ChatGPT solves many of these issues by clearly being a superior experience, maximizing ROI through deflection, and also involving less building (since the chatbot is NLP-powered, so you’re relying on internal learning rather than building complicated decision trees).

Therefore, I feel ChatGPT will help to increase chatbot adoption among skeptics – but again, this has to be taken with the downside of over-obsessing on chatbots.

AI set to permanently change marketing

One major area is that the extremely conversational and intelligent nature of chatbots like ChatGPT – especially when combined with data on customers or prospects – could greatly increase the use cases where chatbots autonomously engage prospects and customers for marketing purposes.

You could imagine a world where ChatGPT could manage an entire sales-related conversation with a prospect from start to finish (especially as sales conversations are typically much more pattern-based and regulated than support conversations).

Looking at AI more broadly we will likely see the increased personalization of everything and more and more diversified channels for delivering that personalization (like chatbots).

This will make companies a lot savvier and more successful in how they position themselves to customers and make it more incumbent on them to tighten in how they form their brand.

Steve Povolny, Principal Engineer and Director at Trellix, has expressed his concerns that the capabilities of ChatGPT could be harnessed by cybercriminals attempting to use it for new forms of phishing attacks.

Below is his commentary of the potential threats posed by ChatGPT.

ChatGPT is a term that has seemingly graduated to dinner table conversation status over the last month. While its predecessors garnered interest in the data science industry, very few had realized practical uses for the average consumer.

That can be put to rest now, as the “smartest text bot ever made” has inspired thousands of innovative use cases, applicable across nearly every industry.

In the cyber realm, examples range from email generation to code creation and auditing, vulnerability discovery and much more.

However, with breakthrough advances in technology, the inevitable security concerns are never far behind.

While ChatGPT attempts to limit malicious input and output, the reality is that cyber criminals are already looking at unique ways to leverage the tool for nefarious purposes.

It isn’t hard to create hyper realistic phishing emails or exploit code, for example, simply by changing the user input or slightly adapting the output generated.

While text-based attacks such as phishing continue to dominate social engineering, the evolution of data science-based tools will inevitably lead to other mediums, including audio, video and other forms of media that could be equally effective.

Furthermore, threat actors may look to refine data processing engines to emulate ChatGPT, while removing restrictions and even enhancing these tools’ abilities to create malicious output.

While cyber security concerns have manifested, it’s important to remember that this tool has even greater potential to be used for good. It can be effective at spotting critical coding errors, describing complex technical concepts in simplistic language, and even developing script and resilient code, among other examples.

Researchers, practitioners, academia, and businesses in the cybersecurity industry can harness the power of ChatGPT for innovation and collaboration.

It will be interesting to follow this emerging battleground for computer-generated content as it enhances capabilities for both benign and malicious intent.