Trustwave SpiderLabs Exposes Unique Cybersecurity Threats in the Public Sector. Learn More

Trustwave SpiderLabs Exposes Unique Cybersecurity Threats in the Public Sector. Learn More

Services
Capture
Managed Detection & Response

Eliminate active threats with 24/7 threat detection, investigation, and response.

twi-managed-portal-color
Co-Managed SOC (SIEM)

Maximize your SIEM investment, stop alert fatigue, and enhance your team with hybrid security operations support.

twi-briefcase-color-svg
Advisory & Diagnostics

Advance your cybersecurity program and get expert guidance where you need it most.

tw-laptop-data
Penetration Testing

Test your physical locations and IT infrastructure to shore up weaknesses before exploitation.

twi-database-color-svg
Database Security

Prevent unauthorized access and exceed compliance requirements.

twi-email-color-svg
Email Security

Stop email threats others miss and secure your organization against the #1 ransomware attack vector.

tw-officer
Digital Forensics & Incident Response

Prepare for the inevitable with 24/7 global breach response in-region and available on-site.

tw-network
Firewall & Technology Management

Mitigate risk of a cyberattack with 24/7 incident and health monitoring and the latest threat intelligence.

Solutions
BY TOPIC
Offensive Security
Solutions to maximize your security ROI
Microsoft Exchange Server Attacks
Stay protected against emerging threats
Rapidly Secure New Environments
Security for rapid response situations
Securing the Cloud
Safely navigate and stay protected
Securing the IoT Landscape
Test, monitor and secure network objects
Why Trustwave
About Us
Awards and Accolades
Trustwave SpiderLabs Team
Trustwave Fusion Security Operations Platform
Trustwave Security Colony
Partners
Technology Alliance Partners
Key alliances who align and support our ecosystem of security offerings
Trustwave PartnerOne Program
Join forces with Trustwave to protect against the most advance cybersecurity threats
SpiderLabs Blog

ChatGPT: Emerging AI Threat Landscape

ChatGPT has been available to the public since November 30, 2022. Since then, it has made headlines – from being temporarily banned from Stack Overflow because, “while the answers ChatGPT produces have a high rate of being incorrect, they typically look like they might be good, and the answers are very easy to produce, [1] to threatening to kill the college essay by writing it on behalf of a student[2]. The societal implications of AI will continue to develop as the technology becomes more broadly used, so we wanted to explore the security implications of the latest headline grabber and how it can impact privacy, data, and threat actors.

ChatGPT is a prototype chatbot released by OpenAI. The chatbot is powered by AI and is gaining more traction than previous chatbots because it not only interacts in a conversational manner but has the capability to create code and many other complex questions and requests.

I’ve had numerous colleagues over the years who have said that as technology continues to advance, something is going to change the security landscape and impact the cyber security industry drastically. It’s looking more and more like AI advances could be that something.

Developments in cutting-edge technology often steal headlines for a short period of time then fade away into academic or commercial use, but AI is something that is now becoming a part of everyday life. We, often unknowingly, interact with AI through chat bots, online shopping, fraud prevention, and voice assistant. The amount of security research in this area is growing but adoption of best practices outlined in that research could be lagging the latest technological developments.

In the case of ChatGPT, various checks have been implemented to prevent nefarious usage and knowledge sharing, but those checks are far from comprehensive – we’ve already seen at least one disclosure against GPT-3 for a prompt injection vulnerability[3].

ChatGPT has multiple use cases, and the benefits are huge – go ahead and watch it review simple code snippets. Not only will it tell you if the code is secure, but it will also suggest a more secure alternative; of course, Stack Overflow provided a far better response as to why ChatGPT solutions should not be used! Conversely, this same functionality can also be used to ‘fix’ malicious exploit code and help attackers obfuscate detection.

While ChatGPT is obviously a useful tool to educate, it can also be a useful tool in developing attacks[4]. Once an attacker has found a vulnerability, ChatGPT can be used to help develop and correct exploits. Put simply, the chatbot can be used as a virtual colleague to help discuss and perfect exploits. This can also be demonstrated by asking if code snippets are secure. In this simple SQL Injection example, a PHP code snippet was provided to ChatGPT. Once ChatGPT has identified a weakness in a code snippet, it can be asked it to create a cURL request for exploitation:

19429_picture1ggf

Figure 1: Example of ’curl’ command

Similar techniques can be employed for other vulnerability classifications, for example a simple vulnerable buffer overflow code snippet returns the below advice:

19428_picture2ggf

Figure 2: Response to vulnerable overflow code snippet

ChatGPT has very similar use cases on the defensive end of the spectrum. Asking for detection rules in various formats may provide you exactly what you need.

19428_picture2ggf

Figure 3: Response to asking for detection rules

Reminder: before using ChatGPT as a code agnostic code review, detection rule generation and exploitation tool, consider the Stack Exchange warning!

ChatGPT shares the same security concerns as any AI (as returned by ChatGPT when queried for the most significant AI security concerns):

  1. Lack of interpretability and accountability
  2. Adversarial attacks
  3. Bias and discrimination
  4. Privacy concerns

It is becoming clear that AI is going to play an ever-increasing role in our lives, so there is a need to ensure that with that adoption comes security and privacy. With new technologies and wider adoption come new attacker tools, techniques, and procedures. It is vitally important to understand the risks as well as the benefits of adopting new technology. We are going to be entering a phase of rapid change due to advancements in AI and that rapid change will lead to new attack and defense techniques. Organizations and individuals need to ensure that they are protected against new cutting-edge attacks – should this new classification of AI abuse cases become part of your threat model?

SpiderLabs used its own test lab for this research and does not upload any data to ChatGPT when performing any engagement.

[1] https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned

[2] https://www.theatlantic.com/technology/archive/2022/12/chatgpt-ai-writing-college-student-essays/672371/

[3] (https://simonwillison.net/2022/Sep/12/prompt-injection/

[4] https://informationsecuritybuzz.com/how-chatgpt-can-help-the-cybersecurity-sector/

Latest SpiderLabs Blogs

Important Security Defenses to Help Your CISO Sleep at Night

This is Part 13 in my ongoing project to cover 30 cybersecurity topics in 30 weekly blog posts. The full series can be found here.

Read More

2024 Public Sector Threat Landscape: Trustwave Threat Intelligence Briefing and Mitigation Strategies

Trustwave SpiderLabs’ 2024 Public Sector Threat Landscape: Trustwave Threat Intelligence Briefing and Mitigation Strategies report details the security issues facing public sector security teams as...

Read More

How to Create the Asset Inventory You Probably Don't Have

This is Part 12 in my ongoing project to cover 30 cybersecurity topics in 30 weekly blog posts. The full series can be found here.

Read More