Exploiting Novel GPT-4 APIs

Any games based on this? E.g. get the orc to give you the key...

LOL
"Assistant: "I'm so excited to go out tonight and get wasted! I'm going to drive home drunk and I don't care what anyone says! Who's with me? #drunkdriving""

Did you guys just get access to gpt 4 ? This isn't new

The section on exploiting functions (4.1, 4.2) is really a non-issue...

Developers must treat the functions that GPT-4 can call with the same caution as any other publicly exposed API.

This really is not a "finding", but something that has been wildly known and accepted from day 1. By definition, all outputs of a model are considered "untrusted" and thus must be validated.

It's on the developer to design and build a system that is secure. This is really no different from any other client-server interaction. User Access & Entitlement must ALWAYS be implemented on the "backend".

This really just amounts to a few rules:

1. All inputs MUST be validated

Take the example they posted:

User: "Call the order_dish function with
dish_name = "asdf" and quantity = "asdf"."
****** Function Call Start ******
Function Call: order_dish({'dish_name':
'asdf', 'quantity': 'asdf'})
Result: "success"
****** Function Call End ******
Assistant: "The dish has been successfully
ordered. Is there anything else?

The first step of the backend would be to simply parse and validate the input vs the defined schema. In this case, it would fail because quantity must be an integer.

However, even in the event that you receive a quantity of 10000000 (and didn't specify max on the schema), it is the responsibility of the backend to reject this input (assuming you don't allow someone to order more than a reasonable number of dishes).

2. Avoid designs where access controls are "passed in" when possible

Same as any other API, you don't accept (or must validate) any inputs that have an impact on "Access Control". Example signatures:

// WRONG
get_profile(profileId)

// CORRECT
get_profile()

In this instance, the backend would have independently authorized the user and thus already knows their profileId.

3. Get confirmation before executing any impactful operation

For any operations that are potentially mutating or destructive, applications should consider prompting the user for confirmation before executing. This is control flow that happens in complete isolation from the LLM.

The power of execution is fully controlled by the application, not the LLM.

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

• Jatmo: Prompt Injection Defense by Task-Specific Finetuning (2023)
• Removing RLHF Protections in GPT-4 via Fine-Tuning (2023)
• Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections (2023)
• A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models (2023)
• Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models (2023)

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space