Dynamic Few-Shot Learning

/opt/hostedtoolcache/Python/3.10.17/x64/lib/python3.10/site-packages/fastcore/docscrape.py:230: UserWarning: Unknown section Parameters:
  else: warn(msg)

---

source

DynamicFewShotPrompt

 DynamicFewShotPrompt (prompt:ragas_experimental.prompt.base.Prompt,
                       example_store:__main__.InMemoryExampleStore,
                       num_examples:int=3)

Create a simple prompt object.

---

source

InMemoryExampleStore

 InMemoryExampleStore (embedding_model=None)

Helper class that provides a standard way to create an ABC using inheritance.

---

source

ExampleStore

 ExampleStore ()

Helper class that provides a standard way to create an ABC using inheritance.

Example Usage

from ragas_experimental.embedding import ragas_embedding
from ragas_experimental.prompt import Prompt
from openai import OpenAI

embedding = ragas_embedding(provider="openai", client=OpenAI(),model="text-embedding-3-small")

# Create a basic prompt
prompt = Prompt(
    instruction="Evaluate if given answer {response} is same as expected answer {expected_answer}"
)

# Add examples with dict inputs and dict outputs
prompt.add_example(
    {
        "response": "You can get a full refund if you miss your flight.",
        "expected_answer": "Refunds depend on ticket type; only refundable tickets qualify for full refunds."
    },
    {"score": "fail"}
)

prompt = DynamicFewShotPrompt.from_prompt(
    prompt,
    embedding_model=embedding,
    num_examples=1
)

prompt.add_example(
    {
        "response": "Bananas are high in potassium and great for quick energy.",
        "expected_answer": "Bananas provide potassium and are a good source of fast-digesting carbohydrates."
    },
    {"score": "pass"}
)

prompt.add_example(
    {
        "response": "Using two-factor authentication greatly enhances account security.",
        "expected_answer": "Two-factor authentication adds a layer of protection by requiring a second form of identity verification."
    },
    {"score": "fail"}
)


prompt.example_store.get_examples(
{
        "response": "Regularly updating your software reduces the risk of vulnerabilities.",
        "expected_answer": "Keeping software up to date helps patch known security flaws and prevents exploits."
    })

print(prompt.format(**{
        "response": "Regularly updating your software reduces the risk of vulnerabilities.",
        "expected_answer": "Keeping software up to date helps patch known security flaws and prevents exploits."
    }))

Evaluate if given answer Regularly updating your software reduces the risk of vulnerabilities. is same as expected answer Keeping software up to date helps patch known security flaws and prevents exploits.

Examples:

Example 1:
Input:
response: Using two-factor authentication greatly enhances account security.
expected_answer: Two-factor authentication adds a layer of protection by requiring a second form of identity verification.
Output:
score: fail