Prolog Example Generator for Fine Tuning/Few-Shot Learning

I need some examples of converting natural language queries into Prolog queries, given some kind of schema or domain context. I want to use these examples to fine-tune a GPT-series model for Prolog.

Oct 06, 2025

Motivation

I need some examples of converting natural language queries into Prolog queries, given some kind of schema or domain context. I want to use these examples to fine-tune a GPT-series model for Prolog.

Given a Prolog file containing:

balance(UserID, AccountNumber, Balance) :- account(UserID, AccountNumber, Balance).
account_transactions(AccountNumber, Transactions) :- findall(TransactionID, transaction(TransactionID, AccountNumber, _, _, _), Transactions).
total_credits(AccountNumber, Total) :- findall(Amount, transaction(_, AccountNumber, credit, Amount, _), Amounts), sum_list(Amounts, Total).

It should be able to create such examples:

## Natural Language to Prolog Query Translation

- **Natural Language Query:** Find the balance for user with UserID `101`.
  - **Prolog Query:** `balance(101, 123456, Balance).`

- **Natural Language Query:** List all transactions for account number `123456`.
  - **Prolog Query:** `account_transactions(123456, Transactions).`

- **Natural Language Query:** Calculate the total amount of credits for account number `123456`.
  - **Prolog Query:** `total_credits(123456, Total).`

By selecting a few of these examples, I can create a few-shot learning prompt for the model, or even use them to fine-tune the model or distill knowledge.

Building the Agent

The instructions for the agent will be:

Your goal is to generate data for fine tuning a GPT-series model for Prolog.
    
    Our task is to translate natural language queries into Prolog queries,
    given a set of pre-existing Prolog facts and rules.
    
    You have access to a tool which runs prolog code for you via
    querying a Prolog interpreter.
    
    Every time you receive a message from the user, you should:
    
    - Understand the example the type of user wants to build.
    - Formulate a set of Prolog facts and rules that would be relevant to the example.
    - Formulate the corresponding Prolog query.
    - Use Prolog to test your query and see it works as expected.
    
    Return a nicely formatted example for use with fine-tuning.
    
    The `run_prolog` tool takes a single string argument which is the Prolog query to run.
    The tool returns the results of the query as a string.
    Add facts and rules by using assert/1. Retract with retract/1. It is a simple
    interface to a SWI-Prolog interpreter via PySwip.

Prolog Tools

We can give the Agent a few tools to run Prolog code in different ways:

Run Prolog from Scratch

If the agent prefers to use a clean knowledge base each time it runs a query, this tool will instantiate a new Prolog interpreter instance for each call.

def run_prolog_from_scratch(query: str) -> str:
    “”“
    Run a Prolog query and return the results as a string.
    “”“
    prolog = Prolog()
    try:
        results = list(prolog.query(query))
        return str(results)
    except Exception as e:
        return f”Error: {str(e)}”

Run Prolog with an Existing Instance

In case the agent wants to maintain state between calls, it can use this tool which takes an existing Prolog instance.

def run_prolog_with_instance(prolog_instance: Prolog, query: str) -> str:
    “”“
    Run a Prolog query using the given Prolog instance and return the results as a string.
    “”“
    try:
        results = list(prolog_instance.query(query))
        return str(results)
    except Exception as e:
        return f”Error: {str(e)}”

Run Prolog with a Prelude

This is useful to prommpt the agent in the right direction by providing some initial facts and rules. I kept a simple line-by-line parser, so we don’t need to delve into temporary files.

def run_prolog_with_prelude(prelude: str, query: str) -> str:
    “”“
    Run a Prolog query after loading a prelude of Prolog code.
    “”“
    prolog = Prolog()
    try:
        # Load the prelude
        for line in prelude.strip().splitlines():
            if line.strip():
                prolog.assertz(line.strip().rstrip(’.’))
        results = list(prolog.query(query))
        return str(results)
    except Exception as e:
        return f”Error: {str(e)}”

Example Runs

Here are actual example runs from the system, showing different domains and the agent’s behavior:

Run 1: Banking Domain with Balance Calculation (Failed)

The agent attempted to create a banking domain example but encountered errors with rule definitions. Initially tried to assert facts and rules together, which caused syntax errors:

Error: ‘error(existence_error(procedure, /(:-, 2)), context(/(pyrun, 2), _198))’

After fixing the syntax by properly using assert() for rules, the agent successfully created:

% Facts
account(12345).
account(67890).
transaction(1, 12345, credit, 1000, ‘2023-01-01’).
transaction(2, 12345, debit, 200, ‘2023-01-02’).
transaction(3, 67890, credit, 1500, ‘2023-01-01’).
transaction(4, 67890, debit, 300, ‘2023-01-03’).

% Rule
calculate_balance(AccountId, Balance) :- 
    findall(Credit, transaction(_, AccountId, credit, Credit, _), Credits),
    findall(Debit, transaction(_, AccountId, debit, Debit, _), Debits),
    sum_list(Credits, TotalCredits),
    sum_list(Debits, TotalDebits),
    Balance is TotalCredits - TotalDebits.

Natural Language Query: “What is the balance of account 12345?” Prolog Query: calculate_balance(12345, Balance)

Run 2: Banking Domain with Balance Calculation (Successful)

The agent created a simpler banking example that worked correctly. Here’s the detailed step-by-step process:

Assert facts: The agent first established the basic facts:

assert(account(1)), assert(account(2)), 
assert(transaction(101, 1, credit, 500, ‘2023-10-01’)), 
assert(transaction(102, 1, debit, 200, ‘2023-10-02’)), 
assert(transaction(103, 1, credit, 300, ‘2023-10-03’)), 
assert(transaction(201, 2, debit, 100, ‘2023-10-01’)), 
assert(transaction(202, 2, credit, 200, ‘2023-10-02’))

Result: [{}] (successful assertion)

Define the balance rule: The agent then properly asserted the balance calculation rule:

assert((balance(AccountId, Balance) :- 
    findall(Amount, (transaction(_, AccountId, credit, Amount, _)), Credits),
    findall(Amount, (transaction(_, AccountId, debit, Amount, _)), Debits),
    sum_list(Credits, TotalCredits),
    sum_list(Debits, TotalDebits),
    Balance is TotalCredits - TotalDebits))

Result: [{’AccountId’: Variable(71), ‘Balance’: Variable(74), ...}] (rule successfully stored)

Execute the query: Finally, the agent tested the balance calculation:

balance(1, Balance)

Result: [{’Balance’: 600}] (Credits: 500+300=800, Debits: 200, Balance: 800-200=600)

Natural Language Query: “What is the current balance of account 1?” Prolog Query: balance(1, Balance)

Run 3: E-commerce Sales Analysis (Successful)

The agent created a comprehensive e-commerce example with multiple analysis rules:

% Facts about transactions
transaction(1, ‘prod_001’, ‘cust_001’, 5, ‘2023-10-01’).
transaction(2, ‘prod_002’, ‘cust_002’, 2, ‘2023-10-02’).
transaction(3, ‘prod_001’, ‘cust_003’, 3, ‘2023-10-02’).
transaction(4, ‘prod_003’, ‘cust_001’, 1, ‘2023-10-03’).

% Facts about products
product(’prod_001’, ‘Laptop’, 1000).
product(’prod_002’, ‘Smartphone’, 800).
product(’prod_003’, ‘Tablet’, 500).

% Facts about customers
customer_info(’cust_001’, ‘Alice’, ‘USA’).
customer_info(’cust_002’, ‘Bob’, ‘Canada’).
customer_info(’cust_003’, ‘Charlie’, ‘UK’).

% Analysis rules
total_sales(ProductID, Total) :- 
    findall(Quantity, transaction(_, ProductID, _, Quantity, _), Quantities),
    sum_list(Quantities, Total).

average_spending(CustomerID, Average) :- 
    findall(Sum, (transaction(_, ProductID, CustomerID, Quantity, _), 
    product(ProductID, _, Price), Sum is Price * Quantity), Sums),
    sum_list(Sums, Total),
    length(Sums, Count),
    Count > 0,
    Average is Total / Count.

high_value_customer(CustomerID) :- 
    findall(Sum, (transaction(_, ProductID, CustomerID, Quantity, _), 
    product(ProductID, _, Price), Sum is Price * Quantity), Sums),
    sum_list(Sums, Total),
    Total > 2000.

Natural Language Queries:

“Find the total sales quantity for product ‘prod_001’.” → total_sales(’prod_001’, Total)
“Find the average spending for customer ‘cust_001’.” → average_spending(’cust_001’, Average)
“Identify high-value customers with total spending over $2000.” → high_value_customer(CustomerID)

All are supported by underlying rules. I guess I can improve the output by asking the agent to always output the original intended query, even if it had to create new predicates.

The agent successfully demonstrated the ability to create complex domain-specific examples across different business scenarios, with the main learning being the importance of proper Prolog syntax when asserting rules programmatically.

Conclusion

This approach shows promise for generating domain-specific examples for fine-tuning language models in Prolog. With some refinements, such as better error handling and more robust Prolog interaction, this could be a powerful tool for creating training data for specialized symbolic AI applications.

Given more engineering effort, I can cleary see this kind of system able to create Prolog examples that can train industrial-grade models, which are so useful for any users that today have to use SQL or pandas for data analysis.

The code is available at https://github.com/roehst/prolog-tuner. AI-free writing with help from Copilot to update the code examples.

Discussion about this post

Ready for more?