Status: True

Assertion

Bananas are berries, but strawberries are not.

Results

Although bananas fit the botanical definition of berries, strawberries do not meet this classification. Therefore, the statement is debatable as it implies that both are classified as “berries” in botany but only one (banana) fits the strict scientific criteria while the other (strawberry) does not.

– False: The definition of a “berry” in botany specifically refers to fruits with seeds enclosed within its flesh that develop from one ovary, which bananas and strawberries do not meet entirely, although there is some overlap for bananas based on seed characteristics.

– Debatable: While the statement can be classified as true or false in a strict botanical sense due to the nuances surrounding the definitions of berries (as seen with bananas and strawberries), it also reflects common language usage, making the classification more subjective and open to interpretation.

Assumptions

<|assistant|> – The definition of a “berry” in botany refers to a fruit with seeds enclosed within its flesh, developed from one ovary.
– Bananas develop from a single ovary and fit this botanical definition of a berry.
– Strawberries have seeds on their exterior surface and are derived from multiple ovaries that fuse together around the central core of the strawberry plant.
– In common language usage, people often refer to fruits based on appearance rather than strict scientific definitions.
– The statement acknowledges that this can lead to confusion between botanical classifications and everyday usage when it comes to defining what constitutes a “berry”.

Fact Checker Output

support: – True: The definition of a “berry” in botany indeed refers to a fruit with seeds enclosed within its flesh that develops from one ovary. This botanical classification helps distinguish berries from other types of fruits based on their reproductive structures.

– True: Bananas fit the botanical definition of a berry since they develop from a single ovary and have their seeds (which are actually very small and often not discernible to humans) enclosed within the flesh, despite being commonly referred to as a “berry” in everyday language.

– True: Strawberries do not fit the strict botanical definition of berries because they have seeds on their exterior surface (the “arils”) and are derived from multiple ovaries that fuse together around the central core, rather than developing from a single ovary with all seeds enclosed within.

– True: In common language usage, people often refer to fruits based on appearance or other characteristics, not strictly adhering to scientific definitions. This leads to some confusion when it comes to defining what constitutes a “berry,” as seen in the case of strawberries and bananas which are

Model Used

microsoft/Phi-3-mini-4k-instruct-gguf

Script Name

fact_checker_mongodb.py

Script Code

import os
import sys
import requests
from langchain_openai.llms import OpenAI
from langchain.prompts import PromptTemplate
from dotenv import load_dotenv
from statements import get_random_statement
from mongodb_helper import insert_record  # Import MongoDB helper functions
import html

# Load environment variables from .env file
load_dotenv()

# Define category IDs
CATEGORY_IDS = {
    "True": 2,
    "False": 3,
    "Debatable": 4,
    "Unknown": 6
}

def create_wordpress_post(title, content, category):
    data = {
        "title": title,
        "content": content,
        "status": "publish",
        "categories": [CATEGORY_IDS[category]]
    }

    response = requests.post(
        os.getenv("WORDPRESS_POSTS_URL"),
        json=data,
        auth=(os.getenv("WORDPRESS_USERNAME"), os.getenv("WORDPRESS_PASSWORD"))
    )

    if response.status_code == 201:
        print("Blog post created successfully.")
    else:
        print(f"Failed to create blog post: {response.status_code} - {response.text}")

def fact_check(assertion):
    llm = OpenAI(temperature=0.7, model=os.getenv("MODEL_NAME"))

    # Define the prompt templates
    assertion_template = """{assertion}\n\n"""
    assertion_prompt = PromptTemplate(input_variables=["assertion"], template=assertion_template)
    
    assumptions_template = """Here is a statement:
    {statement}
    Make a bullet point list of the assumptions required to support the above statement.\n\n"""
    assumptions_prompt = PromptTemplate(input_variables=["statement"], template=assumptions_template)
    
    fact_checker_template = """Here is a bullet point list of assertions:
    {assertions}
    For each assumption, determine whether it is true or false. Explain your reasoning.\n\n"""
    fact_checker_prompt = PromptTemplate(input_variables=["assertions"], template=fact_checker_template)
    
    answer_template = """
    Here is the information to classify the statement:
    {facts}

    Based on the above information, how would you classify the statement? Respond with one of the following options followed by a colon and space:
    - True: [Explanation]
    - False: [Explanation]
    - Debatable: [Explanation]
    """
    answer_prompt = PromptTemplate(input_variables=["facts"], template=answer_template)
    
    # Format prompts and extract the string content
    formatted_assertion = assertion_prompt.format_prompt(assertion=assertion).text
    assertion_output = llm.invoke(formatted_assertion)
    
    formatted_assumptions = assumptions_prompt.format_prompt(statement=assertion_output).text
    assumptions_output = llm.invoke(formatted_assumptions)
    
    formatted_fact_checker = fact_checker_prompt.format_prompt(assertions=assumptions_output).text
    fact_checker_output = llm.invoke(formatted_fact_checker)
    
    formatted_answer = answer_prompt.format_prompt(facts=fact_checker_output).text
    final_output = llm.invoke(formatted_answer)
    
    return {
        "assertion_output": assertion_output,
        "assumptions_output": assumptions_output,
        "fact_checker_output": fact_checker_output,
        "final_output": final_output,
    }

def extract_status_and_reasoning(final_output):
    final_output = final_output.strip()
    if "True:" in final_output:
        status_start = final_output.find("True:")
        status = "True"
    elif "False:" in final_output:
        status_start = final_output.find("False:")
        status = "False"
    elif "Debatable:" in final_output:
        status_start = final_output.find("Debatable:")
        status = "Debatable"
    else:
        return "Unknown", final_output

    reasoning = final_output[status_start + len(status) + 1:].strip()
    return status, reasoning

if __name__ == "__main__":
    if len(sys.argv) > 1:
        assertion = sys.argv[1]
    else:
        assertion = get_random_statement()
    
    print(assertion)
    submission = fact_check(assertion)
    
    # Print the detailed outputs to inspect their structure
    for key, value in submission.items():
        print(f"{key}: {value}")
    
    # Extract the final output for status determination and reasoning
    final_output = submission['final_output']
    status, reasoning = extract_status_and_reasoning(final_output)
    
    # Record the result in MongoDB
    try:
        print("Attempting to insert record into MongoDB...")
        insert_record(
            script_name="fact_checker_mongodb.py",
            script_code=html.escape(open(__file__).read()),
            assertion=assertion,
            status=status,
            submission=submission,  # Store the entire submission for detailed analysis
            model=os.getenv("MODEL_NAME")
        )
        print("Record inserted into MongoDB successfully.")
    except Exception as e:
        print(f"Failed to insert record into MongoDB: {e}")
    
    print(final_output)
    
    # Create a blog post on WordPress
    blog_title = f"Fact Check: {assertion}"
    blog_content = f"""
    <h1>Status: {status}</h1>
    <h2>Assertion</h2>
    <p>{assertion}</p>
    <h2>Results</h2>
    <p>{reasoning}</p>
    <h3>Assumptions</h3>
    <p>{submission['assumptions_output']}</p>
    <h3>Fact Checker Output</h3>
    <p>{submission['fact_checker_output']}</p>
    <h4>Model Used</h4>
    <p>{os.getenv("MODEL_NAME")}</p>
    <h4>Script Name</h4>
    <p>fact_checker_mongodb.py</p>
    <h4>Script Code</h4>
    <pre>{html.escape(open(__file__).read())}</pre>
    """
    create_wordpress_post(blog_title, blog_content, status)