Status: True

Assertion

The Nile is the longest river in the world.

Results

<|assistant|> – **Debatable**: While most experts agree on the Nile River’s status as the longest river and its historical significance is well-documented, the term “longest” could be open to interpretation based on specific criteria. Therefore, without a clear definition of what aspect (length or other factors) the statement refers to, it’s debatable whether this statement fully represents all aspects necessary for classification as true. However, given that there is strong evidence supporting both parts of the statement, classifying it as “Debatable” acknowledges the overall validity but also recognizes potential ambiguities without further context.

Assumptions

<|assistant|> – The definition of “longest” includes both length and historical significance.
– Accurate measurements of the Nile River’s length are available and recognized by authoritative sources (e.g., World Atlas, National Geographic).
– There is a consensus among experts that the Nile River holds the title for being the world’s longest river.
– The importance of historical and cultural significance to civilizations throughout history can be substantiated with credible examples or evidence.
– The statement refers to an official measurement (length in miles/kilometers) as a basis for comparison among rivers globally.
– Other major long rivers are known, allowing for comparison between the Nile River and other notable water bodies.
– Historical civilizations mentioned in relation to the Nile’s significance are well-documented and recognized by scholars.

Fact Checker Output

<|assistant|> – **True**: The definition of “longest” typically refers to the length of a river when discussing which one holds that title, but in some contexts, historical significance could also be considered part of what makes the Nile River notable as the longest. This statement is ambiguous and depends on the specific criteria being used for comparison.

– **True**: There are accurate measurements available from authoritative sources such as World Atlas and National Geographic that establish the length of the Nile River, which supports its title as one of the world’s longest rivers.

– **True**: Most experts in geography agree on the Nile River being the longest river in the world based on measurements of its total length from source to mouth (approximately 4,135 miles or 6,650 kilometers).

– **True**: The historical and cultural significance of the Nile River is well-documented. For example, it has been a cradle for ancient civilizations such as Ancient Egypt, where its annual floods were crucial to agriculture. This can be substantiated with various credible sources and archaeological evidence.

– **True**:

Model Used

microsoft/Phi-3-mini-4k-instruct-gguf

Script Name

fact_checker_mongodb.py

Script Code

import os
import sys
import requests
from langchain_openai.llms import OpenAI
from langchain.prompts import PromptTemplate
from dotenv import load_dotenv
from statements import get_random_statement
from mongodb_helper import insert_record  # Import MongoDB helper functions
import html

# Load environment variables from .env file
load_dotenv()

# Define category IDs
CATEGORY_IDS = {
    "True": 2,
    "False": 3,
    "Debatable": 4
}

def create_wordpress_post(title, content, category):
    data = {
        "title": title,
        "content": content,
        "status": "publish",
        "categories": [CATEGORY_IDS[category]]
    }

    response = requests.post(
        os.getenv("WORDPRESS_POSTS_URL"),
        json=data,
        auth=(os.getenv("WORDPRESS_USERNAME"), os.getenv("WORDPRESS_PASSWORD"))
    )

    if response.status_code == 201:
        print("Blog post created successfully.")
    else:
        print(f"Failed to create blog post: {response.status_code} - {response.text}")

def fact_check(assertion):
    llm = OpenAI(temperature=0.7)

    # Define the prompt templates
    assertion_template = """{assertion}\n\n"""
    assertion_prompt = PromptTemplate(input_variables=["assertion"], template=assertion_template)
    
    assumptions_template = """Here is a statement:
    {statement}
    Make a bullet point list of the assumptions required to support the above statement.\n\n"""
    assumptions_prompt = PromptTemplate(input_variables=["statement"], template=assumptions_template)
    
    fact_checker_template = """Here is a bullet point list of assertions:
    {assertions}
    For each assumption, determine whether it is true or false. Explain your reasoning.\n\n"""
    fact_checker_prompt = PromptTemplate(input_variables=["assertions"], template=fact_checker_template)
    
    answer_template = """
    Here is the information to classify the statement:
    {facts}

    Based on the above information, how would you classify the statement? Respond with one of the following options:
    - True
    - False
    - Debatable
    """
    answer_prompt = PromptTemplate(input_variables=["facts"], template=answer_template)
    
    # Format prompts and extract the string content
    formatted_assertion = assertion_prompt.format_prompt(assertion=assertion).text
    assertion_output = llm.invoke(formatted_assertion)
    
    formatted_assumptions = assumptions_prompt.format_prompt(statement=assertion_output).text
    assumptions_output = llm.invoke(formatted_assumptions)
    
    formatted_fact_checker = fact_checker_prompt.format_prompt(assertions=assumptions_output).text
    fact_checker_output = llm.invoke(formatted_fact_checker)
    
    formatted_answer = answer_prompt.format_prompt(facts=fact_checker_output).text
    final_output = llm.invoke(formatted_answer)
    
    return {
        "assertion_output": assertion_output,
        "assumptions_output": assumptions_output,
        "fact_checker_output": fact_checker_output,
        "final_output": final_output,
    }

def extract_status(final_output):
    final_output = final_output.lower().strip()
    if "true" in final_output:
        return "True"
    elif "false" in final_output:
        return "False"
    elif "debatable" in final_output:
        return "Debatable"
    else:
        return "Unknown"

if __name__ == "__main__":
    if len(sys.argv) > 1:
        assertion = sys.argv[1]
    else:
        assertion = get_random_statement()
    
    print(assertion)
    submission = fact_check(assertion)
    
    # Print the detailed outputs to inspect their structure
    for key, value in submission.items():
        print(f"{key}: {value}")
    
    # Extract the final output for status determination
    final_output = submission['final_output']
    
    # Determine the status based on the final output text
    status = extract_status(final_output)
    
    # Record the result in MongoDB
    try:
        print("Attempting to insert record into MongoDB...")
        insert_record(
            script_name="fact_checker_mongodb.py",
            script_code=html.escape(open(__file__).read()),
            assertion=assertion,
            status=status,
            submission=submission,  # Store the entire submission for detailed analysis
            model=os.getenv("MODEL_NAME")
        )
        print("Record inserted into MongoDB successfully.")
    except Exception as e:
        print(f"Failed to insert record into MongoDB: {e}")
    
    print(final_output)
    
    # Create a blog post on WordPress
    blog_title = f"Fact Check: {assertion}"
    blog_content = f"""
    <h1>Status: {status}</h1>
    <h2>Assertion</h2>
    <p>{assertion}</p>
    <h2>Results</h2>
    <p>{final_output}</p>
    <h3>Assumptions</h3>
    <p>{submission['assumptions_output']}</p>
    <h3>Fact Checker Output</h3>
    <p>{submission['fact_checker_output']}</p>
    <h4>Model Used</h4>
    <p>{os.getenv("MODEL_NAME")}</p>
    <h4>Script Name</h4>
    <p>fact_checker_mongodb.py</p>
    <h4>Script Code</h4>
    <pre>{html.escape(open(__file__).read())}</pre>
    """
    create_wordpress_post(blog_title, blog_content, status)