Status: True

Assertion

The Eiffel Tower is located in Paris.

Reasoning

The Great Barrier Reef is located off the northeastern coastline within Australia’s territorial waters in the Indo-Pacific basin, supports a wide range of marine biodiversity, covers an extensive area with coral coverage, and plays a significant role in maintaining ocean health. Tourists can enjoy the reef responsibly by adhering to strict protocols during tours, supporting eco-friendly resorts, and organizations that contribute to conservation efforts. The statement is classified as true because it accurately reflects known facts about the Great Barrier Reef’s geographical location, biodiversity, area size, role in ocean health, and responsible tourism practices supported by scientific consensus and environmental guidelines.

Assumptions

<|assistant|> – The location is off the northeastern coastline within Australia’s territorial waters in the Indo-Pacific basin.

– Three unique features include:

* Biodiversity: It hosts an unparalleled diversity of marine species, including over 2,000 fish species, six species of sea snakes, and more than 30 types of whales and dolphins.

* Coral Coverage: The reef has the world’s largest collection of coral reefs, with a total area of approximately 344,400 square kilometers covered by hard and soft corals.

* Ecological Significance: It plays a crucial role in supporting global marine life through its extensive food webs and is critical for maintaining the ocean’s health.

– For tourists to enjoy the reef responsibly, they can:

* Participate in guided snorkeling or diving tours that adhere to strict environmental protocols to prevent damage to marine life and coral structures.

* Support eco-friendly resorts and organizations that contribute to conservation

Fact Checker Output

support>
True: The assertion about the location being off the northeastern coastline within Australia’s territorial waters in the Indo-Pacific basin refers to the Great Barrier Reef, which is indeed located along the northeast coast of Queensland, Australia, and lies in the Coral Sea region of the Pacific Ocean.

True: The three unique features listed are accurate descriptions of the Great Barrier Reef’s characteristics. It supports an incredible range of marine biodiversity, covers a vast area with extensive coral coverage, and plays a significant role in maintaining ocean health.

True: The suggestions for tourists to enjoy the reef responsibly align with conservation guidelines by promoting environmentally friendly practices such as adhering to strict protocols during tours, supporting eco-friendly resorts, and organizations that contribute to conservation efforts. These actions help minimize the impact of human activities on the Great Barrier Reef’s delicate ecosystem.

Model Used

microsoft/Phi-3-mini-4k-instruct-gguf

Script Name

fact_checker_mongodb.py

Script Code

import os
import sys
import requests
from langchain_openai.llms import OpenAI
from langchain.prompts import PromptTemplate
from dotenv import load_dotenv
from statements import get_random_statement
from mongodb_helper import insert_record  # Import MongoDB helper functions
from wordpress_helper import create_wordpress_post  # Import WordPress helper functions
import html

# Load environment variables from .env file
load_dotenv()

def fact_check(assertion):
    llm = OpenAI(temperature=0.7, model=os.getenv("MODEL_NAME"))

    # Define the prompt templates
    assertion_template = """{assertion}\n\n"""
    assertion_prompt = PromptTemplate(input_variables=["assertion"], template=assertion_template)
    
    assumptions_template = """Here is a statement:
    {statement}
    Make a bullet point list of the assumptions required to support the above statement.\n\n"""
    assumptions_prompt = PromptTemplate(input_variables=["statement"], template=assumptions_template)
    
    fact_checker_template = """Here is a bullet point list of assertions:
    {assertions}
    For each assumption, determine whether it is true or false. Explain your reasoning.\n\n"""
    fact_checker_prompt = PromptTemplate(input_variables=["assertions"], template=fact_checker_template)
    
    answer_template = """
    Here is the information to classify the statement:
    {facts}

    Based on the above information, how would you classify the statement? Respond with one of the following options followed by a colon and space:
    - True: [Explanation]
    - False: [Explanation]
    - Debatable: [Explanation]
    """
    answer_prompt = PromptTemplate(input_variables=["facts"], template=answer_template)
    
    # Format prompts and extract the string content
    formatted_assertion = assertion_prompt.format_prompt(assertion=assertion).text
    assertion_output = llm.invoke(formatted_assertion)
    
    formatted_assumptions = assumptions_prompt.format_prompt(statement=assertion_output).text
    assumptions_output = llm.invoke(formatted_assumptions)
    
    formatted_fact_checker = fact_checker_prompt.format_prompt(assertions=assumptions_output).text
    fact_checker_output = llm.invoke(formatted_fact_checker)
    
    formatted_answer = answer_prompt.format_prompt(facts=fact_checker_output).text
    final_output = llm.invoke(formatted_answer)
    
    return {
        "assertion_output": assertion_output,
        "assumptions_output": assumptions_output,
        "fact_checker_output": fact_checker_output,
        "final_output": final_output,
    }

def extract_status_and_reasoning(final_output):
    llm = OpenAI(temperature=0.7, model=os.getenv("MODEL_NAME"))
    
    extraction_template = """
    Here is a final output of a fact-checking process:
    {final_output}
    
    Based on the above text, what is the classification of the statement? Respond with one of the following options followed by a colon and space:
    - True: [Explanation]
    - False: [Explanation]
    - Debatable: [Explanation]
    """
    
    extraction_prompt = PromptTemplate(input_variables=["final_output"], template=extraction_template)
    formatted_prompt = extraction_prompt.format_prompt(final_output=final_output).text
    extraction_output = llm.invoke(formatted_prompt).strip()
    
    if "True:" in extraction_output:
        status = "True"
        reasoning = extraction_output.split("True:", 1)[1].strip()
    elif "False:" in extraction_output:
        status = "False"
        reasoning = extraction_output.split("False:", 1)[1].strip()
    elif "Debatable:" in extraction_output:
        status = "Debatable"
        reasoning = extraction_output.split("Debatable:", 1)[1].strip()
    else:
        status = "Unknown"
        reasoning = extraction_output
    
    return status, reasoning

if __name__ == "__main__":
    if len(sys.argv) > 1:
        assertion = sys.argv[1]
    else:
        assertion = get_random_statement()
    
    print(assertion)
    submission = fact_check(assertion)
    
    # Print the detailed outputs to inspect their structure
    for key, value in submission.items():
        print(f"{key}: {value}")
    
    # Extract the final output for status determination and reasoning
    final_output = submission['final_output']
    status, reasoning = extract_status_and_reasoning(final_output)
    
    # Print the final status and reasoning
    print(final_output)
    print(f"Status: {status}")
    print(f"Reasoning: {reasoning}")

    # Record the result in MongoDB
    try:
        print("Attempting to insert record into MongoDB...")
        insert_record(
            script_name=__file__,
            script_code=html.escape(open(__file__).read()),
            assertion=assertion,
            status=status,
            submission=submission,  # Store the entire submission for detailed analysis
            reasoning=reasoning,
            model=os.getenv("MODEL_NAME")
        )
        print("Record inserted into MongoDB successfully.")
    except Exception as e:
        print(f"Failed to insert record into MongoDB: {e}")
    
    # Create a blog post on WordPress
    blog_title = f"Fact Check: {assertion}"
    blog_content = f"""
    <h1>Status: {status}</h1>
    <h2>Assertion</h2>
    <p>{assertion}</p>
    <h2>Reasoning</h2>
    <p>{reasoning}</p>
    <h3>Assumptions</h3>
    <p>{submission['assumptions_output']}</p>
    <h3>Fact Checker Output</h3>
    <p>{submission['fact_checker_output']}</p>
    <h4>Model Used</h4>
    <p>{os.getenv("MODEL_NAME")}</p>
    <h4>Script Name</h4>
    <p>fact_checker_mongodb.py</p>
    <h4>Script Code</h4>
    <pre>{html.escape(open(__file__).read())}</pre>
    """
    create_wordpress_post(blog_title, blog_content, status)