Status: True

Assertion

A penny dropped from the Empire State Building can kill a person.

Results

The statement is mostly consistent with established facts, but it’s important to note that while falling from a significant height like 1,454 feet would not typically result in fatality due to the low energy involved, there could still be injury depending on several factors. Thus, although unlikely, some degree of risk cannot be completely ruled out, making the statement mostly true with nuanced considerations.
– True: Air resistance does indeed play a role in how fast objects fall, but its effect is more pronounced for larger or lighter objects. A penny’s relatively low mass and surface area mean air resistance has minimal impact on its descent speed.
– Debatable: The statement about urban legends being anecdotal and lacking verification aligns with the general understanding of these phenomena, but it doesn’t cover all aspects since some may have elements based on real events that are misinterpreted or exaggerated over time.
– True: Although a penny does not possess significant kinetic energy compared to heavier objects, any object falling from such heights could potentially cause harm due to the force of impact. The risk level might be low but cannot be entirely dismissed.
– Debatable

Assumptions

<|assistant|> – Urban legends are based on unverified anecdotes rather than scientific evidence or real-world observations.
– The size, weight, and material properties (e.g., copper composition) of a penny influence its potential for causing harm upon impact.
– Air resistance has a significant effect on the velocity at which objects fall from great heights.
– A penny falling from 1,454 feet (443.2 meters) would not possess substantial kinetic energy due to limited air resistance and relatively low mass.
– The potential harm caused by a falling penny depends on various factors such as the location of impact, whether someone is directly underneath it, and their physical condition.
– There have been no recorded incidents of fatalities caused by falling pennies from skyscranks worldwide.
– Fear associated with objects like pennies falling from heights serves more to illustrate cautionary principles rather than actual hazards present in high-traffic areas.

Fact Checker Output

<|assistant|> – True: Urban legends are indeed often based on anecdotes that lack verification and may include exaggerated details. They typically do not have a basis in scientific evidence or reliable observations.
– True: The size, weight, and composition of objects like pennies can influence their potential for causing harm. A heavy penny made of copper will have more mass than smaller, lighter counterparts.
– True: Air resistance does affect the velocity at which falling objects descend; however, its impact on a light object such as a penny is minimal due to the low air resistance relative to the object’s weight and size.
– False: While a penny would not have significant kinetic energy compared to larger or heavier objects, it still has some energy upon impact. However, considering the height (1,454 feet) from which it falls, its energy is relatively low and unlikely to be lethal. Still, this does not imply there are no risks at all.
– True: The potential harm caused by a falling penny would indeed depend on various factors such as location of impact, directness of the hit, and physical condition of the person involved. It’s extremely improb

Model Used

microsoft/Phi-3-mini-4k-instruct-gguf

Script Name

fact_checker_mongodb.py

Script Code

import os
import sys
import requests
from langchain_openai.llms import OpenAI
from langchain.prompts import PromptTemplate
from dotenv import load_dotenv
from statements import get_random_statement
from mongodb_helper import insert_record  # Import MongoDB helper functions
import html

# Load environment variables from .env file
load_dotenv()

# Define category IDs
CATEGORY_IDS = {
    "True": 2,
    "False": 3,
    "Debatable": 4,
    "Unknown": 6
}

def create_wordpress_post(title, content, category):
    data = {
        "title": title,
        "content": content,
        "status": "publish",
        "categories": [CATEGORY_IDS[category]]
    }

    response = requests.post(
        os.getenv("WORDPRESS_POSTS_URL"),
        json=data,
        auth=(os.getenv("WORDPRESS_USERNAME"), os.getenv("WORDPRESS_PASSWORD"))
    )

    if response.status_code == 201:
        print("Blog post created successfully.")
    else:
        print(f"Failed to create blog post: {response.status_code} - {response.text}")

def fact_check(assertion):
    llm = OpenAI(temperature=0.7, model=os.getenv("MODEL_NAME"))

    # Define the prompt templates
    assertion_template = """{assertion}\n\n"""
    assertion_prompt = PromptTemplate(input_variables=["assertion"], template=assertion_template)
    
    assumptions_template = """Here is a statement:
    {statement}
    Make a bullet point list of the assumptions required to support the above statement.\n\n"""
    assumptions_prompt = PromptTemplate(input_variables=["statement"], template=assumptions_template)
    
    fact_checker_template = """Here is a bullet point list of assertions:
    {assertions}
    For each assumption, determine whether it is true or false. Explain your reasoning.\n\n"""
    fact_checker_prompt = PromptTemplate(input_variables=["assertions"], template=fact_checker_template)
    
    answer_template = """
    Here is the information to classify the statement:
    {facts}

    Based on the above information, how would you classify the statement? Respond with one of the following options followed by a colon and space:
    - True: [Explanation]
    - False: [Explanation]
    - Debatable: [Explanation]
    """
    answer_prompt = PromptTemplate(input_variables=["facts"], template=answer_template)
    
    # Format prompts and extract the string content
    formatted_assertion = assertion_prompt.format_prompt(assertion=assertion).text
    assertion_output = llm.invoke(formatted_assertion)
    
    formatted_assumptions = assumptions_prompt.format_prompt(statement=assertion_output).text
    assumptions_output = llm.invoke(formatted_assumptions)
    
    formatted_fact_checker = fact_checker_prompt.format_prompt(assertions=assumptions_output).text
    fact_checker_output = llm.invoke(formatted_fact_checker)
    
    formatted_answer = answer_prompt.format_prompt(facts=fact_checker_output).text
    final_output = llm.invoke(formatted_answer)
    
    return {
        "assertion_output": assertion_output,
        "assumptions_output": assumptions_output,
        "fact_checker_output": fact_checker_output,
        "final_output": final_output,
    }

def extract_status_and_reasoning(final_output):
    final_output = final_output.strip()
    if "True:" in final_output:
        status_start = final_output.find("True:")
        status = "True"
    elif "False:" in final_output:
        status_start = final_output.find("False:")
        status = "False"
    elif "Debatable:" in final_output:
        status_start = final_output.find("Debatable:")
        status = "Debatable"
    else:
        return "Unknown", final_output

    reasoning = final_output[status_start + len(status) + 1:].strip()
    return status, reasoning

if __name__ == "__main__":
    if len(sys.argv) > 1:
        assertion = sys.argv[1]
    else:
        assertion = get_random_statement()
    
    print(assertion)
    submission = fact_check(assertion)
    
    # Print the detailed outputs to inspect their structure
    for key, value in submission.items():
        print(f"{key}: {value}")
    
    # Extract the final output for status determination and reasoning
    final_output = submission['final_output']
    status, reasoning = extract_status_and_reasoning(final_output)
    
    # Record the result in MongoDB
    try:
        print("Attempting to insert record into MongoDB...")
        insert_record(
            script_name="fact_checker_mongodb.py",
            script_code=html.escape(open(__file__).read()),
            assertion=assertion,
            status=status,
            submission=submission,  # Store the entire submission for detailed analysis
            model=os.getenv("MODEL_NAME")
        )
        print("Record inserted into MongoDB successfully.")
    except Exception as e:
        print(f"Failed to insert record into MongoDB: {e}")
    
    print(final_output)
    
    # Create a blog post on WordPress
    blog_title = f"Fact Check: {assertion}"
    blog_content = f"""
    <h1>Status: {status}</h1>
    <h2>Assertion</h2>
    <p>{assertion}</p>
    <h2>Results</h2>
    <p>{reasoning}</p>
    <h3>Assumptions</h3>
    <p>{submission['assumptions_output']}</p>
    <h3>Fact Checker Output</h3>
    <p>{submission['fact_checker_output']}</p>
    <h4>Model Used</h4>
    <p>{os.getenv("MODEL_NAME")}</p>
    <h4>Script Name</h4>
    <p>fact_checker_mongodb.py</p>
    <h4>Script Code</h4>
    <pre>{html.escape(open(__file__).read())}</pre>
    """
    create_wordpress_post(blog_title, blog_content, status)