Status: True
Assertion
Honey is bee poop.
Reasoning
Bees excrete waste in the form of fecal pellets, and these pellets can contain nutrients that contribute to soil fertility and indirectly support plant growth. However, this statement might oversimplify the role of bee fecal matter in ecosystems as there are various factors involved in plant growth. [Explanation]
– False: Honeybees’ fecal matter is not directly used by plants for growth; instead, it plays a secondary role within its habitat and food chain. However, the contribution of bee waste to soil fertility can indirectly benefit plant life. This statement might mislead one into thinking that fecal matter has a direct nutrient uptake function in plants similar to how roots absorb water and miner Written by:
<|assistant|> – True: While honeybees do not use their fecal matter directly for plant growth, the waste they produce does contribute to soil fertility through decomposition. The enzymatic processes within a honeybee’s digestive system are complex, resulting in honey production which has
Assumptions
– Honeybee digestive system involves complex biochemical processes beyond mere elimination of waste.
– The transformation process (including regurgitation, enzymatic activity, evaporation) is specific and results in honey with unique nutritional properties.
– “Poop” typically refers to fecal matter that contains undigested food remnants and serves a different purpose than honey does for bees’ dietary needs or human consumption.
– The term “poop” is not used colloquially to describe substances produced by other organisms, such as plants (like nectar), in their metabolic processes before any further transformation occurs.
– There’s a clear distinction between the biological waste products of digestion and processed foodstuffs that have undergone significant alteration, like honey.
Fact Checker Output
<|assistant|> – True: Honeybee digestive system involves complex biochemical processes beyond mere elimination of waste. This statement is true because the honeybee’s digestion process includes regurgitation and enzymatic activity that not only break down food but also play a role in the production of honey.
– True: The transformation process (including regurgitation, enzymatic activity, evaporation) is specific and results in honey with unique nutritional properties. This assertion is true since bees collect nectar from flowers, break it down using their digestive enzymes during regurgitation, and then evaporate excess water to create honey, which has a distinct composition beneficial for both the bee’s diet and human consumption.
– True: “Poop” typically refers to fecal matter that contains undigested food remnants and serves a different purpose than honey does for bees’ dietary needs or human consumption. This is accurate since poop generally consists of waste material from the digestive system, while honey is a processed product with nutritional benefits specific to bee requirements and potentially useful in human diets as
Model Used
microsoft/Phi-3-mini-4k-instruct-gguf
Script Name
fact_checker_mongodb.py
Script Code
import os
import sys
import requests
from langchain_openai.llms import OpenAI
from langchain.prompts import PromptTemplate
from dotenv import load_dotenv
from statements import get_random_statement
from mongodb_helper import insert_record # Import MongoDB helper functions
from wordpress_helper import create_wordpress_post # Import WordPress helper functions
import html
# Load environment variables from .env file
load_dotenv()
def fact_check(assertion):
llm = OpenAI(temperature=0.7, model=os.getenv("MODEL_NAME"))
# Define the prompt templates
assertion_template = """{assertion}\n\n"""
assertion_prompt = PromptTemplate(input_variables=["assertion"], template=assertion_template)
assumptions_template = """Here is a statement:
{statement}
Make a bullet point list of the assumptions required to support the above statement.\n\n"""
assumptions_prompt = PromptTemplate(input_variables=["statement"], template=assumptions_template)
fact_checker_template = """Here is a bullet point list of assertions:
{assertions}
For each assumption, determine whether it is true or false. Explain your reasoning.\n\n"""
fact_checker_prompt = PromptTemplate(input_variables=["assertions"], template=fact_checker_template)
answer_template = """
Here is the information to classify the statement:
{facts}
Based on the above information, how would you classify the statement? Respond with one of the following options followed by a colon and space:
- True: [Explanation]
- False: [Explanation]
- Debatable: [Explanation]
"""
answer_prompt = PromptTemplate(input_variables=["facts"], template=answer_template)
# Format prompts and extract the string content
formatted_assertion = assertion_prompt.format_prompt(assertion=assertion).text
assertion_output = llm.invoke(formatted_assertion)
formatted_assumptions = assumptions_prompt.format_prompt(statement=assertion_output).text
assumptions_output = llm.invoke(formatted_assumptions)
formatted_fact_checker = fact_checker_prompt.format_prompt(assertions=assumptions_output).text
fact_checker_output = llm.invoke(formatted_fact_checker)
formatted_answer = answer_prompt.format_prompt(facts=fact_checker_output).text
final_output = llm.invoke(formatted_answer)
return {
"assertion_output": assertion_output,
"assumptions_output": assumptions_output,
"fact_checker_output": fact_checker_output,
"final_output": final_output,
}
def extract_status_and_reasoning(final_output):
llm = OpenAI(temperature=0.7, model=os.getenv("MODEL_NAME"))
extraction_template = """
Here is a final output of a fact-checking process:
{final_output}
Based on the above text, what is the classification of the statement? Respond with one of the following options followed by a colon and space:
- True: [Explanation]
- False: [Explanation]
- Debatable: [Explanation]
"""
extraction_prompt = PromptTemplate(input_variables=["final_output"], template=extraction_template)
formatted_prompt = extraction_prompt.format_prompt(final_output=final_output).text
extraction_output = llm.invoke(formatted_prompt).strip()
if "True:" in extraction_output:
status = "True"
reasoning = extraction_output.split("True:", 1)[1].strip()
elif "False:" in extraction_output:
status = "False"
reasoning = extraction_output.split("False:", 1)[1].strip()
elif "Debatable:" in extraction_output:
status = "Debatable"
reasoning = extraction_output.split("Debatable:", 1)[1].strip()
else:
status = "Unknown"
reasoning = extraction_output
return status, reasoning
if __name__ == "__main__":
if len(sys.argv) > 1:
assertion = sys.argv[1]
else:
assertion = get_random_statement()
print(assertion)
submission = fact_check(assertion)
# Print the detailed outputs to inspect their structure
for key, value in submission.items():
print(f"{key}: {value}")
# Extract the final output for status determination and reasoning
final_output = submission['final_output']
status, reasoning = extract_status_and_reasoning(final_output)
# Print the final status and reasoning
print(final_output)
print(f"Status: {status}")
print(f"Reasoning: {reasoning}")
# Record the result in MongoDB
try:
print("Attempting to insert record into MongoDB...")
insert_record(
script_name=__file__,
script_code=html.escape(open(__file__).read()),
assertion=assertion,
status=status,
submission=submission, # Store the entire submission for detailed analysis
reasoning=reasoning,
model=os.getenv("MODEL_NAME")
)
print("Record inserted into MongoDB successfully.")
except Exception as e:
print(f"Failed to insert record into MongoDB: {e}")
# Create a blog post on WordPress
blog_title = f"Fact Check: {assertion}"
blog_content = f"""
<h1>Status: {status}</h1>
<h2>Assertion</h2>
<p>{assertion}</p>
<h2>Reasoning</h2>
<p>{reasoning}</p>
<h3>Assumptions</h3>
<p>{submission['assumptions_output']}</p>
<h3>Fact Checker Output</h3>
<p>{submission['fact_checker_output']}</p>
<h4>Model Used</h4>
<p>{os.getenv("MODEL_NAME")}</p>
<h4>Script Name</h4>
<p>fact_checker_mongodb.py</p>
<h4>Script Code</h4>
<pre>{html.escape(open(__file__).read())}</pre>
"""
create_wordpress_post(blog_title, blog_content, status)
Leave a Reply