(9 Dec 2024) In many cases, AI systems gather external information to use as context when answering a particular query. For example, to answer a question about a medical condition, the system might reference recent research papers on the topic. Even with this relevant context, models can make mistakes with what feels like high doses of confidence. When a model errs, how can we track that specific piece of information from the context it relied on — or lack thereof?
To help tackle this obstacle, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers created ContextCite, a tool that can identify the parts of external context used to generate any particular statement, improving trust by helping users easily verify the statement.
“AI assistants can be very helpful for synthesizing information, but they still make mistakes,” says Ben Cohen-Wang, an MIT PhD student in electrical engineering and computer science, CSAIL affiliate, and lead author on a new paper about ContextCite. “Let’s say that I ask an AI assistant how many parameters GPT-4o has. It might start with a Google search, finding an article that says that GPT-4 – an older, larger model with a similar name — has 1 trillion parameters. Using this article as its context, it might then mistakenly state that GPT-4o has 1 trillion parameters. Existing AI assistants often provide source links, but users would have to tediously review the article themselves to spot any mistakes. ContextCite can help directly find the specific sentence that a model used, making it easier to verify claims and detect mistakes.”
When a user queries a model, ContextCite highlights the specific sources from the external context that the AI relied upon for that answer. If the AI generates an inaccurate fact, users can trace the error back to its original source and understand the model’s reasoning. If the AI hallucinates an answer, ContextCite can indicate that the information didn’t come from any real source at all. You can imagine a tool like this would be especially valuable in industries that demand high levels of accuracy, such as health care, law, and education.
More details can be found here.
