Before opening SCOUT to all faculty and staff, we invited a smaller group of colleagues to put it to work during an Early Access preview.
To everyone who participated, thank you! Their participation brought us real questions and real work, found rough edges, challenged answers, and shared feedback will help us better understand where SCOUT provides value and where we still have work to do.
People brought SCOUT real university work.
Early Access participants didn't use elaborate prompts or artificial test cases. They brought questions and tasks that were already part of their work; often starting with a simple question and continuing the conversation from there.
Find & Navigate
Finding policies, forms, services, people, responsibilities, and information without already knowing where UC Merced keeps it.
Work Through Processes
Making sense of university processes; especially the ones people encounter only occasionally and may not remember how to complete.
Bring Information Together
Connecting information spread across university websites, documentation, policies, services, and other resources into something easier to understand.
Create & Refine
Summarizing information, drafting communications, revising content, helping with digital accessibility, and turning what people asked about into something they could use next.
One of the clearest lessons: you don't need to learn how to write the “perfect prompt.” Ask naturally and tell SCOUT what you're trying to accomplish. Then, most importantly, continue to engage with SCOUT once it gives its initial answer; give it further clarity to help refine its work/actions.
The best answers helped people move work forward.
Finding an answer was just the beginning. Early Access members followed up, asked for clarification, applied information to their situation, and used SCOUT to move from “where do I start?” toward something they could act on.
“It helped me answer a student question that I would have normally been challenged to answer in such detail.”
A good-looking answer isn't always the right answer.
Early Access also helped us see several different ways an answer can fall short. Sometimes the problem was SCOUT. Sometimes it was the information SCOUT found. And sometimes SCOUT only scratched the surface without getting far enough to resolve what the person was actually trying to do.
SCOUT can get it wrong.
Like other AI assistants, SCOUT can misunderstand a question, miss important context, or produce an incorrect answer. Important information should always be verified before you act on it.
The source can be wrong, too.
In some cases, SCOUT found the right UC Merced source and accurately reflected what it said; however, the source itself was outdated, incomplete, inconsistent, or incorrect.
Relevant isn't always enough.
A source can be related to the question without being the most authoritative, current, or complete source. We saw cases where SCOUT got close to the answer without fully resolving what someone needed to accomplish.
Capabilities need to be clearer.
As SCOUT gains access to new knowledge and university systems, people reasonably expect it to know what it can use and when. We're continuing to make those boundaries clearer as its capabilities grow.
SCOUT is helping us learn about UC Merced, too.
UC Merced's knowledge is spread across websites, policies, documentation, services, and systems maintained throughout the university. Bringing that information together can make it easier to use, but it also makes gaps and inconsistencies easier to see.
When SCOUT returns outdated information, conflicting guidance, a dead link, or a process that is difficult to follow, the solution won't always be to change the AI. The better fix is to improve the university information behind the answer; this not only helps SCOUT be more effective, but also those discovering information on their own or using other tools. We are investigating a way to provide feedback to website owners more directly to help with this problem. In the meantime, the best way to report bad information is to use the website’s “Contact Us” information or use the “Site Feedback” link located on the bottom of every UC Merced website.
Feedback helps us improve SCOUT and improve the knowledge SCOUT depends on. Both make the experience better.
Early Access was a beginning, not a finish line.What we learned is already shaping what comes next for SCOUT. As more faculty and staff use it, we'll continue paying attention to what people bring to SCOUT, where it helps, and where it gets in the way. That may mean making improvements to SCOUT, connecting SCOUT to more university knowledge and systems, and/or improving the information behind the answer.
We learned a great deal from a relatively small group of Early Access participants. As SCOUT reaches more of UC Merced, we expect to keep learning and we hope you’ll keep telling us what works, what doesn’t, and what we haven’t thought of yet.
