# Whose Knowledge Belongs in the Context? - Author: Thomy Gölles - Published: 2026-11-06 - Language: en - URL: https://thomygoelles.com/four-questions-context/ - Topics: AI Agents, Microsoft Graph > Part 3 of four questions we ask before anyone builds an agent. An agent is only as good as the context it gets, and every piece of that context comes from somewhere and belongs to someone. At some point we sat down as a team and asked ourselves what our job actually is. Not our job titles, but our job as people who use these tools every day, and as the people who are supposed to help others work better with them. Brutally reduced, it comes down to one thing, and it's the same whether you use Copilot, ChatGPT, Claude or anything else. Make sure the right context gets into the context window. That's where the third question of our [Solvion](https://www.solvion.net/) masterclass comes from, and it's the most technical of the four. ## Why it always comes back to the context window All of these tools put a harness between you and a large language model, and I've written about that layer [after CollabDays](https://thomygoelles.com/collabdays-belgium-2026/) and in [Cogito, Ergo Build](https://thomygoelles.com/cogito-ergo-build/). The short version is that the model behind it is stateless. It knows nothing about you, your company or yesterday's conversation, only what's in the context window right now. So if we want to raise the odds of a good answer, and good means correct and useful for us, the job is to get as much good context into that window as we can, so the model has something to work with. That happens in two or three ways. The first is implicit. Copilot does a lot of it for you through Microsoft Graph and its connectors. You write a prompt, and before the model sees it, the system has looked into Teams, SharePoint and your mail around the topic and added what it found, limited to what you're allowed to see. You don't have to do anything for that part. The second way is organisational, and that's where it gets uncomfortable. Do people actually have access to the information they need for their role? Is it stored where it can be found, and is it current? Then there are the systems outside Microsoft 365, the ERP, the CRM, whether that's SAP, Salesforce, a planning tool or something built in-house. Can that information reach the context window at all? And right context means factually right and right from a security point of view, both at once. ![Diagram titled "Every piece of context comes from somewhere and belongs to someone". Three sources on the left, heads of experts, documents in SharePoint, Teams and OneDrive, and systems like ERP, CRM, SAP, Salesforce and in-house tools, each marked "Owner?". They flow into the harness, Copilot or your agent, where Graph and connectors pull in what the user is allowed to see, and from there into the context window, all the stateless model knows. Below: for every source, write down where it comes from and who keeps it current and decides access.](https://thomygoelles.com/img/2026/10/four-questions-3-flow.webp) ## The question reads two ways on purpose "Whose knowledge belongs in the context?" means two things, and "whose" doesn't only mean people. The first reading is about origin. Which head, which document, which system does the knowledge come from? An agent is only as good as the business context it gets, and that context sits with experts, in documents and in systems. Armin Einfalt's example in the session was a Teams folder with a file called something like "Offer final final v2 2019", and an agent that happily answers from it, because that's the file that best matches the prompt. As Armin put it, an agent doesn't lie, it's just honestly wrong. Lying needs intent, and the model only takes the best match it was given. The second reading is about ownership. Who does that knowledge belong to, and who is allowed to release it? Who keeps it current, who decides on access, and who is liable when it's wrong? For system knowledge the answer is the owner of the system, and not the person who happens to build the agent that reads from it. That's the point people tend to skip, because the builder is usually the one in the room. If you only answer the first reading, you build agents that know a lot and belong to no one. ## For company numbers, the answer already exists For one kind of knowledge, both readings often have an answer already, and it isn't the person building the agent. In October Jens Kröhnert from [ORAYLIS](https://www.oraylis.com/) and I built a whole talk around this, [AI happens twice](https://thomygoelles.com/ai-happens-twice/). Jens calls the two sides frontend AI and backend AI. Frontend AI is Copilot and the agents people build in the business. Backend AI is the data platform where IT has pulled the important numbers out of the operational silos into one version of the truth, with definitions everyone has agreed on and governance around them. ![Jens Kröhnert and Thomas Gölles on stage in a purple-lit event hall during the demo, with the agents available in Microsoft 365 Copilot on the big screen, the moment the demo switches from the Fabric data agent to Copilot.](https://thomygoelles.com/img/2026/10/four-questions-3-stage-fabric-demo.webp#center) Seen through this question, that data platform is origin and ownership answered once, centrally, for everyone. The KPI comes from there, and the people who run the platform own its definition. The trouble starts when an agent skips that layer. With the right permissions I could build an agent that goes straight down to the source systems in about an hour, and a colleague would rightly tell me it's my solution with my interpretation of the data. Five people doing that means five definitions of the same number, and none of them belongs to anyone. The pattern we showed is the opposite. A Fabric data agent sits on top of the purchasing data and answers in plain language, and an agent in Microsoft 365 Copilot taps into it instead of working out the top suppliers on its own. The business user gets the answer, and the definition stays with the people who own it. When the agent needs to write something back, an MCP server does it under the user's identity, so the change is traceable as if they had made it in the system themselves. ## What this means if you build agents For anyone building agents, whether that's a declarative agent, something in Copilot Studio or a custom engine agent, I'd turn both readings into a short list before the first knowledge source gets connected. For every source, write down where the knowledge comes from and who owns it, meaning the person who keeps it current and the person who decides who may see it. If you can't fill in the owner, you've found the problem before your users find it for you. It also changes who you invite to the first meeting. If the agent reads from the CRM, the CRM owner belongs at the table, because they're the one who'll get the call when the agent says something wrong about a customer. Which leads straight into the last question of the series, who carries an agent once people start relying on it.