Interpretable context methodology.
That's the name of my paper that I published about three months ago, and now thousands of people and hundreds of GitHub stars as well over 30,000
people in the community are all using my methodology to build their own systems with AI.
But they're not building multi agentic frameworks and all these absurdities.
They're building folders and markdown files on their computer and getting
huge results from it. Now I have quite a few videos on that, but today, I wanted to talk less about the methodology and more about what I've been building, what me and my team have been building as a future thing. What can you actually build using this methodology, and what happens when you bring in some traditional software programming into it? Now for those of you who are new to my channel, essentially, my methodology is about structuring
folders, structuring markdown files. It's very similar to the LLM Wiki that Kaparthi, the founder of OpenAI
and, uh, currently just moved to Anthropic to teach there, uses and is very big in Anthropic's actual use cases. They are very big around skills, which is the idea that the right amount of scripts,
processes, and ideas
can all be kind of broken down into just plain text and folders, and the agent can navigate that to get the context.
Rather than using really complicated workflows or infrastructures and harnesses, you can create one really good harness for a single type of model, and then that model can then navigate structure
as it needs to. Instead of creating rag that pulls everything in, you just give it access to normal databases. You give it access to folder and context structure. If you haven't read anything or dive into this process, I do a deep one with Claude code in my video right here that you should see tagged.
Essentially, it just looks into how to use this, how I use it, and a lot of people are seeing twenty, thirty, 40% token reduction and really just getting outcomes a lot faster without a lot of infrastructure.
Even further, you're able to use it to really build out teams and build out processes
that traditionally would have taken, you know, a vendor six months and an entire infrastructure update versus just working with them. And as the people learn how to use the AI, they're building the structure for you. Your team become the vendor,
and then you can hire technical talent or bring in your own technical talent to amplify it, look at the security and governance behind it. But all of that aside, let me chat about transcripts,
about dialogue,
about content,
and why it's so important to this entire process.
Now this is an actual slide that I gave to a real enterprise company that's bringing in, I think, 60,000,000 a year or something like that, and they have learned and used this process.
One of the biggest things that I try to share with people when we're trying to understand how to use AI, how to use these tools in your workflow
is my layer process. Right? Layer one
is where you're just chatting with a chatbot. You're copy and pasting.
The layer one is this concept where I started years ago when I first started messing around with language processing systems, and a lot of people are today. It's where you log in to ChatGPT
or Claude,
and all you're doing really is just kind of pasting in ideas. Right? You're just chatting back and forth in it. You're copying and pasting.
You're throwing in concepts in there.
You're just asking it basically questions and things like that. What you actually wanna really understand is the amount of effort to get something back
is low,
and you can get some pretty decent things.
But the amount of output and impact that you get is actually very weak.
Right? It's not gonna be huge. You can get some very good stuff.
But earlier, three years ago, it was very challenging to get any sort of outputs. And in order to really get a better output out of it, you have to longer and longer conversations.
As in, you're essentially adding multiple chats. You have saved prompts.
Maybe you have a tone style. And we see a lot of people sharing prompt libraries.
They have shared prompts. Maybe they're adding the prompts in little by little. A lot of other, like, kind of integration tools do this where they're grabbing a prompt and giving it to the AI automatically.
It's doing chain of loop processing.
And these skills are actually the layer two. Someone else went through that layer two workflow, found the right amount of prompts that you actually wanna be able to give, in what order you wanna give them,
and kind of handled that as a skill. But then skills evolve even further, and you realize you only need certain parts of a skill at a certain time,
you don't wanna overload the context window, and you wanna start creating automations.
That's where you start leaning into level three or layer three,
which is where you're actually doing this huge workflow.
You're taking multiple skills, multiple prompts, multiple workflow processes, maybe even multiple AI,
and actually linking them together. Then they realized that you want to do even more steps. You want these agents, And that's where you have these other markdown files or scripts in many areas. Right? People actually put Python
for people to actually run, for the AI to actually run itself
to be able to handle the creation of stuff because they needed more determinism.
They wanted the output to be the same every time. And so if they could package all of that into a prompt,
all of this could be uploaded
and sent in the right order,
that's where skills come from. And this allows them to then, in this case, create more skills, but you have hundreds of skills available. And if you have Claude installed on your computer,
you can actually come through and look through these skills deeply. Some of these skills, right, for algorithmic art, a skill for creating art with, like, kind of an algorithmic process,
has an HTML viewer and a template built into it. Maybe you're trying to create a doc file in Markdown.
You have the actual skill explanation.
This is how you do it. These are the prompts they put in order. Traditionally, two years ago, I was putting these prompts in manually. First, I would I knew that, okay. If I told the chatbot
this in a certain order, right, I come in and I paste this here,
right, it would do that process. I would wait for its output, verify that it's right, and then I would move on to the next step, which is, okay. Well, then we need to edit that and have these things. And over time, we realized we could drop that into the AI all at once, and it would recognize that. But then we could drop all of it into the AI at once and tell it to reference certain areas or actually use commands,
and the single agent could then do those things. Right? Now, traditionally,
before the way they solve that is using lang chain or semantic kernel or all of these things to do these actions separately or to inject these
at the right time
in the thinking process of the AI.
But now we can just break down the thinking process into folders and structures
and trigger them as they're needed,
which is what these folder skills and processes
are doing. But, again, a common theme. There's a reason I'm describing all of this. All of these skills,
all of these folders and markdown files,
all have one core theme,
discussion
and dialogue.
They all came
from an initial layer one of discussion
and dialogue breakdown. At layer one, you're pasting a prompt into ChatGPT and then copying the output or going through iterations of an output. At level two, you already expect what the output is from level one, so you have refined prompts and processes
that are automatically being injected in. At level three, you're doing all of the level two automated multiple times so that you can then come in and look at these processes. And that goes for code review, data reporting, writing and drafting,
all of those. That's what we really want to understand here. But it all comes from dialogue.
Conversation
has the structure we're looking for. The intent is carried in the conversation. And instead of just describing this randomly,
here's Atul Kay, who is one of my, uh, cofounders and a really brilliant person, certainly studying at the University of Edinburgh, created to really help exemplify this process.
So imagine here on the left. This is the chat, and we're gonna break this down to the simplest process.
This is a chat that may have happened forever ago. And the idea is, can you tighten this paragraph? It feels too worry. Climate change, blah blah blah, and you put in the paragraph in there. Now in the past,
right, the agent would just say, here's a new one. And then we would say, hey. This feels weirdly formal now. The original had more of a conversational rhythm. And so the agent does something, and then it's this constant back and forth until we finally get the output we wanted out of it. Now everywhere along this conversation, even in this simple one, there is a chain of decisions being made. And if we actually extract those chain of decisions using Kaye's tool here, here on the left is the decisions made by the user.
Here on the right are the decisions made by the model.
Now when we look in, the goal
was to tighten a paragraph.
All of this is all of the goals and focus. And as you can see, it links directly here. Can you tighten this paragraph?
It feels too worry.
Now we set up constraints along the process and at different points. Perhaps there was a constraint saying, hey. There was too much wordiness.
We want to perceive the rhythm or conserve what the rhythm in the original paragraph was. Maybe we wanna have it more styled or evolved. Right here, we want it scarier, way behind. The voice we're going for is more measured,
right, in that same sentence.
Maybe we wanna match the entire voice reference. The assumption
is that we need it to match the original,
and both the model and the AI follow this assumption. And don't worry. We're gonna go over to this right one in a second, but I really want you to understand this. Finally, the goal of what we're making, the process,
is an actual blog post or a reference, and we give an example
of what that should be. There's also dialogue happening. Right? Questions that are happening. I think there's a way in which to do these things. When we go and look at the model over here, the same thing. The model sees the goal. It sees the constraints, but it is making decisions
on those constraints. It's coming through and saying, hey. This retains a cleaner structure. I made this decision for this goal. I made this decision for this goal. And it's also making assumptions
along the whole way.
We create all of this data. We look at this, and instead what we can do
is we can create a system of tone, of style, of ideas
that capture all of these problems and constraints already.
So that when I say tighten this paragraph,
this tree of processes,
this tree of prompts is called the first time. Again, thinking about where we're at, originally,
it was layer two. We had to sit there and describe these processes, verified outputs. We captured those, turned those into maybe markdown files or structure,
and then the original request becomes a layer one process
where I'm able to simply say, can you tighten this paragraph?
Again,
this also is the same thing for everything.
Let's say we're doing the opposite side. We're actually refactoring code. And I actually say, hey. My parthon my my Python is treating code a certain way. I noticed that this parser, can you kind of take a look and fix it possibly?
And, of course, the instant assistant comes in, says what it thinks its fix is. Maybe we say some sort of reference, because we're an expert in the field, maybe. This is how they did training early on is this kind of back and forth, and you eventually get to the conclusion, the goal. When we actually extract that data, right, you end up getting the same thing. You have goals,
then you maybe even have subgoals.
Right? Can we just patch the empty cell instead and keep the original function?
You have constraints
within this dialogue,
and you have processes
that live inside of it as well. Same for the model. The model now has the goal. It's coming through, and it's making decisions
on that goal, and all of it can be tracked through
dialogue. That goes for anything. Database schemas, you can track the decision making and goal processes,
where and when it was saved within dialogue.
All of this becomes either training data or even better,
markdown files that can be used later, converted into a methodology.
Right? And it's important to understand this,
not necessarily that you could use this tool. We might make this tool open source. Right now, it's being used for a research project at the University of Edinburgh.
Um, but the idea here isn't for you to all use this tool right now. The idea is what this tool shows about dialogue,
about prompting,
and about engineering
context,
where and what you want to engineer context, at what levels
you do, and that's what has informed
a lot of the work I'm doing. And an example is one here.
My dialogue with the AI made me realize it sucks at certain processes of reading and research.
So I created a kind of pillar of idea and my voice and my patterns and constraints that capture my process of writing and researching
as well as my tone. No matter what project I'm working on, I use these processes.
So that's all just captured in a very simple short prompt.
Right?
That is referenced whenever I need it through the initial prompt, which is my Claude MD. That gives folder structure of wherever I'm hiding. And wherever that voice is,
I say, hey. Every session,
I'm going to have certain things built in. My voice and TomeMD directly says how I speak and teach. And just by me simply saying, right, without creating any crazy agentic architecture,
what is my voice
and tone?
Now if I had asked an AI model that was at layer one or two years ago, it would have no idea. I'd have to paste a whole bunch of prompts in there, set them all in. If I was one at layer two, it would have to read the whole database,
go through everything, finally find the structure. At layer three, I've solved the problem of it navigating. So it can easily come in, find my voice and tone markdown file,
actually look through and see how I teach in tone with one simple prompt. This prompt is actually thousands of prompts, which if any of you have watched my Python video, 12,000 lines of Python,
it's the same thing. When I type hello world, it is thousands and thousands of lines of code that are executing to print
hello world in Python.
If you haven't seen that, I highly recommend it. It's very useful for this kind of conceptual.
This same thing is happening now with these processes. This is the next layer of software engineering.
But with every layer comes requirements,
comes processes,
and that is exactly what I'm encoding here. These processes keep expanding until I finally have an output.
In this case, my output was the ability to not only write and research in my own area. So I had a company, NLP Logics, that wanted me to create videos for them. I spent time building ideas and my tone around it, and I said, I wanna make three videos based on three of their products. So it goes through. It has access to my research and creates a research folder
on each of the products, the angle,
where they are, what type of hooks might work, what type of outlines, and materials. And, again, it's all markdown files. These are just more prompts to be added to context in the future,
whether it's this AI or any other AI that I'm using to read these folders. And then all I have to say is, cool. Now do the research based off of all of that. Then with a third prompt, again, I'm only technically
said two prompts so far in this whole process of getting all of this research,
all of this data, all of this possibility,
and it was able to create me these scans, these source materials
to then move on to the next step, in which I say, cool. Let's make a script for each. Now before,
if I had said, let's make a script, it would try to assume what a script should be. I have pillars and processes on how scripts should be written, where, and why. Think of these as my own personal skills.
So it goes through. It reads my voice.
Right? It automatically knows my voice constraints. It does the actual processing. It knows where the documents need to go. Again,
with no crazy injection harnesses. Yes. The Claude code agent has harnesses in its own right, but that's the point. I don't need to reinvent the wheel. It's already been invented.
Then, boom, I get scripts. And guess what? I say, can you generate the audio for those scripts?
I already had a workflow that pulls from my 11 labs on my voice that's already been trained,
pulls all that data, and says, cool. We're gonna go ahead and produce that audio. Then I have the workflow,
which I at this point, I can open up a fresh conversation if I wanna save tokens. All of the process has been saved. All of the context is in the structure.
I don't need to inject it back in at the right time. It can pick right up off.
And I can then say, hey. Let's make a video animation. And rather than show you the code, let me just share with you the animation.
It took my voice,
and it took my placeholder
files that I had already created for animations for this entire project and simply built from them rather than generating new ones. And it took my voice, chopped it up, and allowed me to create what you see here. Insurance company that emails provider networks
to ask if a provider is still in network.
Right? I didn't need to go through and edit. I might refine it. But the best part is this isn't random video generation. This is structured output that I can then edit at scale. I can edit with my personal touches. I can come in and find all of those little pieces of data. Again, this entire workflow
is probably four or five startups in the startup world right now, and it's all in folders and markdown files with one agent, one singular agent.
Now this is where it gets interesting.
Let's say everyone starts doing this. I already have, like, 30,000 people in the community building their own ICMs.
How do you share it? And more importantly,
what happens when we go back to dialogue?
What happens when we think about dialogue being the most important part? What if I could sit inside of a group call and control
someone else's Claude code or AI through my voice and immediately
access all of that data that's locally on their computer. What would that look like? Well,
we built that, and I'm gonna show you exactly what that looks like. Over the next few minutes, you're gonna see an actual recording from a live call that me and my team did built around this idea. This is going to be myself, David McDermott, and Kay Kumar working on a project that Kay specifically has been building for a little while now around this concept.
And it's not perfect,
but it gives you an idea of where the future. Take it with a grain of salt, and when the clips are done, I'll come back and I have more to explain. Alright. This is test one of one. We've totally never tested this before. We've hooked up my local Claude code. It is currently running locally on my computer, obviously, through my subscription account. I opened it up inside of, um, it's running as a CLI inside of audit engine folder setup here, which is a big old database I used for my psychometric research
at the University of Edinburgh. We're gonna see how it works.
Kaye, can you go ahead and ask my agent
if it's ready to work?
Agent, are you ready to work? Yes. I'm ready to work. I have access to the ethics engine code base, a web application for psychometric assessment of large language models. Just let me know what you need help with, and I'll get started.
Uh, agent, can you describe what the workspace is in less than 50 words? Ethics engine tests AI models using psychology questionnaires.
It sends questions to different LLMs like GPT and Claude, applies different persona framings, parses their responses into numerical scores,
and generates comparative results. It also supports visual assessments using images like Rorschach inkblots.
Very good. Kaye, could you control my agent with your voice through this meeting
to read through the scales
and then add a new one that it believes should be added? Oh, can you explain me what scales exist in the repo right now? The repo has 10 built in psychometric scales. One, one,
RWA,
right wing authoritarianism.
Can you
looking at the scales, what scales are we missing from the psychometric analysis engine?
Agent?
Looking at what's currently implemented versus common psychometric instruments used in AI ethics research,
here are some notable scales that could be added, missing scales that would be relevant. One dark triad measures, Machiavellianism,
narcissism, psychopathy.
Two empathy scales like the interpersonal reactivity index. Three, moral disengagement scale. Four, honesty, humility from Hexaco personality model. Five, cognitive reflection test for reasoning biases. Six, Schwartz value survey for basic human value. Seven, discuss sensitivity
scale. Eight. Okay. That sounds good. Which one Nine. Which of those is your top pick, though? Just pick one.
My top pick would be the dark triad. It measures Machiavellianism,
narcissism, and psychopathy in one instrument. Okay. That sounds like a good addition.
Can you add that to the psychometric analysis of the engine?
I'll add the short dark triad scale to the engine now. Let me first check the existing scale structure to match the format. I see the scale structure. Now I'll add the short dark triad scale. This is the SD three by Jones and Paul Hus.
Okay.
Good job. Uh, can you go into more detail about the documentation that you changed?
Sure. I updated the Claude MD file in two places. First, in the built in data section, I changed the scales table from 10 validated instruments to 12 validated instruments and added two new rows. IHS,
the intellectual humility scale with six items on a one to five range, and SD three, the short dark triad with 27 items on a one to five range covering Machiavellianism,
narcissism, and psychopathy.
Second, in the key file section under back end, I updated the description of builtinscales.pwai
from 10 built in validated scales to 12 built in validated scales to reflect the additions. Looking at the front end, is there anything we're missing in the front end that would make this
app better from the perspective of the scale you just added?
Good question. Let me take a look at the front end to see if there's anything that needs update for the new
The component shows the scale name, description, item count, and scale range, all of which we defined. One thing that could be improved for the dark triad specifically,
the front end doesn't currently display subscale information.
The s d three has three subscales, Machiavellianism,
narcissism,
psychopathy,
but users would just see it as one scale with 27 items. They wouldn't know which items match which trait. Would you like me to subscale display support so users can see the breakdown against my trait? That would be useful, not just for s e three, but also for other scales like RWA, LDA, and FQ to have subscales too. Mind you, this was my master's thesis that took me a year to build out.
Yeah. It's doing a pretty good job of understanding
good job. What I'm requesting.
Now we proceeded to play with it for about another hour or two. And I think in total, with the tokens we were using, my custom AI voice that's routing in through API, we're using 11 labs at the time, but I think we're gonna switch over to Voxtril, which is a more open source voice model. Uh, we ended up spending a dollar and 20¢ for about an hour of our time and effort, and I think we could drop that number down exponentially if we used more open source models and whatnot. And, realistically, the voice one is something that can be amplified by a lot of NVIDIA works and things like that. The point is not only are we able to control someone else's computer with their agent and their context, or someone else can control mine,
but we can use that dialogue,
that process of multiple people working together with an AI to generate better dialogue structure. One of the things that we're working on right now is that kind of process of interruption,
because I see an opportunity in the future where this dialogue structure, these structured workflows, all of these things are being triggered by keywords in conversations.
When we have a meeting, we sit there and dive in and talk together, and I think that would be absolutely amazing to see these come and flow while we're in a meeting, when we capture transcripts,
when we capture meetings. Instead of them being plans,
they can be actions before the meeting is even done, and we can even hear about those actions directly from these agents. Again, this is more of an open source tool. It's part of a larger project that I'm putting together because, again, I think this is one solution
to a larger problem that needs more solutions added in. And you'll see as I'm posting videos the kind of structure I'm building for it. But other than that, I hope this video kind of taught you a lot today.
And if you are watching this as just a demo video for Vox, which I'm cutting this video up, then please go check out my larger I think it's gonna probably be thirty or forty minute video where I describe the content and the structure behind
dialogue,
behind my interpretable
context methodology.
If you are just watching this because this is the Vox video and this is the cut I did, well, then I hope you enjoyed it. I hope you check out more things. Other than that, thank you everyone for staying to the end. I hope it was interesting to watch. I hope you learned a lot. And until next time, happy learning.