While it's clear that chat-based UI's are currently the most popular way to interact with LLMs, I'm always curious about what other more customized experiences might make sense for certain interaction patterns.
For DevTools, a common sub-task is needing to manage which files should be in context for a given code-related task. Besides obviously needing to give an LLM the right set of files to consider or modify, you also want to remove as many irrelevant files from its context as you can, because LLM effectiveness degrades as you give it more and more stuff to consider.
A lot of AI IDE's like Cursor already have customized UI's to handle file management, but they always felt clunky to me and shoe-horned into existing code editor UI's. This post is about an alternative approach that I came up with: it borrows ideas from Git's DX (related to staging files) and applies them to LLM file context management, in a way that at least I think feels more natural and even reduces a certain amount of anxiety :)
I'll first go over some ideas about modal viewing vs editing and some background info, but if you're interested in skipping straight to the meat, go down to the Modal Editing section below. There's also a brief reflection on how far you can extend the Git-based analogy, some lessons learned, and a YouTube demo video at the end.
Modal Viewing vs Editing
One concept I've always been fascinated with is that of modality, which comes up often in surprisingly different places (cough cough, resist the academic urge to mention Modal Logic).
For example, if you've ever used Vim, you're familiar with the normal mode for viewing/navigating text, versus the insert mode for editing/writing text. Within each mode, different keyboard shortcuts that are most appropriate to navigation or editing become available.
In web apps, we're all familiar with the concept of modal dialogs, which usually overlay the primary page content with some more specific info and UI.
When it comes to LLM file context management, I like modal dialogs as a way to separate viewing files in context, from editing or managing the files that should be in context.
Where is this UI/UX pattern relevant?
Before getting into things further, let me set a little context (no pun intended) for where these UI/UX concepts came from and where else they are relevant. Colimit is a GitHub app that helps you resolve failed CI builds, so the screenshots below are from the primary screen that:
- Displays the most significant errors, isolated from the rest of the noise in the build logs
- Shows the list of the most relevant files for debugging the error that Colimit discovered
- Shows you a root cause analysis report, as well as a fix that you can apply
Even though I'm giving examples of nice DX for file context management in Colimit's CI analysis setting, the same UI/UX concepts are applicable to other devtools like Cursor, Copilot, etc, where you have an LLM that must be aware of a subset of files in your repo.
Modal Viewing to only Jump to Files
Now let's go back to discussing the modal viewing experience:
When we are viewing the files relevant to some development task, what we really want to do is clearly see them at a glance, and then jump to them to see the code when we need to. A spread-out grid layout with file links, highlighting filenames but still showing muted file paths, supports these goals.
But when it comes to design, what's not there is just as important as what is there. If you look closely again above, what you won't find is little x buttons to remove files from context.
Why is that? A lot of work goes into searching through the codebase to figure out which files are relevant for a dev task, and the last thing we want to do is accidentally click a subtle delete button when we're merely trying to view a file.
Instead, we decided to make it an intentional decision to click Manage Files if you want to swap modalities to go from viewing/consuming effort to editing/producing effort.
Modal Editing of Files in Context, inspired by Git Staging
We've finally arrived at the meat of the matter, the modal editing experience:
Because getting the context right is so important, I wanted to encourage this mode by making it feel fun, interactive, and non-threatening.
On the left, we have a fuzzy filtered search to help you find files you want to add, and on the right you can see which files are currently selected to be in context.
Checking a file that was previously missing from the context will mark it with a green plus (+), while unchecking a file that was previously in context will mark it with a red minus (-). You can also see a Git-like summary of the additions and removals next to Selected Files. For anyone that's interacted with staging and unstaging files in git to prepare a commit, this should feel very at home.
Accidentally unchecked a file that took a while to think of and put into context? No problem, it's only been visually marked rather than removed, so you can recheck it easily. Got things in a weird state and want to start over? Just click Reset.
Importantly, you can also close the modal and go back to looking at the root cause analysis report, in order to cross-reference files or symbols mentioned in the potential causes or fix options. Then, you can jump back and forth between the file manager and the analysis report, searching for useful files and unchecking red-herrings, until you are finally ready to "commit" via Save.
How Far does the Git Analogy Stretch?
When managing files in context via the Git-inspired UI, when you click Save you're really just altering the files in context for AI-based CI analysis, rather than creating a commit or changing file contents in your repo. However, it remains to be seen how far the Git analogy should be stretched:
- For example, should we create points in history for each file context change, so you can view past contexts or revert to a previous contextual state?
- Should this whole process actually be backed by Git, where we commit some representation of the files relevant to fixing a bug into the codebase somewhere?
- Should we really be saving "workspaces" that consist of collections of relevant files for performing certain LLM-assisted tasks?
I'm not sure yet. What I am sure about is that this UI/UX pattern has increased interactivity within Colimit, encouraging updating the files in context to produce better analyses and fixes. One thing I'm proud of is that this has a bit of that "video game" effect by being kinda fun to mess around with. Maybe less full-fledged video game, and more Hearthstone interactive tavern hah.
Lessons Learned
It's obvious that having good context is important for AI to produce effective fixes. But, encouraging human-in-the-loop file context tweaks had another fortuitous consequence:
The ultimate files in context act as a historical artifact that you can reference in the future, when trying to remember what you needed to know to fix a particular bug! This is an interesting piece of information, because it's not captured alone by Git diffs / Git history. Git will tell you what was actually changed, but not what related files needed to be understood to come up with the changes (if you're using Colimit, you can always go back to view previous build failure reports to access that valuable contextual info.)
Demo Video
That's it for now, and if you're curious about what this looks like in action, here's a demo video: