TL;DR: We expected sending the full prompt to win every time. It did not. On our controlled benchmark, Trimwise Hybrid at 512 tokens matched the full-prompt answer-pass baseline on GPT-5.4 Mini and GPT-5.6 Luna, while beating it slightly on Nano. Trimwise Lexical got very close too, but at roughly 8 ms instead of Hybrid’s ~52 ms. The bigger finding was not just score: token-level compressors can leave behind broken Markdown, clipped identifiers, damaged JSON, and context fragments that look relevant but are no longer safe to use. Smaller context is useful only when it is still intact, traceable, and actually usable.
At Tenwrite, we are hard at work writing an internal agent that manages our SEO footprint
by creating and managing content. Naturally, for such a system to be able to learn constantly and stay updated with changing
SEO trends, it will have a lot of blogs to analyze. Now, we just cannot shove full blog content into every prompt. It works
for smaller blogs but, for larger ones, they result in excessive token usage and longer run times. We initially tried
the cursed first N truncation but the results came back to haunt us so bad we had to revert and think of other solutions.
Our problem was simple - too many huge blogs to process so, condense them into a token budget each with minimal reduction of information density to hurt performance. The usual methods that we analyzed were LLMLingua, LongLLMLingua and RECOMP. Although their main use-case is context compression, they still allow query-aware compression. But a few trials revealed a deeper problem - these methods remove a lot of tokens which affect information density quite significantly. This was especially visible when the source texts had institutional knowledge the models had not trained on.
So, we did what any team would do after seeing this mess - we went digging.
The first thing we realized was that “context compression” is a very overloaded phrase. In one place, it means “make this prompt shorter”. In another place, it means “extract the useful passages”. In another place, it effectively means “delete tokens until the model is okay with the input length”. These are very different things, even though all of them get marketed under roughly the same name.
For our case, we did not want a summary. A summary is useful when a human asks, “what is this blog about?”. But our agent may ask something far more specific later: what was the old recommendation for canonical tags in this particular article? What example did the author give for internal linking? Was a caveat mentioned near the end? Did the blog say something that contradicts a newer internal document? These are not summary questions. These are retrieval and evidence questions.
And this is where the usual “compress aggressively and hope” approach becomes dangerous.
A lot of blog content is repetitive. Great, remove repetition. A lot of it is filler. Great, remove filler. But the same blog can have one weird paragraph containing a piece of institutional knowledge, a client-specific exception, an experiment result, or an old SEO decision that no foundation model has ever seen. Remove that one paragraph and the output can still look very clean, very short, and very wrong. This is the annoying part. The failure does not always look like a failure. Sometimes the context is still grammatically fine. The agent simply loses the only thing that mattered.
We initially tried the usual family of solutions: LLMLingua, LongLLMLingua, RECOMP, and a few simpler baselines. We also had the classic first N truncation baseline because, well, everyone has to make that mistake at least once before moving on with their life.
first N is not ideal for obvious reasons. It works suspiciously well when the answer happens to be near the top, which makes you think your benchmark is fine. Then the useful thing is near the end, or split between the beginning and the middle, and suddenly your agent starts confidently answering a question with half the evidence missing. It is not really a compression strategy. It is a positional assumption pretending to be one.
The existing compressors were more interesting, but they came with a different class of problem.
They can be very aggressive at token removal. On normal prose, that can look okay at first. But on real working material - headings, Markdown, lists, code snippets, JSON, identifiers, links, technical instructions, weird internal terminology - token-level removal can damage the actual shape of the source. We saw LLMLingua-family outputs where headings became glued fragments, identifiers got clipped, Markdown fences got damaged, and JSON started looking like punctuation soup. Not “the model missed a sentence” bad. More like “the context is now technically present but no longer a thing you can safely hand to another system” bad.
For example, this kind of output is not useful provenance:
### 3aching reduces and. latency##
Neither is this:
`{-3 "_ ],
This matters a lot more than a generic question-answer benchmark will tell you. If your context is only prose, you may get away with it. If it contains rules, code, configuration, exact claims, links, citations, or source material that needs to be quoted back later, you really cannot.
So we made a list of the things we actually cared about.
We wanted query-aware compression because the agent normally knows what it is trying to do. We wanted a hard output budget because “roughly shorter” is not a useful systems contract. We wanted the retained text to stay source-backed instead of being quietly rewritten. We wanted the output to preserve source order. We wanted omissions to be visible instead of pretending two distant paragraphs were originally adjacent. And, because this was going into an agent pipeline, we wanted to know exactly where every retained piece came from.
That last one was important enough that it shaped the API.
Trimwise returns source spans for the retained content. These are Python-string offsets into the original input: inclusive start, exclusive end. If Trimwise keeps two regions from a long blog, the caller gets two source spans in source order. If Trimwise inserts an omission marker between them, that marker is intentionally not part of either source span.
Why do we care? Because now we can take a trimmed excerpt and still map it back to the original blog, the original document section, or the original repository file. We can keep paths and line references accurate. We can cite the actual source instead of citing a synthetic compressed blob. And if we want to split a retained block further during final prompt assembly, we can do that without losing provenance.
That is the basic design of Trimwise: do the ranking work, but keep the source relationship intact.
Under the hood, the library starts by segmenting the input into structural units. It tries to respect real boundaries: headings, paragraphs, lists, fenced code blocks, source lines, and smaller fallback units when needed. Then it can rank those pieces in a few different ways.
There is a structural mode when there is no useful query. There is a lexical mode for query-aware retrieval using term relevance. There is a semantic mode when the caller provides embeddings. And there is a hybrid mode which combines lexical and semantic signals. We deliberately made embeddings caller-owned. We did not want the library to quietly decide which embedding model gets downloaded, loaded, cached, billed, or trusted inside someone else’s application. If you already have an embedding stack, use it. If you want a small local model, use that. If you want a stronger remote one, that is your decision too.
The output is still source text. The ranking passages can have extra context to help the scorer understand where a paragraph sits in a document, but the final composition only uses the original slices. This sounds small, but it avoids a very weird class of bug where a ranking helper accidentally leaks enriched context into the final prompt.
Then, naturally, we benchmarked it. And then naturally, the benchmark became its own project.
At first, we had a dataset that looked reasonable on paper and was absolutely not reasonable once we inspected where the evidence lived. Most answers were near the beginning. This made first N look much better than it deserved to look. The benchmark was basically rewarding a method for preserving the part of the document we had accidentally made most important. That is not a result. That is us measuring our own dataset bias.
So we fixed it.
We built a position-controlled set of 160 cases:
The cases cover answerable content, instructions, procedures, structured/code-heavy material, adversarial material, and source-backed real content. We kept these tracks separate because one score cannot honestly describe all of them.
A short answer question should be measured differently from “did the required instruction survive?”, which should be measured differently from “did these ordered procedure steps remain in order?”, which should be measured differently from “is this JSON/code/config block still exact?”. Trying to force all of those through one fuzzy “quality” score is how benchmark dashboards become very pretty and very useless.
We tested 128-token and 512-token output budgets. We compared Trimwise Lexical and Trimwise Hybrid against LLMLingua, LongLLMLingua, and RECOMP. We then took the full contexts and compressed contexts and ran them through three downstream evaluators: GPT-5.4 Nano, GPT-5.4 Mini, and GPT-5.6 Luna. The full prompt was kept as a reference line, not as a compressor, because obviously it has no compression latency or output budget to compare fairly.
The first result was pleasantly boring: Trimwise Lexical is fast.
On our controlled set, Lexical was around 8 ms median compression latency. Hybrid was around 52 ms because embedding work is not free. At 128 tokens, both were around the same source-retention level: roughly 50% macro case pass across the different task tracks. At 512 tokens, Hybrid pulled ahead more clearly: about 62.8% macro source pass compared to about 58.4% for Lexical.
So there is a real trade-off here, not magic.
If you need low-latency query-aware compression and your source is mostly normal prose, Lexical is a very serious option. It gets you most of the way there at a fraction of the time. If you can afford the embedding step and care about semantic matching across larger contexts, Hybrid starts earning its cost as the budget grows.
The downstream answer results were the more interesting part.
At 128 tokens, Trimwise Lexical was already very close to the full-prompt answer-pass baseline for all three evaluators. At 512 tokens, Trimwise Hybrid matched the full-prompt baseline on GPT-5.4 Mini and GPT-5.6 Luna, and slightly exceeded it on GPT-5.4 Nano in this controlled setup.
That does not mean compression somehow makes a model universally smarter. It means that for these tasks, the compressed context often removed enough irrelevant material that the smaller context was at least as usable as the full one. This is exactly the type of thing we wanted to know before wiring a compressor into an agent system.
RECOMP was interesting too. It preserves cleaner source pieces than the token-deletion approaches because it is selecting passages rather than aggressively shaving text within them. But in our run it was much slower, around 220 ms, and it did not retain as much usable source material as Trimwise on the controlled set. This is not a dunk on RECOMP. It is trained around particular retrieval/compression objectives, and that matters. A compressor tuned for one kind of QA corpus is not automatically the best fit for long SEO material, mixed Markdown, internal documentation, or repository excerpts.
LLMLingua and LongLLMLingua had another issue beyond score: budget compliance.
At the 128-token target, LLMLingua exceeded the budget on around 70% of cases. LongLLMLingua exceeded it on around 86.9% of cases. At 512 tokens, LLMLingua improved a lot, but LongLLMLingua was still over budget on a large chunk of cases. This makes comparison awkward. If one method gets to use more context than the others, it may get an unfair quality advantage. In our run, it still did not perform well, but the point remains - a token budget should be a contract, not a suggestion.
We also kept LLMLingua2 out of the main head-to-head chart. Not because it is unimportant, but because it is queryless. Our main question was: if every method receives the same source and the same query, which one gives us the best context? Giving LLMLingua2 less information and then presenting it beside query-aware methods would not be fair.
We did run it separately. It performed poorly on the diagnostic set, but that result belongs in a queryless comparison, where it can be compared fairly with structural compression, Selective Context, and positional baselines. Mixing all of those together would make a nice crowded graph and a bad conclusion.
There is another caveat worth saying out loud: the absolute answer-pass baseline was not amazing. Full prompt scored around 41.6% to 44.4% macro answer pass across the three evaluators. That does not automatically mean the models are bad or that the compressor is bad. It means the benchmark is strict, the tasks are varied, and not every source task is naturally reducible to “produce one short answer matching this gold string”. We therefore keep answer metrics separate from instruction survival, procedure ordering, and exact structured-source preservation.
This is also why we are not going to publish a graph saying “Trimwise is 2x better”. That would be silly. The interesting result is more specific:
For query-aware context assembly, Trimwise Lexical is extremely fast and preserves a lot of useful source. Trimwise Hybrid costs more but preserves more useful context at larger budgets. Both preserve source shape and provenance. In our controlled set, they were substantially more usable than the token-level compression baselines we tested, especially when source boundaries mattered.
And source boundaries do matter.
If you are compressing generic English prose before asking a generic question, a slightly damaged sentence may not hurt you. If you are building an agent that works with institutional knowledge, SEO policies, old blog decisions, code, URLs, configuration, or materials that need to be cited accurately later, you need a stronger guarantee than “the output kind of looks relevant”.
You need to know what stayed, what got omitted, what got damaged, and where the retained text came from.
That is the actual reason we built Trimwise.
Not because long prompts are bad. Long prompts are sometimes exactly what you need. But because when context needs to get smaller, we wanted the process to be explicit, query-aware, budgeted, source-backed, and honest about what it removed.
No haunted first N truncation. No clean-looking token graveyards. Just a smaller context that still knows where it came from.
]]>TLDR: I didn’t like Codex usages hitting limits in the middle of a session. Was looking for a way to save tokens and extend my usage. Wanted to target the two places where token usage occurs most - command outputs and state recollection. Other opensource solutions either targeted just one problem or was too heavy and complex for my use-case. Inspired by Distill, CtxSift came into being, extending upon those ideas.
With AI providers introducing stricter usage limits, I often found myself burning through the usage windows quickly. Quite often, they would abruptly end in the middle of some work and I would be left writing it out myself while I waited for the limit to reset.
I use Codex as my primary agent harness and, what I noticed was inconsistent session lengths. Sometimes, a few prompts would blow through the 5 hour window while at other times, I could get more turns out of it before the limit hit. Upon a bit of analysis, I found that the amount of tokens the agent sees and outputs contribute to how fast the usage gets over. The sessions where it had to constantly refresh its recollection and state led to shorter usages while, the ones where it had didn’t need to perform many recollections went on longer. This recollection loop is noticeable especially after context compaction events after which, agents mostly start re-reading files and re-running commands to get back to where they were.
Modern LLMs often do not need to view full command outputs to get what they need for a task or to reason. It became evident that I’d have to control what the agent sees and how much time it spends re-exploring during state recollection. I started looking at what was already available in opensource and came across a few brilliant projects - RTK, context-mode, squeez, LeanCTX and Distill to name a few. They all had good approaches to address the token wastage problem but, I felt that some added more complexity to the agents’ workflow.
I wanted something simple and lightweight - minimal addition to the agents’ workflow, no extra MCP server dependency with multiple tools,
run locally, no complex knowledge graphs and not just heuristics-based compression. I was not looking for a semi or full-fledged memory
system as Codex internally handles its own memory. What I looked for was something to complement this. Distill was the
closest to what I wanted - a command exposed as a skill - the agent runs a command and, pipes its output to distill
along with an instruction of what it wants from the output. Distill uses a language model to compress the output to only
what was asked for and returns it to the agent.
Distill example:
bash frame="none"
$ pytest -q | distill "Return only the failing tests. No explanations."
tests/api/test_users.py::test_create_user_requires_email
tests/jobs/test_reconcile.py::TestReconcile::test_retries_deadlock
Distill worked well for output compression and I could see limits draining slowly…till a context compression event or when I asked something unrelated to the current task flow. As the agent was “distilling” command outputs, its working memory did not store raw details - which is a double-edged sword. On one hand, it keeps the memory small and reduces how much tokens it uses but, on the other hand, the agent now has to re-read files and re-run commands to recollect and get back to a state from where it can continue the given task. It shifted the token tax to recollection events! What I saved during distillation got used during state recollection.
What was remaining was a simple way for the agent to recall its actions and recollect state faster and more efficiently. I didn’t want a complex knowledge management system or a full-fledged memory layer - they can get noisy fast. Instead, I opted for a simple, local, workspace-scoped caching system which the agent could access with just one command - no multiple commands, no chained tool calls - just one command with 3 optional flags. That became by vision for CtxSift - distill commands when necessary, save those distilled results and let the agent recollect them when it needs context.
My goal was to introduce an agent skill that plugs in two simple steps in an agents’ workflow - compress and recall.
I wanted it to be local and have a simple caching system. I started building CtxSift as an extension of Distill’s capabilities
and, after a few iterations, here it is, ready for public use.
In its current state, CtxSift can use local and remote LLMs for compression and works with both CPU and GPU. For caching, it uses SQLite with FTS5 and SQLite-Vec to store records. A hybrid retrieval pipeline with deterministic scoring and filtering ensures fresh, grounded context is made available to the agent. It also maintains record freshness and superseded, older records get marked stale as context updates.
I am really excited to share this with the broader community. Hope it helps you squeeze that extra bit out of your daily sessions. Happy Sifting!
]]>The website traffic is okay - over 1k visitors in a month BUT - the main metrics - installs, active users, time spent on site, CTAs clicked, bounce rate - are sad. I spent a couple of weeks analyzing what is going wrong. And it should have been obvious from the start (though, I chose to build around it).
After a discussion with Rupam and, careful consideration, decided to keep TabMate activities on a pause. Let it exist as is for now but no need to do further development or SEO or marketing work on it. Let Dib return from his exams, then we can see.
Meanwhile, before we start hardening work on TenWrite, I’ll be working on a skill to reduce token usage and extend coding sessions. Yeah, it’ll be opensource. Let’s see how it goes.
]]>Moving from on-prem to cloud changed the tooling, speed, and scaling model, but not the core laws of systems design. Capacity, latency, state, failure handling, and topology still matter just as much. Microservices can help, but only when they solve a real scaling or operational problem instead of just moving complexity around.
A recent conversation with Raymond Oyondi on Peerlist made me rack my memories a bit and reflect on how much software and infrastructure have changed over the years.
I joined the industry back when cloud still felt more like a concept than a default. A lot of systems were still being built and maintained in environments where the infrastructure was very much in your hands. You knew the machines, the network, the limits, the weak points. If something needed scaling, it was not a button click and a dashboard graph. It meant spinning up another server, configuring it, wiring it into the network and load balancer, deploying the application, syncing state, setting up monitoring, and making sure the whole thing did not fall apart under pressure. We had automation in places, of course, but nowhere near the kind of convenience people now take for granted.
A lot has changed since then. But the funny thing is, the biggest lesson for me is that the old principles never really went away.
Cloud changed the speed. It changed the abstractions. It changed how easily we can provision, scale and recover. But it did not change the laws underneath. Capacity still matters. Latency still matters. State still causes pain. Network boundaries still introduce failure. Bad assumptions still come back to collect interest.
That is probably the biggest thing I learned moving from on-prem and bare-metal thinking into cloud-native systems: the tooling changed more than the fundamentals did.
Earlier, a lot of software lived as one big application. One service, one deployment unit, one giant block with hard coupling inside it. It was not always pretty, but it was straightforward in one sense: most of the complexity lived inside the application itself. Since the infrastructure was under our control, nobody really panicked about it. You managed the box, tuned the app, scaled when needed, and kept things moving.
Then cloud became normal, and with it came speed, flexibility, and a different cost model. Suddenly, scaling was easier. You no longer had to treat infrastructure changes like a mini project every single time. But that convenience also exposed something important: a lot of monoliths were expensive in ways people had not fully noticed before.
You would see an application chewing through resources and the default response would be to scale the whole thing. More compute, more memory, more replicas, more money. But when you looked closer, often only certain parts of the application were actually responsible for that load. Maybe one workflow was CPU-heavy. Maybe one module was doing aggressive I/O. Maybe one part had bursty traffic while the rest of the system just sat there minding its own business.
That is where the architectural shift really starts to make sense.
Instead of treating the software like one sealed black box, you begin to see it as a collection of components with different scaling patterns and different operational needs. So you start isolating them. You break out the hot paths. You separate the parts that need to scale from the parts that do not. Pretty soon, what used to be a monolith starts becoming a patchwork of smaller services talking to each other.
And yes, that can absolutely be the right move.
But I also think this is where a lot of people get seduced by architecture diagrams and forget the bill that comes later.
Microservices are not free. They reduce one kind of pain and introduce another. You gain independent scaling, but you also gain more network hops, more deployment surfaces, more observability needs, more operational coordination, more failure modes, and more opportunities for state to become inconsistent. The complexity does not disappear. It just moves.
Earlier, if two parts of the system needed to coordinate, that problem often lived inside one process boundary. Now it may live across services, shared storage, queues, caches, retries, and eventual consistency rules. You may need supporting software to make the architecture work. You may need shared storage. You may need to handle read-write races and stale data. You may need to think much harder about idempotency, ordering, duplicate events, and what “correct” even means in a distributed system.
So for me, the lesson was never “microservices good, monolith bad.” That is too simplistic and honestly a bit lazy.
The real lesson was this: design around the behavior of the system, not around fashionable architecture labels.
If one deployable unit works, keep it one deployable unit. If certain modules clearly have different scaling needs, isolate them. If you are introducing distributed complexity, make sure the benefits are worth the operational cost. Use the minimum supporting software necessary. Every extra moving part is one more thing to monitor, patch, debug, secure and explain at 2 AM.
Another lesson that became much more obvious in the cloud-native world is that deployment topology matters a lot more than many developers initially think. Two services talking to each other on a diagram is easy. The actual topology, where they run, how they communicate, what latency sits between them, how failover behaves, where state lives, and what happens during partial failure, is where reality begins.
I have also come to appreciate observability discipline much more over time. In distributed systems, tracing tools are great and OpenTelemetry has helped a lot, but tooling alone does not save you. If your logs are inconsistent, your labels are exploding in cardinality, your trace attributes are a mess, and every team names the same thing differently, you are not observing a system. You are generating noise. Good observability needs discipline: standard log formats, sensible naming conventions, rules for metrics and labels, and a sampling strategy that matches the criticality of the application. Otherwise, you either drown in telemetry or pay too much to keep it.
So when I think about high availability at scale, my biggest lesson learned is actually a simple one.
Break systems into modules where it genuinely helps. Keep supporting software to a minimum. Be aware of deployment topology. Respect state. And never assume cloud removed the need for sound systems thinking. It did not. It just made it easier to build distributed systems before earning the scars required to run them well.
Cloud is powerful. But it is still someone else’s computer. And the old bare-metal lessons still hold stronger than people think.
]]>TL;DR: Generic, I know, but it’s the truth. I spent 10 years in TCS, moved into SaaS with my friend Rupam, and learned that building is only one small part of founder life. While helping grow our products, I felt the pain of messy browser research and built TabMate to solve it. Shipping it felt great. Now comes the harder part: marketing, distribution, doubt, and the daily fight to keep going.
Aloha, back to blogging after a while! So much has happened since!
I quit my job at TCS after serving 10 years to take a foray into the Founder world. I learned a lot during my time there (I was lucky enough to be in projects that had a mix of dev, devops, infra, platform, security and AI) and matured as a developer. The only drawbacks were - constraints that would not let me build solutions for common pains we faced unless the client approved and, not being paid as much for the work I did. Well, in hindsight, it’s just how service based companies work, and I am not complaining. I wanted to have the freedom to build what I want and be able to say truthfully that I get paid for doing what I love.
I started my founder journey under an old college friend, Rupam. Since college days, he always had a founder mindset. I remember he had built a social networking website for our college - all with plain old HTML, CSS, PHP and SQLite. He is one of those old-school programmers whose learned the trade through sweat and toil. He, too, was in TCS though, he quit 6 years before me to start his own ventures.
When I joined him, he already had two profitable products in the market and one in the pipeline. I am grateful he agreed to mentor and guide me, given how differently we saw things when it came to managing the software lifecycle! See, I was from a controlled, constrained world where the user base was guaranteed hence, I optimized for longevity with failsafes, fallbacks, redundancy and best practices. He, on the other hand, optimized for velocity, stable MVPs, user-building and feedback driven features - and it makes sense because in the SaaS world, it doesn’t matter how polished or robust your solution is if no one uses it. I joined him with the project, Smart Bank Statement where, I learned the basics of the Founder life - and the hard truth that code is only a small part of it!
With Smart Bank Statement now stable and slowly gaining users, we turned our focus to improving what we have and, if possible, start on a new project. He started addressing feature requests from one of the products while I started looking into the other. During this time, my work involved browsing way more than I used to as I was actively learning the part of the software lifecycle that most devs don’t get to experience - selling it. The workflow was the same most days - google search, open up tabs, open a ChatGPT session, go here, copy this, paste that, fight ChatGPT over forgetting stuff, re-find the same info again - it got annoying, monotnonous and boring pretty soon. I wanted a ChatGPT in my browser which had context across all my tabs and also, remember where we were when I came back the next day. I basically wanted a retrieval agent IN THE BROWSER!
Well, we developers love a challenge. I set out to build it (hence the delay in this post). It initially started out as a side panel that could just see the text you selected, save memories which you choose to save and used a heuristic retrieval system. With time, however, it evolved into an assistant that could remember what you saved across your sessions and bring them up when necessary! Whenever I found something useful, I would just pin it or save it as a memory. Later, when I was prepping and needed reference, I would just ask it and that piece of info saved god knows how many tabs and hours ago just shows up! I named it TabMate out of love (and because I couldn’t think of anything else).
I pitched this idea to Rupam. My goal was to contribute to the pot. If I daresay, to me, our partnership sounds like the faint whispers an institute of products in the making. He has already contributed three products, it’s only fair for me to pitch in and pull my weight. He was skeptical and cross-questioned the idea, just like a rigorous co-founder should. One thing he said really stuck with me - “We, devs, build tools for ourselves then, we think everyone will find it useful. But most often, that is not the case.” I realized that he was right - the tool started out as a dev’s assistant but, dev workflows are pretty niche and varies from dev to dev. I couldn’t pitch a generic dev assistant at the browser level, I had to find the proper group of people whose work involves scouring across websites and living in “tabland”.
He let me take time to think and build it through and, after a lot of brainstorming and researching across the internet (lol), I finally managed to build a stable MVP. Yeah, it took longer than usual as I had to iterate and tune the retrieval loops and user flows. I was finally able to release it publicly on 27th April 2026. Now begins the hard game. The dopamine rush of building and shipping is over.
As I sit here now, putting into action the marketing strat I have for TabMate (trust me, it adapts every day), so many thoughts are playing across my mind. Did I do the right? When will conversion actually start? What else can I do to improve its reach and distribution? How do I tune my strategy? Do I give ads? Do I give it all up and go back to working for someone else? I read a quote on IndieHackers which read - “I built and I failed and I kept building.” Now, that guy sits at over $30K/month revenue. While stories and quotes are motivating, the human mind is a prison when left alone to think about and contemplate all possibilities. Mostly, it tends to converge on the negatives. There are times I think that it’s best to keep TabMate for myself and concentrate on the products we already have. I guess I am still human. Yet, the thought of having real users for a system I built with my own hands is really seducing and I keep doing what is necessary - ethical and fair but, necessary.
It’s the early days and yes, there are a lot of uncertainties. TabMate might live for a while then be integrated into browsers, get shelved and stay as my personal tool or, truly live its potential. I mean, this context switching pain is something everyone must be feeling, I just have to get to the right kind of people. I might pivot and build other ideas - let’s see how it goes. For now, the dopamine rush of building has settled and the dread of marketing has set in. I have to find ways to get a dopamine rush out of this phase too. Maybe take some programmatic help? Hmmm, let’s see.
]]>TL;DR: This was not my idea originally. Rupert had already started exploring the space when I got involved. Once I joined, we looked harder at the market, the actual workflow pain and (aakashh242.github.io). What started as a broader finance direction became a much narrower product: take messy bank statement PDFs and turn them into structured, usable data. From there, we worked together to get the MVP out.
An old college friend and roommate of mine, Rupert, had already started thinking in this space before I joined. So this was not one of those stories where two people sit down on day one with a blank page and magically arrive at the final product. The motion had already started. I entered after that, and once I did, my role became less about “coming up with the idea” and more about pressure-testing it, sharpening it and helping move it toward something that could become a real product.
At the time, the idea-space was wider. Like many things around finance, it is very easy to drift toward the flashy layer first: dashboards, summaries, personal finance views, spending insights, charts and all the things that look good in a demo. On paper, that feels like the obvious direction. People do want visibility into their money, after all.
The problem is that this part of the market is crowded and, more importantly, the pain is softer. There is a difference between a problem people find interesting and a problem they are willing to pay to make disappear. The more we looked at it, the more it felt like the “analyzer” route sat closer to the first category. Useful, maybe. Attractive, sure. But harder to build a serious business around unless there is a very strong edge.
So we kept looking.
The more practical side of the workflow started standing out. Not the part where somebody wants prettier insights. The part where somebody already has the data locked inside a bank statement PDF and needs it in a usable format for real work.
That was more interesting.
Because on the surface, converting bank statements to Excel sounds solved. It sounds like one of those dull utility problems the internet has already handled ten times over. But once you look at the actual inputs people deal with, the ugliness shows up quickly. Scanned statements, inconsistent layouts, broken table structure, different debit-credit conventions, weird balance columns, low-quality images, multi-page files, sometimes even multiple accounts in the same document. Suddenly this “simple conversion” problem stops being simple.
And that is before the downstream pain even begins.
Getting rows out of a PDF is not the same as getting reliable data. That distinction matters a lot more in accounting and bookkeeping workflows than it does in casual consumer use-cases. If the extraction is only mostly correct, somebody still has to sit there and verify the output line by line. One shifted row, one wrong amount, one broken balance trail and the time savings start collapsing. In these workflows, “almost correct” is not a nice middle ground. It is often just another form of manual work.
That was the point where the product started becoming more real to me.
Once we teamed up, the conversation changed from “what can we build in finance?” to “what painful workflow exists here that people actually need solved?” That is a much better question, because it forces you to stop thinking in vague product language and start looking at where time is genuinely being lost.
We looked at existing players too. There were already tools in the market, obviously. Some looked dated. Some were too broad or enterprise-heavy. Some could handle easy statements but would struggle as soon as the documents became messy. Some could extract data, but still left enough cleanup and checking on the user that the problem was not really solved.
That gap mattered.
To me, the opportunity was never “nobody is doing this.” That is usually the wrong lens anyway. The real opportunity was that the problem was still painful enough, despite existing tools, that there was room for a more focused and more accurate product.
So the idea got narrower.
Not a personal finance dashboard. Not a generic document AI platform. Not a bloated accounting suite. Just a focused workflow: upload a bank statement PDF and get back structured output that is usable enough to save real time.
That kind of narrowing is easy to say and much harder to do.
Once the direction became sharper, the implementation questions also became sharper. You stop thinking only in terms of OCR and start thinking about statement variance, normalization, row structure, balances, reconciliation, scanned versus digital PDFs, error detection and the difference between extraction that merely looks plausible and extraction that can actually be trusted.
That distinction shaped how we approached the product.
The goal could not just be “convert PDF to Excel.” There are too many ways to technically do that while still dumping the messy part back onto the user. The output had to be clean enough that it reduced work, not just moved work to a different stage.
That is what we built the MVP around.
Rupert had the initial seed. I joined once things were already underway. From there, together, we took it through the more difficult but more valuable phase: questioning the original direction, tightening the scope, understanding the market better and actually getting a usable MVP built instead of staying stuck in idea-land.
A lot of startup stories get rewritten after the fact to sound cleaner than they were. Two founders see a giant market, spot a perfect gap, align instantly and start executing with full clarity. Real life is usually more uneven than that.
This one certainly was.
The idea was already in motion before I came in. The initial space was broader than where we ended up. The clearer version of the product only emerged after spending more time with the pain, the market and the workflow details.
But honestly, I prefer that kind of story.
It feels more real. Better products often come out of that process: not from trying to sound ambitious from the beginning, but from being honest enough to keep narrowing until the pain becomes sharp and the value becomes obvious.
That is how I got into building Smart Bank Statement.
Not by inventing the idea from zero, but by joining an old friend, helping pressure-test it and then building with Rupert toward something much more grounded than where it began.
]]>TL;DR: In the last post, I said v0.3.0 would harden the Remote MCP Adapter against poisoned tool metadata and weak session semantics. This post is the proof. I built a mutable mock MCP server, ran live adapter instances against it, and captured evidence for four security controls: tool-definition pinning, metadata sanitization, description minimization, and session-integrity binding.
In the last post, I wrote about four security issues I wanted to tackle in the Remote MCP Adapter.
That post was about design and intent.
This one is about evidence.
I did not want to stop at unit tests and say “trust me, it works.” So I built a local test harness around a mutable FastMCP server and used it to exercise the adapter end to end. The result is a set of reproducible evidence artifacts showing what the adapter actually does when tool metadata changes, when descriptions are too verbose, and when a session is reused under the wrong authenticated context.
I ran four live security scenarios:
Each scenario produced:
So this was not a mocked “assert function returned true” setup. It was a real adapter process talking to a real mutable MCP server.
The setup was simple on purpose:
The upstream could change its catalog on demand. That made it possible to test:
This was exactly the kind of thing I wanted to prove before claiming the adapter had become a safer boundary.
This was the most important one.
The adapter was configured to:
The first tools/list call established the session baseline.
Then I changed the upstream tool description.
The next tools/list in the same session failed with a drift message. The adapter invalidated that session and refused to keep using it. Reusing the same session again resulted in 409 Conflict. Starting a fresh session succeeded and picked up the upgraded catalog.
This closes the “rug pull” path where an upstream server can look safe during the initial review and then quietly mutate the tool surface after trust has already been established.
The trust boundary becomes:
That is exactly the behavior I wanted.
The second scenario targeted dirty model-visible metadata.
The mock upstream exposed tool metadata with:
I ran the adapter twice:
With sanitization enabled, the adapter cleaned the visible metadata before forwarding it.
The differences were visible in the captured tool snapshots:
With block mode enabled, the dirty tool disappeared from the adapter-visible catalog entirely.
The adapter no longer has to behave like a naive tunnel for model-visible metadata.
It can:
That gives operators a real first layer of defense against poisoned tool metadata.
The third scenario was about description surface minimization.
This is different from metadata sanitization.
Sanitization cleans obviously suspicious text. Description policy answers a different question:
How much tool prose should the model see at all?
The mock upstream exposed very long tool descriptions and very long nested schema descriptions.
I ran the adapter in two modes:
truncatestripIn truncate mode:
In strip mode:
This is not just a UI tweak on the top-level tool description.
The policy applies to the model-visible description surface more broadly, including nested schema prose. That matters because otherwise an upstream could just move the same persuasive or poisoned text from the tool description into schema field descriptions.
So this control now works the way it should.
The fourth scenario focused on session integrity.
This one matters because the adapter is stateful. It stores uploads, artifacts, tombstones, and other per-session state. That means session handling is part of the product’s security posture, not just a transport detail.
For this test I:
The old session had already been bound to the first authenticated context.
When I tried to reuse it under the rotated token, the adapter rejected it with 409 Conflict.
Then I started a fresh session under token B, and that worked.
The persisted SQLite state showed separate trust-context fingerprints for the old and new sessions.
Knowing or reusing an Mcp-Session-Id is not enough on its own.
When auth is enabled, the adapter now binds the session to the authenticated context that created it. A stale session cannot just be picked up under a different auth context and treated as valid.
That is the right direction for a stateful gateway.
I saved the full evidence bundle and will attach it as a ZIP artifact.
Evidence bundle: Download the full ZIP artifact
It includes:
So anyone interested can inspect the actual evidence instead of relying on screenshots or paraphrases.
There is one small wrinkle in the captured client behavior.
When a blocked session is retried, the FastMCP client sometimes surfaces the failure as a generic 409 Conflict instead of preserving the full response body each time.
That does not weaken the result, because the evidence still shows:
409 responseStill, it is worth calling out plainly.
The Remote MCP Adapter started as a way to make remote MCP servers more practical by handling uploads, artifacts, and stateful mediation.
That is still true.
But once a gateway starts mediating tool metadata and storing session state, it can no longer pretend it is just a dumb transport wrapper. It is part of the security boundary whether it wants to be or not.
That is what v0.3.0 is really about.
Not security theater. Not vague “hardened mode” marketing.
Actual controls. Actual live tests. Actual evidence.
I am not done with the security work yet.
But this release crosses an important line: the adapter is now starting to defend the boundary it creates, instead of just expanding it.
That was the goal.
]]>TL;DR: I built Remote MCP Adapter to solve remote file and artifact handling. Then I realized the same adapter could also be abused unless it actively defends against poisoned tool metadata and session misuse. So v0.3.0 is about turning that middleware into a safer boundary, not just a convenient one.
After finishing off the core work for the Remote MCP Adapter, I took a step back and started sharing it in forums to see how others are solving the same issue. You can read more about the inspiration behind it in this blog.
I read this blog on dev.to that talks about how the MCP protocol has an attack layer via tool descriptions. This issue might not affect folks running MCP servers locally but becomes a headache for teams and organizations wanting to host them centrally. An attacker can essentially poison tool descriptions or manipulate tool arguments to make the Agents perform sinister stuff! The author of that post has built a tool, Pipelock that acts as a firewall for AI agents. Do show some love to his work!
While pondering over the article, I realized that I have created a monster that could easily be used to infiltrate systems. And it’s out in the wild! So I’m taking the next logical step - learn from the blog, explore MCP attack surfaces and provide built-in defense against these attack surfaces.
While writing the v0.3.0 release, I am focusing on addressing four issues -
list_tools call for a session. Any drifts or changes in tool titles, schemas and descriptions in subsequent calls will be detected and either warned or blocked entirely.I can see a long night up ahead as I gear up to write and test these guardrails out. I shall publish my test results here in a new blog once I have them ready.
]]>TL;DR: Too many MCP tools in the context window slow agents down and worsen tool selection. GitHub tackles this with clustering and embedding-guided routing. In remote-mcp-adapter, Code-Mode avoids the problem by letting agents discover tools progressively instead of loading them all upfront.
The Model Context Protocol has been both - a boon and a curse for Agentic workflows. Yes, it allows you to connect your agent with diverse systems without needing to write custom integrations for every one of them. But as your tasks grow in breadth and complexity, the more integrations you need hence, the more MCP servers you run. This eventually results in large of number of tools that get shoved into your agent’s context window. Maybe the agent needs just 4-5 tools to perform the task, but you still end up paying the price for those extra tokens in the context window.
If you look at it from the Agent’s perspective, it sees
Do you see the problem? It sees the task at the end and, a lot of unrelated information beforehand can confuse even the best of the LLMs. You’ll notice an increased latency, a tendency to choose the wrong tools, losing context in between and claiming a half-finished task as complete and an overall decrease in quality and consistency.
A simple way to restrict this issue is by limiting the number of tools you can have active in a request. GitHub Copilot used to restrict to 128 tools per request but after a lot of flak from the community, they decided to remove that limit. So how did they solve the too-many tools problem? You can read about it here. In a nutshell, GitHub improved tool selection in Copilot by grouping tools into clusters and using embeddings to pre-select the most relevant ones, so the model doesn’t have to reason over hundreds of tools every time.
While working on my remote-mcp-adapter, I came to the realization that my adapter will contribute to increased token usage.
One of the adapter’s functionality is to override tools that required file uploads from clients. While
overriding the tool, it also appends instructions on how to perform a staged-upload to the original description.
This is done so that the model knows about the original semantics and constraints but also aligns with the staged-upload
procedure. This means for every upload-type tool configured, more tokens are sent in a list_tools call.
Although each upstream has its own MCP mount path, clients configured to connect to all upstreams would eventually get
hit by context bloat.
I did not want to replace the tool description entirely with the upload-staging instruction and risk losing semantics so, I went for keeping only the first sentence of the upstream tool description trimmed to 50 token. I figured the savings in tokens should make up for sacrificing a bit of semantics.
After reading that FastMCP 3.1.0 brought support for Code-Mode, I was ecstatic. My adapter uses FastMCP so I could just implement a config toggle to enable code-mode.
Code-Mode allows the Agent to progressively discover the tools it needs without having to bloat the context window with
all tool definitions. Instead of all your 1000 tools, it surfaces 5 tools - search, tags, list_tools, get_schema
and execute. The Agent searches for certain keywords, discovers tools matching those, decides which ones to use, get
their schemas then triggers an execute call. The video below will demonstrate Agent behavior without vs with code-mode.
With Code-Mode’s progressive discovery integrated into remote-mcp-adapter, teams can configure as many upstreams as they want (within their infra limits, of course). The latest v0.2.0 release of remote-mcp-adapter now includes Code-Mode. Let’s see how it fares.
]]>TLDR; Check out remote-mcp-adapter which provides stateful proxies for upstream MCP servers and handles the file exchange interaction with “file-touching” tools.
Recently, a use-case came along where we were tasked with hosting a central MCP platform. The idea was to bring all MCP servers under a common umbrella, apply guardrails and governance on them and make it the go-to place for all things MCP in the org. And it made sense, given the dangers unverified MCP servers in the wild pose. If we could provide most of the tools teams needed and enabled a secure self-service model to add more servers, we could push teams to use the org-approved MCP platform.
Although mainly built for local usage, most MCP servers did implement the Streamable HTTP protocol, allowing them to be hosted remotely. We did not run into many hiccups till we got around to adding servers that work with files - either consume or produce them. An example is the Playwright MCP server that can produce artifacts in the form of console logs, screenshots, saved PDFs and consume local files for browser uploads.
While other tools worked as expected, problems arose with the file-touching tools. Since the server and the agent did not share a filesystem, artifacts generated would never reach the agent and whenever the agent needed to upload files, the server would not find it.
The MCP specs define a construct called resources to share data that provides context to language models. They are perfect for sharing the server generated artifacts with the agents. However, file uploads require special handling too as the MCP specification does not support file uploads via elicitation yet.
I initially wrote a wrapper that acted as a proxy between the agent and the Playwright MCP server but pretty soon, need arose to host many more of these types of “file-touching” MCP servers centrally. As a result, instead of writing separate proxies for each, I wrote the remote-mcp-adapter which provides stateful proxies for upstream MCP servers and handles the file exchange interaction with “file-touching” tools.
Being a consumer of opensource, I am hopeful it will be beneficial to the community.
]]>