From Simplicity to Transparency - A Reflective Journey

This article was originally published at drstarry.github.io.

---

Text in, Text out

Last Christmas holiday, driving home from Crystal Mountain after a fulfilling day of snowboarding. We shared the ride with a PM friend from the Bay Area who works on AI products. As conversation drifted between snow and work, she mentioned something that barely registered at first.

"LLM models are essentially our vendors now," she said, "like AWS. We depend on their availability and reliability the same way."

I nodded, not thinking much of it. As an infrastructure engineer, countless vendor horror stories had imprinted in my memory — eternal migrations, compatibility nightmares, teams working months or years moving between vendors, versions, or libraries.

Almost reflexively, I asked, "So LLMs have vendor lock-in too?"

An innocent question. Besides conversing with ChatGPT from time to time, I knew almost nothing about AI systems' inner workings, except vaguely noting "fancier" model versions in the news.

"No," my partner, also our driver answered, "no lock-in¹. LLM is text in and text out."

Something uncomfortable grew in my mind during that drive home. Those simple words—"text in and text out"—seemed to be a betrayal of what I understood.

How could something so powerful have such an unpredicable and uncontrolable interface?

Human Engineers

For years, my work centered on empowering engineers by creating systems that simplified their lives, whether through better microservices framework or streamlined database access layer. I'd consistently ask myself:

• Who are my customers and why do they need this?
• What challenges can I solve?
• Why build these systems this specific way?

My customers are other engineers — practical people navigating complex technical landscapes through documentation, code, and API calls. Understanding the humans behind the code mattered most. The worst mindset in my opinion, was blaming users when they didn't interact with systems as expected, like the infamous "RTFM". So we listened through conversations, surveys, and user journey analysis, each interaction revealing struggles, hesitations, and interesting nuances.

Throughout my journey, one refrain dominated:

"We should make things dumb."

The ideal? People shouldn't need to understand anything — everything should be as easy as clicking a button. This philosophy seemed unquestionably right.

Overfitting?

I wanted to tell a story. Many years ago, my company had an aggressive roadmap to dismantle our monolithic Ruby on Rails app, the source of endless pain: spaghetti code, weirdly duplicated logic, database schema nightmares, teams stepping on each other's toes, merge conflicts, deploy bottlenecks. This monolith had become our primary technical liability!

"This migration will solve everything!" You can guess what followed when everyone raced to achieve this "dismantle mission" in creative and chaotic ways. We created a sprawling ecosystem of interdependent services that would take years to untangle.

So many new questions emerged: How to handle distributed transactions? How to debug across services? Libraries versions explosion? These were intellectually challenging problems, but in our rush, we traded familiar difficulties for alien ones.

A messy diagram showing the complexity of microservices architecture with many interconnected services, arrows going everywhere, and multiple dependency layers

The story, to me, is more than yet another distributed systems cautionary tale:

We optimize for what we understand today, and too often things can go to another extreme with single-dimensioned incentives.

Some service boundary reflected random choices made with incomplete information, often influenced by team structure; people did whatever it takes to move away from the "monster".

In reality, looking back, one framework is rarely inherently worse or better than another; there are always tradeoffs. When we rush to move away mostly based on one's disadvantages, we risk overfitting to our current understanding of the problem.

What problems are we solving?

Another common story. It begins innocently, we identify a legit painpoint and architect a solution. But months or years later, something strange happens: roadmaps become self-sustaining ecosystems filled with complexity we created. Team stand-ups focus entirely on implementation. New team members learn "how" without hearing "why."

Teams become so invested, and spend years grinding away at complicated solutions while, possibly, drifting from the original problem. Or, by delivery time, that problem has evolved or vanished entirely.

AI Agents: my new customers?

The "text in, text out" conversation and my exploration of LLM APIs triggered a paradigm shift.

As an infrastructure engineer, my customers would no longer be exclusively human engineers, but increasingly AI agents — potentially our systems' primary future users.

These agents differ remarkably from humans:

• They generate precise code from (clear²) instructions.
• They read fragmented documentation without complaint.
• They debug systematically through knowledge bases.
• They retry relentlessly when blocked.
• They wait patiently for feedback.

These differences fundamentally change how we approach developer-facing system design. AI systems thrive on transparency and comprehensive information. Designing for AI means rethinking our abstractions—not to hide complexity, but to expose it in ways that allow machines to reason, adapt, and optimize.

I reflected on typical infrastructure projects — coding paradigm migrations, testability improvements in distributed systems, streamlining critical yet risky operations—and saw not just technical challenges, but playgrounds for AI agents that could transform our approach.

The Complexity Paradox?

We're trained to "hide complexity", creating "magical" interfaces that protect human (engineers) from tedious details, which might be true—hiding complexity reduces cognitive load and prevents mistakes.

But what if exposing some complexity helps AI systems reason and generate code more effectively?

When we over-abstract, we don't eliminate complexity — we relocate it. Each layer of abstraction adds hidden costs, often creating technical debt that slows innovation. What looks elegant from one angle is often complexity pushed elsewhere, hidden but still present. In the AI era, these hidden complexities become barriers. Features that made interfaces "friendly" for humans—such as limited options, metaphors, and overly simplified workflows—might render them impenetrable to AI systems.

I continued to reflect on some abstraction layers I had worked with in the past, which hid critical details—like data transformations or system behavior—that could have been useful for debugging or optimization. Fast forward to recently, I came across the concept of machinery interoperability, and later Anthropic's approach to Constitutional AI, where they make system values and behaviors explicit rather than hiding them behind layers of abstraction.

For AI systems, which depend on full transparency to reason about and adapt to their environments, this lack of interoperability is a critical failure. Systems designed with machinery interoperability in mind—exposing necessary complexity through clear documentation and flexible interfaces—are far better suited for AI.

Premature abstraction also creates tomorrow's constraints. Each layer impedes transformation, yet continuous transformation remains our industry's only constant.

To design systems that can evolve with AI, we must rethink our approach to abstraction, prioritizing transparency and adaptability over the illusion of simplicity.

A Backwards Incompatible Future

Human progress has never been truly "future-proof." We constantly impose new requirements, evolve values, and discover challenges our predecessors couldn't imagine.

The profound realization: we must design not just for what we know, but for what we cannot yet comprehend.

As engineers and system builders, this demands a few shifts in thinking:

1. Choose honest exposure of necessary complexity over illusory simplicity. When complexity cannot be eliminated, make it visible rather than hiding it behind abstractions.
2. Think twice when you want to re-invent some wheels. Just like LLM models are iterating super fast, a lot of the foundational infra capabilities may evolve faster than you think.
3. Return repeatedly to the first principles. Don't lose yourself implementing current best-guess solutions. Iterate quickly, fail fast, and pivot³ when needed.

Learning to Unlearn

The critical skill for navigating this landscape isn't technical expertise alone, but learning to unlearn—remaining curious and reimagining solutions at technology's accelerating pace. Sometimes the most powerful interface is raw—text in, text out—leaving room for intelligence on either side to navigate its way.

Success lies not in knowing all answers, but in asking better questions and willingly discarding yesterday's perfect solutions—embracing the humility to admit our best abstractions only reflect our current understanding. In the AI age, this understanding might be obsolete before finishing this sentence.

Footnotes

1. 1.

   Fast forward to today, after a few months of exploring AI applications, I need to admit, there is some "coupling" between prompts and "raw" LLM model; but since end of 2022, many LLM orchastration framework came out to provide a thin abstraction layer, to help you organize or "abstract" how AI systems should interact with LLM. ↩

2. 2.

   Don't get me wrong, I fully ackownledge the existance and risk of hallucination, here I wanted to point out the behavior change, since human make mistakes too ↩

3. 3.

   This seems impossible for big organization, where many decisions were made and the stakes are very high; it sparks some facinating and daunting reflection on how should we approach multi-year roadmap ↩