Hello, I'm Tak@, a System Integrator. Have you ever thought of AI as just a question-and-answer tool?

If so, that way of thinking might already be becoming outdated. When it comes to leveraging AI for business, we've entered an era where how effectively you provide "the right information" to AI "at the right time" determines its success, more so than the AI model's performance itself.

AI's "Intelligence" is More About the "Setting the Stage" Than the Query

Previously, "prompt engineering" was a commonly discussed technique for effectively using AI. This method focused on enhancing the quality of AI responses by crafting specific and detailed instructions for the AI.

In essence, it centered on perfecting a "single sentence." I've certainly wrestled with crafting that "perfect sentence" when developing AI tools.

However, Philipp Schmid, formerly a Technical Lead at Hugging Face and now a Senior AI Relations Engineer at Google DeepMind, strongly argues that this focus is shifting towards "context engineering," a broader and more powerful concept.

https://www.philschmid.de/context-engineering

So, what exactly is "context engineering"?

Schmid explains it as "the art of providing all the necessary context for an AI to realistically solve a task." This means it includes not only the instructions (prompts) sent to the AI but also all the information the AI refers to before generating a response.

This represents a shift in how we interact with AI, from a simple "question and answer" exchange to a more practical process of "information provision → environment setup → problem-solving."

Components that Make Up Context

For an AI to give a "smart" response, a combination of various pieces of information is required. The main components that constitute context are as follows:

• Instructions/System Prompt: These are the initial settings and rules that define the AI's fundamental behavior. You could say they determine the AI's "personality" or "role."
• User Prompt: This is the specific question or task the user directly asks the AI.
• State/History (Short-Term Memory): This refers to the flow of past conversations and information the AI temporarily remembers. It's like picking up "where we left off" for humans.
• Long-Term Memory: More permanent information learned by the AI from past interactions, such as user preferences, knowledge, or project summaries.
• Retrieved Information (RAG): External, up-to-date information obtained from documents, databases, APIs, etc. This enables the AI to "access external knowledge." Research has even shown that using OCR to extract titles and tables from PDF documents and preserving their structural information for RAG can improve AI answer accuracy.
• Available Tools: Definitions of functions or capabilities the AI can execute. For example, checking inventory, sending emails, or referencing a calendar — these refer to the means by which the AI can "take action."
• Structured Output: Specifies the format of the AI's response. This is important when you want the AI to provide information in a specific format, such as JSON.

How appropriately you provide these elements to the AI is key to maximizing its capabilities. As an SIer, I feel this is quite similar to defining requirements when building a complex system.

What Separates a "Cheap Demo" from an "Exceptional Product"

Schmid points out that "the secret to building a great AI agent is not writing complex code, but how well you can provide high-quality context." He goes on to state that many AI agent failures are not due to the limitations of the AI model's capabilities, but rather due to a lack of context provision.

For instance, let's consider a case where you ask an AI assistant to schedule a meeting via email with "Do you have time for a quick meeting tomorrow?"

The "Cheap Demo" AI

If the AI agent only sees the text of the email it received (meaning the quality of the context is low), its reply might be a mechanical and unhelpful "Thank you. Tomorrow works. What time would you prefer?"

This is like a "cheap demo" that just uses a template. The AI remains just a "question-answering machine."

The "Exceptional Product" AI

However, what if the AI agent was provided with context such as your calendar information, past email history with the recipient, your contact list (indicating the recipient is an important partner), and the tools to send emails?

Considering this rich information, the AI agent would likely send a reply like this:

"Hi! Unfortunately, my schedule is completely booked tomorrow. How about Thursday morning? I've sent an invitation, please let me know if that works for you."

This is an "exceptional product" experience, a thoughtful response that, like a human, considers the situation and your relationship with the other person.

The AI functions this way not because the AI model is incredibly smart or the algorithm is particularly clever. It's the result of providing the right context for the right task. My own hobby AI tool development also begins with understanding the context of users' "true needs."

The "Depth" and Practice of Context Engineering

Context engineering differs significantly from prompt engineering, which focuses on creating a perfect instruction in a single text. It functions not merely as a "string of text," but as a "system" that operates before the AI's processing runs.

The "depth" of this field lies in designing how information is provided and how the AI utilizes that information.

Dynamic Context Management

One of the core aspects of context engineering is dynamically generating context on the fly to suit the immediate task. This is like a seasoned secretary subtly handing over necessary documents during a meeting, providing the AI with the right information at the opportune moment.

Upon request, calendar data, past emails, web search results, and more are loaded into the AI's "working memory (context window)" in real time.

Information Organization and Presentation (Information Mechanisms)

Simply feeding vast amounts of information to an AI can confuse it and degrade its performance. This is where "information mechanisms" become crucial. Techniques for organizing and compressing information based on importance or chronology are necessary.

For example, a concise summary is often more efficient for AI to utilize than raw data. It's similar to how summaries and diagrams help humans understand complex information.

Tool and State Integration

For AI agents to act in the real world, integration with external systems is essential. Defining the tools an AI can use, such as inventory check systems, email sending APIs, and databases, and designing seamless integration between them is required.

This enables AI not just to generate information but to take concrete "actions."

Facing Challenges and How to Address Them

While context engineering holds immense potential, it also presents several challenges:

• Implementation Complexity: Designing and implementing such systems requires advanced expertise and experience.
• Tuning Costs: Significant trial and error are needed to bring AI behavior closer to the ideal. This is because models can be non-deterministic, meaning they don't always produce the same result.
• Computational Costs: Processing large amounts of information can increase the cost of using AI. Specifically, tool call feedback and long contexts can significantly increase token count and cost.

These challenges are precisely why the skill of not just "using AI" but "designing the environment to master AI" is becoming increasingly valuable.

The Reality of Context Engineering from an SIer's Viewpoint

As an SIer, I've spent years understanding clients' "business contexts" and designing systems tailored to them. The concept of context engineering in the AI world highlights the reality that this "ability to grasp context" determines the outcome of AI.

In system development, "requirements definition" is paramount.

It's not just about listening to user requests, but digging into the deeper context behind them — "why is this needed?" and "what do we want to achieve?" — that is essential for building truly valuable systems.

I feel that context engineering demands exactly the same approach from AI.

The "exceptional performance" that AI delivers depends not solely on the intelligence of the AI model itself, but on how well the "environment" in which that AI operates is prepared. This is akin to how a high-performance CPU cannot reach its full potential without sufficient RAM (memory), storage, and peripheral devices.

Thinking of an AI's context window as analogous to this "working memory" makes its importance even clearer.

We often mistakenly believe AI provides answers like "magic." However, behind that lies the "information" we provide as stage-setting.

If you're dissatisfied with an AI's response, it might not be because "the AI isn't smart," but because you haven't provided the AI with an appropriate stage.

Considering the "Next Step" in AI Utilization

The increasing attention on context engineering indicates that the AI industry is moving beyond mere "question-answering" to a stage where it solves more complex and realistic business problems.

Industry leaders like Shopify CEO Tobi Lutke, former Tesla/OpenAI's Andrej Karpathy, and Apple researcher Awni Hannun are endorsing this concept and emphasizing its importance, truly symbolizing this shift.

They state that if deep learning is likened to "learning while sleeping" (pre-trained knowledge), then context engineering is "learning while awake" (the technique of accurately supplying necessary information in real-time).

This provides an answer to the question of how to leverage AI's vast knowledge to solve "this very moment's" challenges.

The emergence of context engineering also promotes a change in skills required for AI development. Going forward, not just learning how to write prompts, but also knowledge of "information architecture" for designing information structures and "system integration" techniques for linking external tools with AI will become more crucial.

Furthermore, a continuous "evaluation framework" for assessing and improving AI behavior is indispensable.

The Importance of Information "Selection" and "Representation"

Simply providing a large amount of information to an AI is not always good. Providing unnecessary or irrelevant information can not only increase AI costs but also degrade performance.

That's why "selection" and "representation" of information are extremely important in context engineering.

• Information Gathering via RAG (Retrieval Augmented Generation): Selectively retrieve only necessary information from documents or databases and provide it to the LLM to improve response quality.
• Tool Selection: If an AI has too many tools available, it can become confused. Providing only the necessary tools can improve AI accuracy.
• Context Organization and Summarization: Long conversation histories or large documents can burden an AI. Summarizing or structuring important parts for provision allows the AI to utilize information more efficiently.

This is very similar to a project management approach where selecting necessary information and presenting it in an appropriate format enhances overall team productivity.

Tactics such as removing unnecessary information (context pruning) and separating information by context (context isolation) are also effective.

The Future of AI is Shaped by Your "Context"

The era of treating AI like a "magic box" is ending, replaced by a time when the focus is on how to build the "right context" internally. While prompt engineering focused on "what to say," context engineering encompasses "what to show and how to show it."

AI's "exceptional performance" doesn't arise solely from the evolution of AI models.

It is only achieved when humans design the "best stage" for the AI, gather the necessary information, and provide the appropriate tools. In other words, the future of AI depends on how deeply we, as humans, understand and intentionally construct the "context" with AI.

What kind of "stage" do you want to set for AI? And what kind of "exceptional performance" do you want the AI to deliver on that stage?

Follow me!