Implementing AI in the Midmarket: A 3-Phase Path to Productive AI Solutions

Companies that use AI in a targeted and meaningful way increase their efficiency, automate processes, and create new digital products. This will enable them to meet the specific challenges facing small and medium-sized enterprises in the coming years, whether it be a shortage of skilled workers, increasing competitive pressure, or digital transformation.

But every beginning is difficult: many small and medium-sized enterprises and organizations face the challenge of finding and implementing meaningful use cases for AI:

How can we turn an idea into a truly productive AI application?

This is a legitimate question that requires a well-founded answer and should not simply be dismissed. We support companies on this very path. It is worthwhile to seek outside help early on, gather more expertise, and leverage the synergy between internal team knowledge and external understanding of AI and business applications.

From previous projects, it has become clear to us that the implementation of AI in companies can be divided into three phases, which are based on the typical maturity level and needs of a company. Read on to find out where you are and where you can start as an SMB or medium-sized company.

Right now, you’re at a stage where: You are interested in AI, but don't know where its use actually brings real benefits.

Getting started with AI rarely begins with code, but rather with the right questions and strategy. One thing that is unique about AI in the midmarket: While startups can often experiment with venture capital and large corporations have large innovation budgets, small and medium-sized businesses have to operate with a sense of proportion and responsibility. Here, projects must work, create added value, and pay off. Every step should really move you forward.

To identify your use cases for AI, we recommend going through the following three steps:

1. Identify key processes
   Start with the company's essential processes, as this is where the greatest leverage usually lies. AI brings the most added value when applied to business-critical, frequent, or resource-intensive processes.
2. Break down processes into subtasks
   Take a detailed look at each step of the process: What exactly happens? Who does what? To find your use cases, it helps to understand the actual steps that make up a process.
3. Recognize AI potential
   Where are there manual, repetitive, error-prone, or time-consuming tasks or activities involving:

• A focus on text and information processing
• simple and repetitive nature or high susceptibility to errors
• high manual time expenditure
• a digital basis.

Here is an example use case: responding to a customer inquiry:

• Identification and evaluation of suitable use cases
• Basic workshops ("What is possible with AI? Do we even need AI for this?")
• Creation of checklists and guidelines for getting started with implementation

Successful companies start with specific goals, such as:

• "Support requests should be answered automatically."
• "Knowledge documents should be searchable in an intelligent way."

It is equally important to involve specialist departments at an early stage, as they have the necessary knowledge about processes and data.

Start with a realistic scope: in some cases, this means starting small. In our projects, we regularly see how applications evolve over time. What starts out as a small, clearly defined use case often grows into a central component of business processes. Many functions that are taken for granted today would have been unthinkable in the initial development phase.

Many initiatives fail because they are launched without a clear objective or based on a vague impulse. "We need AI too" – does that sound familiar?

Other risks in this phase include unclear data, a lack of prioritization, or misjudging what AI models can do well.

At the end of this phase, you will have a solid understanding of where and how AI can be used effectively in your company, as well as a concrete roadmap for gradually introducing it into practical use. You will know whether you can implement the application yourself or whether you need a technical partner who has already implemented AI projects. This gives companies the basis they need to develop a tangible solution from an idea in phase two.

Once it is clear where AI can add value, the question arises as to how a solution can be implemented in detail. This phase marks the transition from idea to implementation. At this point, the technical feasibility should be assessed and the data situation and potential benefits should be realistically evaluated.

• Cost estimation & budget planning
• Technical feasibility analysis
• Definition of requirements and functions
• Development of a prototype or proof of concept
• Decision template for management or project launch

Successful AI projects are developed iteratively. Instead of spending months working on a big project, a functioning prototype should be created early on to enable genuine feedback.

It is equally important to have realistic expectations: AI is not a magical substitute for human intelligence, but a tool that, when used correctly, can deliver enormous efficiency gains.

Data quality also plays a key role: poor or unstructured data leads to poor results.

Many challenges in this phase arise from overly high expectations. People often underestimate where AI models are strong and where their limitations lie. If results are not immediately convincing, this quickly leads to disappointment.

Another common misconception is that AI projects are treated like traditional software projects. But AI follows different rules. Models do not deliver completely deterministic results, but are always based to some extent on probabilities. Prototypes therefore need room for trial and error, and only through trial and error can it be determined which approaches work reliably.

Data protection and compliance must also be considered at an early stage. Anyone who uses personal or sensitive data must define clear framework conditions as early as the design phase. This not only contributes to GDPR-compliant data handling, but also shows why local models and proprietary infrastructures are often the better choice.

The last point that is often overlooked concerns rights and role concepts. AI applications must not undermine existing authorizations. If a model can access internal documents, for example, it must be ensured that it does not accidentally disclose information that users are not actually allowed to see – for example, through seemingly harmless questions such as "How much does my colleague earn?"

At the end of phase two, there will be a validated concept and a functioning prototype that demonstrates the benefits of the AI application. Based on the prototype, it will be possible to assess whether the application achieves its objectives and should be further scaled and expanded.

If the prototype proves successful, the decisive phase begins: the transition into productive operation. This is where a test system becomes a resilient, maintainable, and secure application that can run long-term within the company. In this phase, it is important to place a strong focus on stable operations, monitoring, and testing.When developing a productive AI application, three things are important, regardless of whether it is a medium-sized company, a start-up, or a large corporation:

1. Reliability
   Applications must deliver reproducible, traceable results. This should be ensured through testing, among other things.
2. Data security
   Ensure that the architecture complies with data protection regulations (e.g., local LLMs, GDPR-compliant hosting).
3. Maintainability
   AI is dynamic. Processes for model changes, versioning, and monitoring must be considered from the outset.

A structured transition from prototype to productive application ensures that AI does not remain a one-off project, but becomes a sustainable part of your digital infrastructure.

• Scaling & performance optimization
• Test setup (e.g., LLM-as-a-Judge, unit tests)
• Operational concepts (monitoring, retraining, versioning)
• Ensuring security & compliance
• Further development & feature iteration

AI projects do not end with the go-live. A continuous improvement process is crucial for further optimizing AI applications. For users of AI applications, too, more and more functions can be added iteratively to make work even easier and processes more effective. You will see that even if you start with a small application, you will quickly find other possibilities that can be easily implemented in a well-developed AI application.

Many companies have their first AI prototype built "quickly." As long as only a few people are working with it, everything seems to be fine. But when the prototype is finally put into actual operation, it becomes apparent what is missing: a good operating setup, monitoring, testing, logging, and clear responsibilities.

At the end of phase three, you will have a production-ready AI application that is scalable, secure, and maintainable—and that creates lasting value for your company or organization.

Whether AI implementation in small and medium-sized enterprises is successful is not left to chance, but rather the result of a well-thought-out approach.

If you are familiar with and follow the three phases – orientation, concept & prototyping, scaling & operation – you will lay the foundation for using AI in a targeted, secure, and effective manner. Be aware of which phase you are in so that you do not oversize the project. Be mindful of pitfalls and success factors in each phase, because quality and functionality are particularly important for SMBs.

At makandra, we are happy to support you on this journey—with technical expertise, strategic understanding, and a focus on data sovereignty and security. This is how an idea becomes a productive AI application that will advance your company for years to come.