Cost calculator for local AI: developing a PoC

Not for the workstation configurations. They’re located in the office, run off a standard outlet, and are quiet during normal use, though they’re audible under full load. The server configurations (2 to 4 height units) belong in a server cabinet with cooling, which most companies have anyway. Power connections and cooling only become planning considerations for large multi-GPU systems; we’ll clarify that before placing the order.

Honest answer: The largest cloud models are still ahead at the top. For most specific business tasks—such as classification, extraction, document search, or code assistance—the difference in day-to-day use is minor or imperceptible, provided the model is a good fit for the task. The largest open-weight models match the performance of commercial cloud models in many tasks. The key factor is the fit between the model and the task, and we evaluate this in the potential analysis using your data.

Then swap out the model, not the hardware. Open-Weight models are interchangeable; switching to a new one involves just a download and a test run—no need to buy new hardware. Technology is working in your favor: The quality per gigabyte of graphics memory has been increasing for years, so the same machine can run increasingly better models over its lifetime.

Yes, and that’s the key to cost-effectiveness. A server can host multiple models or use a single model for different purposes—for example, as an assistant for employees during the day and for batch processing of documents at night. Utilization increases, while the cost per use case decreases. Configuring the server is therefore often just the beginning—not the end—of its use.

The calculator estimates the purchase price and spreads the cost over 36 months. Leasing through hardware partners is an option and helps preserve cash flow, but it has little impact on the total cost. Renting dedicated GPUs from European data centers is a middle ground for cases where data may not be transferred to U.S. providers but can be stored in an external data center. We’ll discuss which option best suits your needs on a case-by-case basis.

Unfortunately, there’s no such thing as automatic GDPR compliance. But the biggest structural hurdles are eliminated: no transfers to third countries, no U.S. data processors—your data never leaves the premises. The due diligence requirements you’re familiar with from other internal systems—such as access policies and deletion rules—apply here as well. For your data protection officer, this is a much simpler starting point than any cloud connection.