Buy AI Infrastructure Against Completed Work

From component lists to a service envelope

Oracle’s 24 September AMD architecture discussion describes a coordinated rack-scale approach. The 29 September Oracle–NetApp announcement proposes a managed storage option, with availability still on the roadmap. Together they prompt a procurement question: how should a buyer compare integrated systems without treating a vendor’s reference design as proof of business value?

This framework starts with a service envelope: the work to be completed, its quality threshold, expected demand and acceptable recovery behaviour. It applies to cloud services and owned infrastructure, although the cost and responsibility boundaries differ.

References: [1][2]

Define the unit of useful work

A support agent, document-processing pipeline and interactive search service do not produce the same unit of output. Define a completed case, validated document or accepted answer before comparing throughput. Record the model, context size, retrieval steps and tool calls used in the test.

A supplier should report both normal and peak behaviour. Response-time percentiles, queuing and rejected work can matter more to users than average throughput. Keep the workload definition stable across proposals so that a change in assumptions cannot masquerade as an improvement in infrastructure.

Find the constraint before scaling the system

Follow a request from intake through data retrieval, model execution, tool use and final validation. Measure time and resource consumption at each stage. If an external application throttles requests, adding accelerators may increase idle capacity without improving completion. If retrieval is slow, better data placement may matter more.

Then repeat the test under a realistic failure: a dependency becomes unavailable or a node is removed. The question is whether the service can still meet its agreed operating boundary, not whether every component remains at peak utilisation. Spare capacity may be a resilience requirement rather than waste.

Use a complete cost boundary

For a defined measurement period, divide attributable operating cost by the number of accepted work units. Include compute, storage, network transfer, platform licences, support and human exception handling. For owned infrastructure, use an agreed allocation of capital and facilities costs over the same period.

Show sensitivity to demand, retry rates and utilisation rather than a single optimistic number. A low-volume pilot cannot establish the economics of a busy production service. Equally, committing to peak capacity before demand is credible can turn an attractive unit price into an expensive idle asset.