Perspective
Model Choice Is Becoming a Procurement Discipline
Specialist models and broader model catalogues increase choice. Buyers need a repeatable way to compare completed work, operating constraints and the cost of changing models.
Two different responses to the same market
On 15 September, Salesforce announced Koa, a CRM-focused model built on NVIDIA Nemotron, with selected customer pilots and broader availability expected later in the year. The same day, its AWS announcement expanded model choice through Amazon Bedrock. A vendor can therefore offer both specialisation and access to a wider catalogue.
These options should change how buyers evaluate AI. The useful question is which combination of model, tools and operating controls completes a defined business task acceptably. A model’s name or general benchmark position cannot answer that question on its own.
A benchmark is a starting hypothesis
A specialist model may perform well on familiar tasks while struggling with a company’s unusual product rules, languages or exception cases. A general model may reason well but require more intervention to use enterprise tools correctly. Neither outcome can be inferred from the category label.
Build the evaluation set from representative work, including cases that previously caused rework. Keep some examples out of the tuning process. Run competing configurations against the same input data and tool permissions. Separate the accuracy of a proposed answer from the correctness of the action eventually recorded in the business system.
Compare cost per accepted outcome
Token prices describe one input, not the full cost of service. Include model calls, retrieval, tool execution, monitoring and the human time needed for approval or correction. Count only outcomes that pass an agreed quality threshold; otherwise repeated failed attempts can make a cheap configuration look productive.
For a service workflow, the denominator might be a correctly resolved case with no avoidable reopening during a defined review period. For an internal search tool, it might be a verified answer delivered within the response-time target. These are proposed evaluation units, not claims that either supplier currently prices its products this way.
Treat switching as an operating capability
A second model in a catalogue is not yet a tested fallback. Prompts, tool schemas, output validation and error handling can behave differently after a substitution. Regional availability and data-handling commitments may also vary by configuration.
Procurement should ask how model changes are approved, how regression tests run and how the previous configuration can be restored. Retain versioned evaluation results and a clear allocation of responsibility between the application supplier and model provider. The aim is a repeatable change process, rather than an unrealistic promise that models are interchangeable.
Buy evidence that survives the next release
A buyer with one low-risk use case may sensibly choose a single model and a simple support arrangement. Maintaining several models adds cost and is justified only when it addresses a real availability, quality or commercial requirement.
What should remain reusable is the task definition and the evidence standard. Those assets let an organisation test a new specialist model, challenge a price increase or retire an underperforming configuration without restarting the business case. Model choice then becomes a managed procurement discipline rather than a recurring contest between launch claims.