PP&A designed two structured interview guides, one for each perspective, and led in-depth interviews with senior AI and engineering leaders building production generative AI systems in banking, healthcare, insurance and technology, alongside practitioners with experience inside model-provider organizations. The developer guide walked through the sophistication of current deployments, selection criteria such as performance, latency, cost and licensing, the additional engineering needed to work around model limitations, and the total cost of self-hosting open models versus managed APIs. The provider guide probed prioritization frameworks, community engagement channels, benchmark strategy and the reasons capability launches fail to land.

The findings were synthesized into a report structured around four questions: what developers are building and how, where open and closed models stand today, how models are evaluated and selected, and what this implies for a provider's capability roadmap. Each theme paired interview evidence with a specific recommendation for the client.

The study gave the client a candid map of the market it competes in. Enterprise adoption was concentrating in coding assistance and multimodal document processing, which together accounted for the large majority of current use, with agentic multi-agent systems emerging as the next frontier. Most organizations ran a multi-model strategy, with closed-source models still carrying most production workloads and open models chosen for fine-tuning, cost and privacy. Interviewees were direct about where open models fell short: context length and reasoning, run-to-run consistency, and real-time latency.

On evaluation, developers described a universal trade-off between performance, speed and cost in which only two can be optimized at once, and dismissed public benchmarks as contaminated by training data in favor of internal evaluations on real business tasks. Trust in the vendor, reliability and uptime drove switching decisions more than headline capability.

The recommendations were concrete: prioritize coding and document-processing capabilities as the foundation for agentic use; position the model family across several points of the performance-speed-cost triangle rather than chasing all three; solve context, consistency and latency first in the open-source line; replace ad-hoc feedback with a standing monthly developer advisory group; invest in domain-expert data curation over raw data volume; and launch with dimension-specific claims backed by reproducible evaluations rather than broad capability promises.

• --