LLMs in the Enterprise: What They Can Do, What They Cannot, and What Goes Wrong

Large language models have attracted more business attention than any AI technology since the spreadsheet, and with it a level of both hype and anxiety that makes it difficult to form a clear-eyed view of what they actually offer enterprises. The honest picture is neither as exciting as the advocates suggest nor as threatening as the critics warn. LLMs are powerful tools with specific strengths, specific limitations, and a consistent set of deployment failure modes. Understanding all three is the starting point for using them well.

What LLMs do well

Large language models are exceptionally capable at tasks involving natural language: generating coherent, contextually appropriate text across a wide range of styles and formats; summarising long documents accurately; answering questions based on provided context; translating between formats and levels of formality; explaining complex material in accessible terms; and drafting initial versions of documents that a human refines. These are not trivial capabilities. For organisations that spend significant portions of skilled time on these tasks — and most do — LLMs can create substantial leverage. Custom LLMs, trained on an organisation's specific knowledge, add a further capability: answering questions specific to that organisation's context. A model trained on a firm's procedures, product documentation, client records, and institutional knowledge can answer questions that no general-purpose model could answer accurately, because the answers depend on information specific to that organisation. This is the highest-leverage enterprise application of LLM technology and the one that most consistently delivers measurable return.

What LLMs do not do well

LLMs are unreliable at precise numerical reasoning. They can perform simple arithmetic and recognise quantitative patterns, but their accuracy on complex calculations is inconsistent and their failure mode — confident presentation of a wrong answer — is particularly problematic in financial or analytical contexts. Any enterprise application that requires precise numerical outputs should verify those outputs rather than trusting them directly, or should use LLMs in combination with deterministic calculation tools rather than as the calculation layer itself. LLMs have knowledge cutoffs and do not have access to real-time information unless that access is explicitly provided. A model trained on data through a certain date does not know what happened after that date, and will not reliably acknowledge this limitation in its responses. Enterprise deployments that require current information must be designed to provide that information to the model as context rather than relying on the model's training. LLMs can generate confident-sounding text that is factually incorrect. This property — sometimes called hallucination — is not a bug that will be entirely fixed in future versions. It is a characteristic of how these models work. Applications that require high factual accuracy must include a verification layer, either human review or automated fact-checking against reliable sources. Deploying an LLM in a high-stakes context without a verification layer is deploying on the assumption that the model will not hallucinate, which is not a safe assumption.

How enterprise deployments go wrong

The most common enterprise LLM deployment failure is scope creep past the model's reliable capability. A deployment that starts within the model's strengths — answering questions about documented procedures — gradually expands to include tasks the model handles less reliably, without the corresponding expansion of oversight. The deployment appears to be working well until it fails in a way that reveals the scope has drifted beyond what the model should be trusted with. The second common failure is inadequate context provision. LLMs are only as good as the information they can access when answering a question. A model deployed to answer questions about a company's products that is not given access to current, accurate product information will answer from its training data, which is either outdated, incomplete, or absent for a specific company's offerings. The model's confidence in its answers does not diminish when its information does. Deployment architecture must ensure the model has access to the information it needs to answer reliably.

The governance gap

A third failure mode is the absence of governance: no defined owner for the LLM system, no process for identifying and correcting errors, no mechanism for updating the system as information changes, and no monitoring for outputs that indicate the system is failing. An LLM system without governance will degrade over time as the information it is based on becomes stale and as edge cases accumulate without being addressed. Governance does not require elaborate infrastructure. It requires someone who owns the system's accuracy, a process for reporting issues, a cadence for reviewing and updating knowledge, and a monitoring approach that catches systematic failures before they become visible to end users. This is basic operational discipline applied to a new category of system, and it is the discipline that separates deployments that perform consistently over time from those that impress initially and disappoint subsequently.

The practical implication

LLMs are genuinely powerful tools for specific enterprise applications. Deploying them within their strengths, with appropriate oversight and governance, produces real and measurable value. Deploying them beyond their strengths without oversight produces the failures that accumulate into the impression that AI did not deliver. The technology is not the variable. The design of the deployment is. Getting that design right — scoped correctly, governed properly, with appropriate verification where accuracy matters — is the difference between an LLM that becomes part of how the organisation works and one that becomes an expensive cautionary tale.

For further reading on this topic, check out our guide on How to manage maternity leave compliantly and supportively.