AI infrastructure

Running a model yourself makes sense when a law or a contract stops you sending the data to an outside provider, as with patient records under PHIPA, or when you use it at a steady, high volume. What you gain is control over where the data goes. At small volume it costs more than paying per use.

The break-even article puts the floor for a model you run yourself at roughly $500 to $1,500 a month, and below about 10 million tokens a month (a token is a word or part of one) a cloud service stays cheaper. Cloud prices have kept falling since it was written, which pushes that point higher.

Many of the newer articles are about AI agents, the tools that read your email or files and act on them. Any text an agent reads can carry instructions, a problem called prompt injection, and no vendor has a fix for it. The safe design keeps a person approving anything the agent sends out, and the same care applies to AI tools you host yourself, which attackers scan the internet for.

Before you buy an AI tool, write down what it is for and plan to train the people who will use it. In BDC's survey of Canadian small businesses using AI, 86 percent of those that trained their staff were satisfied with the return, against 53 percent of those that did not.