Open-Source LLMs and Running AI In-House
Learn what open-source and open-weight LLMs are, the pros and cons of running AI on your own infrastructure, Thai-language options, and a low-risk way to start.
A company's legal team wants AI to help read hundreds of contracts, but there is one condition: these documents must never leave the company network. So the question comes up: can we have something like ChatGPT running on our own servers? The answer is yes. This is the world of open-source LLMs and self-hosted AI. Before deciding, though, you should understand both the benefits and the burden that comes with them.
What is an open-source LLM?
An open-source LLM is a language model whose files the developer lets you download and run yourself, unlike an API service where you send text to be processed on the provider's servers. Technically, it helps to separate:
- Open-weight: the model weights are downloadable, but the training data or code may not be published.
- Fully open source: more is released, under a clear open-source licence.
The most important thing is each model's licence, which sets whether commercial use is allowed and under what conditions.
Examples related to Thai language
Typhoon from SCB 10X
SCB 10X introduced Typhoon as a family of models optimised for Thai. SCB 10X's blog says it released the pre-trained Typhoon-7B model as open source under the Apache 2.0 licence, built on Mistral-7B.
OpenThaiGPT
According to the Bangkok Post, OpenThaiGPT was launched by NECTEC with the Artificial Intelligence Entrepreneur Association of Thailand (AIEAT), the Artificial Intelligence Association of Thailand (AIAT) and the ThaiSC supercomputer centre, positioned as free, open-source and customisable software.
Before using either in production, check the latest version and licence on each project's official release page.
How do you run one yourself?
For experiments, there are tools like Ollama, which Real Python describes as a free, open-source tool for downloading and running models on your own machine. It works on macOS, Windows and Linux, and once the model is downloaded, your prompts are not sent to an external server. Organisation-wide use with many simultaneous users usually needs GPU servers and a team to run them.
Benefits and burdens to weigh
Benefits
- Full control over data, which never has to leave the organisation
- The ability to fine-tune for specific work
- Independence from external providers' price or policy changes
Burdens
- Hardware, electricity and operations staff
- You must update models, patch security and set up access control yourself
- Models small enough to run on ordinary hardware may not handle complex work as well as large cloud models
Running AI yourself does not make it safe automatically. It moves the responsibility for security to you.
Which option suits your organisation?
Self-hosting is attractive when: data is so sensitive it cannot leave, you have high, steady volume where the numbers show it beats paying per use, or you have an IT team ready to run it long term.
Cloud services fit better when: your team is small, you need to start quickly, the work needs the most capable models, or volume is still uncertain. Many organisations therefore choose a hybrid and gradually move some work in-house once the need is clear.
Watch out for
- Read the licence every time. Some models have conditions on commercial use.
- Download models only from official sources. Model files from untrusted sources are a supply chain risk, which OWASP includes in its Top 10 for LLM Applications 2025.
- Even if data stays in-house, using personal data must still follow the PDPA. Read more in PDPA and AI.
How to get started
- Identify the work whose data is truly too sensitive to leave the organisation.
- Try a small model on a test machine with a tool like Ollama.
- Compare quality against a cloud service using the same set of real tasks.
- Estimate the total cost, including hardware and people, not just the model.
- Consider a hybrid: in-house models for sensitive work, cloud for general work.
If you are still unsure which kind of model to pick, read Choosing the right LLM. And if this field interests you, browse AI and data roles at Plants.work Jobs.