Developing the Lintech Local AI Platform

Project snapshot

#

• Status: In Development
• Project type: Local AI, language-model engineering and controlled automation
• Role: Architecture, implementation, testing, model evaluation and operational ownership
• Initial delivery: Locally hosted conversational assistant
• Long-term objective: A privacy-conscious whole-house intelligence platform
• Model strategy: Proven open model first, original Lintech foundation-model research in parallel
• Processing strategy: Dedicated server resources for inference and training workloads
• Platform strategy: Kubernetes for APIs, orchestration, portal integration and future controlled actions

Overview

#

The Lintech Local AI Platform is the next stage of a wider homelab engineering programme. The long-term objective is to create a locally operated whole-house intelligence system that can assist with technical operations, household services, information retrieval and carefully governed automation.

The project is not beginning with an autonomous agent or unrestricted infrastructure access. The first milestone is a conventional chatbot running against a proven, locally deployable language model. This provides a stable baseline for evaluating response quality, local inference performance, privacy, conversation handling and integration with the existing Lintech Intelligence Platform.

Alongside that useful first system, a second track will establish the datasets, tokenizer, architecture experiments, training pipelines, evaluations and governance required to research, design and train an original Lintech language model from the foundation up. The initial open model may help propose candidate examples, analyse failures and support experiments, but reviewed data, architectural decisions, evaluation criteria and model promotion will remain under human control.

Environment and operating context

#

The existing homelab already provides the platform services needed to support an AI system responsibly. Kubernetes hosts applications, GitOps controls deployment, Prometheus and Grafana provide observability, Longhorn supplies persistent storage, Wazuh and CrowdSec provide security visibility, and the Lintech Intelligence Platform provides a unified operational interface.

A dedicated Dell PowerEdge server will supply the heavier compute and memory resources required for model inference, dataset processing, evaluation and future fine-tuning work. Kubernetes will host the lighter but security-sensitive control layer, including the chatbot API, policy enforcement, portal integration and any future tightly controlled action services.

This division keeps resource-intensive model processing away from the power-efficient Kubernetes nodes while preventing the model runtime from becoming a privileged infrastructure controller.

Project objective

#

The immediate objective is to deliver a useful local chatbot that can provide general technical assistance without claiming access to systems or information that have not been connected.

The longer-term objective is to develop a Lintech-specific assistant that can:

• Follow an evidence-led troubleshooting process.
• Understand the terminology and operating practices used across the homelab.
• Retrieve approved project and Runbook content with citations.
• Request current state through narrow read-only tools.
• Distinguish live observations from documented or planned information.
• Produce recommendations with impact, validation and rollback considerations.
• Propose tightly bounded actions for explicit human approval.
• Support future household services without becoming an unrestricted automation engine.

Why two development tracks are being used

#

The project deliberately separates immediate usefulness from long-term model development.

Track A: Build a useful assistant

#

Track A begins with a proven locally deployable instruction model. The model will be hosted locally and exposed through an internal, OpenAI-compatible interface. A separate Lintech AI API will manage conversations, enforce policy, apply request limits and later broker access to approved information sources.

This approach provides a usable chatbot early in the project and creates a stable baseline against which later models can be measured.

Track B: Develop an original Lintech foundation model

#

Track B establishes the research, data, training and evaluation foundations required to design and train an original Lintech language model:

• A versioned evaluation suite.
• A curated instruction dataset.
• Sensitive-data filtering and review procedures.
• Governed pre-training dataset development.
• Tokenizer design and evaluation.
• Model architecture experiments.
• Small-scale pre-training runs.
• Training and validation pipelines.
• Model checkpoints and reproducibility records.
• Optional LoRA comparison experiments.
• Candidate model cards.
• Open-model-versus-Lintech-model comparison reports.
• Safety and evidence-boundary tests.
• A controlled model promotion and rollback process.

The initial operational assistant will use a proven open foundation model so that the surrounding platform, evaluation framework, safety boundaries and user experience can be established against a reliable baseline. In parallel, the longer-term research objective is to design and train an original Lintech language model from the foundation up, including its architecture, tokenizer, governed dataset, training process and evaluation framework. LoRA adaptation of an existing model may also be explored as an intermediate comparison track, but it is not the final objective of the foundation-model programme.

Operational value and risk reduction

#

Before the project

#

The homelab contains strong operational data and documentation, but no conversational layer connects that information into a guided investigation experience. General-purpose AI services can help with individual questions, but sending detailed internal operational context to an external provider would introduce privacy, governance and availability concerns.

After the initial release

#

The first release will provide a locally operated chatbot with a defined security boundary, a versioned deployment process and a known model baseline. It will not have infrastructure tools, persistent memory or private repository access.

Intended risk reduction

#

The architecture is intended to reduce several risks:

• Sensitive operational context can remain within the local environment.
• Model credentials and provider access remain outside browser code.
• The model runtime receives no general Kubernetes administrator credentials.
• Infrastructure-changing capabilities are separated from language generation.
• Candidate models cannot replace the stable model without evaluation and approval.
• Live system state will come from approved tools rather than model memory.
• Changing facts will not be embedded permanently through fine-tuning.

These are architectural objectives. Their effectiveness will be validated as each phase is implemented.

Proposed architecture

#

User
  |
  v
Lintech Intelligence Platform
  |
  v
Lintech AI API on Kubernetes
  |-- conversation validation
  |-- policy enforcement
  |-- model selection
  |-- streaming responses
  |-- read-only tools later
  `-- approval workflow later
  |
  v
Model service on dedicated server resources
  |-- baseline model inference
  |-- embeddings later
  |-- evaluation runs
  |-- fine-tuning experiments
  `-- candidate model storage

The model service generates language. Kubernetes-hosted services determine which information or action interfaces are available. The model will not receive a general-purpose shell, unrestricted Kubernetes credentials or direct administrative access to infrastructure platforms.

Architectural decisions and alternatives

#

Hosted external model API

#

A hosted provider could reduce the time required to launch the first chatbot and might offer stronger performance than the initial CPU-only environment. The trade-off is that prompts and contextual data would leave the homelab, creating additional privacy, cost and governance requirements.

Model runtime directly inside Kubernetes

#

Running inference on the Kubernetes nodes would simplify service discovery, but it would compete with operational workloads and place heavy processing on the power-efficient ARM64 cluster. The nodes are better suited to orchestration, APIs and controlled tools.

Dedicated model-processing VM

#

A dedicated virtual machine on the server provides substantial memory, isolated model storage and a clean API boundary. It can support CPU-based prototyping immediately and GPU acceleration later without changing the portal or orchestration architecture.

Training a complete foundation model immediately

#

Training a competitive general-purpose language model from scratch would require a much larger dataset and considerably more accelerator capacity. It would delay the useful assistant while creating significant engineering and evaluation complexity.

Selected approach

#

The selected approach combines a proven open model, a dedicated processing environment, a Kubernetes-hosted control layer and a parallel foundation-model research programme. This provides useful capability early while preserving a credible route towards an original Lintech model whose architecture, tokenizer, training data and checkpoints are developed under the project’s own governance.

Implementation plan

#

1. Create the model-processing environment. A dedicated virtual machine will be provisioned on the server with separate operating-system and model storage. The initial deployment will use CPU inference so the architecture can be validated before any GPU investment.
2. Install a local model runtime. An OpenAI-compatible runtime will host one small test model and one stronger baseline model. The service will remain internal and will not be published directly to the Internet.
3. Benchmark the baseline models. Model load time, time to first token, generation rate, memory use, context handling and answer quality will be recorded. The largest model will not automatically be selected if its latency makes the chatbot impractical.
4. Build the Lintech AI API. A Python and FastAPI service will validate requests, apply the system policy, forward conversations to the model runtime, stream results and return safe structured errors.
5. Deploy the API through GitOps. The service will be containerised, built for the required architecture, published through CI and reconciled to Kubernetes by Flux. Credentials and configuration will use SOPS-encrypted Kubernetes Secrets.
6. Add the chatbot interface. The Lintech Intelligence Platform will gain an Intelligence Assistant page with conversation history, streaming responses, connection status, retry behaviour and a clear advisory boundary.
7. Establish the evaluation baseline. A reviewed evaluation suite will test general assistance, Kubernetes, GitOps, storage, security, evidence handling, live-state boundaries and action safety against the original model.
8. Create governed research datasets. Candidate material may be derived from approved Runbooks, project content and suitably licensed public sources, but every dataset component will require provenance, licence, quality and sensitive-data review before use.
9. Design and evaluate a tokenizer. Tokenizer experiments will measure vocabulary coverage, sequence efficiency and handling of technical language before a tokenizer is selected for model training.
10. Run small architecture experiments. Small transformer configurations will be trained first to validate the data pipeline, loss behaviour, checkpointing, evaluation and recovery process without presenting those experiments as production-capable models.
11. Train the first original Lintech candidate. A selected architecture will be pre-trained from an initial checkpoint using the governed dataset and reproducible configuration. Optional LoRA work may be used as a comparison track, not as the final foundation-model objective.
12. Compare the open and original models. The open-model assistant and Lintech candidates will receive the same held-out tests. Technical correctness, hallucination rate, evidence discipline, clarity, latency, safety compliance and resource use will be compared.
13. Use shadow evaluation before replacement. A candidate may process selected prompts alongside the stable model without becoming the user-facing response. This allows comparison without risking the working assistant.
14. Promote only after approval. An original candidate will replace the open-model baseline only if evaluation demonstrates a meaningful improvement without unacceptable regression. The stable open model will remain available as a rollback option.
15. Add read-only context. Approved portal summary tools will provide live system state with timestamps and source links. The model will request these tools rather than memorising changing infrastructure facts.
16. Add documentation retrieval. Approved Runbooks and architecture material will be indexed separately from model training and returned with citations and document review dates.
17. Add controlled proposals and actions later. The assistant may eventually propose narrow actions, but execution will require an independent Kubernetes service, deterministic validation, visible impact, rollback information, human approval and post-action verification.

Model-development strategy

#

The original Lintech model will be trained to learn general language and reasoning patterns from governed data while avoiding permanent encoding of changing environment state. Behavioural adaptation through LoRA may be evaluated separately, but the foundation-model track covers tokenizer selection, architecture design, pre-training, checkpointing and evaluation from an initial model state.

Suitable fine-tuning targets include:

• Evidence-aware language.
• Safe admission of uncertainty.
• Troubleshooting order.
• Runbook structure.
• Recommendation quality.
• Tool-selection decisions.
• Approval and rollback language.
• Lintech-specific terminology.

Changing facts will remain outside the model:

• Current node readiness.
• Current Pod counts.
• Current alerts.
• Current image digests.
• Current service availability.
• Current storage health.

The separation is intentional:

Fine-tuning = how the assistant behaves
Retrieval   = what approved documents say
Live tools  = what is true now
Actions     = what may change after approval

Dataset and evaluation governance

#

The baseline model may help propose examples, categorise failures and generate question variations, but the model will not approve its own training data or control its own promotion.

The dataset lifecycle will be:

Proposed example
      |
      v
Sensitive-data and secret scan
      |
      v
Human review
      |
      +-- approved dataset
      `-- rejected examples

Full chat histories will not be added automatically to training. Only selected, sanitised and reviewed examples will be eligible.

Every candidate release will record:

• Base model and exact revision.
• Base-model licence.
• Dataset version.
• Evaluation-suite version.
• Tokenizer and vocabulary version.
• Model architecture and parameter count.
• Pre-training configuration.
• Optional fine-tuning configuration.
• Known strengths and weaknesses.
• Safety-test results.
• Artifact checksum.
• Approval status.
• Rollback model.

Evidence and success criteria

#

The project is currently in development, so future capability is not presented as completed work. Initial success will be measured through repeatable technical evidence.

Phase 1 acceptance criteria

#

• Model-processing VM provisioned and internally reachable.
• Baseline model responds through an OpenAI-compatible API.
• Memory use and response latency recorded.
• Chat API returns a healthy status.
• Streaming responses work correctly.
• Invalid roles and oversized requests are rejected.
• No conversation content is persisted by default.
• No infrastructure tools are enabled.
• Kubernetes deployment is reconciled through Flux.
• The LIP chatbot renders and handles provider failure safely.

Foundation-model acceptance criteria

#

• Open-model baseline evaluation results recorded before original-model training.
• Dataset provenance, licensing and sensitive-data reviews recorded.
• Training, validation and held-out test data separated.
• Approved datasets contain no detected credentials or private keys.
• Tokenizer selection is supported by recorded coverage and efficiency tests.
• Small-scale training validates the full pipeline before a larger run.
• Every training run is reproducible from versioned data and configuration.
• Candidate checkpoints are compared against the stable open-model baseline.
• Safety and evidence-boundary tests do not regress.
• Stable-model rollback requires only a controlled configuration change.

No performance improvement, accuracy percentage or response-speed claim will be published until measured.

Security and action boundaries

#

The initial assistant will have:

• No Kubernetes credentials.
• No shell or SSH access.
• No Proxmox administrative access.
• No access to Kubernetes Secrets.
• No persistent personal memory.
• No unrestricted repository access.
• No infrastructure tools.
• No ability to claim that an action was completed.

Future action handling will remain outside the model runtime. A safe action flow would be:

User request
   |
Model creates structured proposal
   |
Policy service validates scope
   |
Portal displays evidence, impact and rollback
   |
Human approval
   |
Narrow action service executes
   |
Post-action validation and audit result

The model will never receive a general command-execution interface.

Limitations and residual risks

#

• Initial CPU inference may have a noticeable response delay.
• A larger model may fit into available memory but still be too slow for interactive use.
• Future GPU support will require hardware, power, cooling and passthrough validation.
• An original model may underperform the established open-model baseline for a considerable period.
• Scaling an original model beyond research size may exceed the available compute, storage, power or training-time budget.
• Training data can transfer errors or outdated assumptions into the model if review is weak.
• Local operation improves data control but does not automatically guarantee privacy or security.
• Prompt injection remains relevant when retrieval and tools are introduced.
• Model output remains advisory and may be incorrect.
• Controlled actions will increase risk and are intentionally deferred.
• Optional LoRA candidates will remain dependent on the licence and characteristics of their base models.
• Original-model training data must remain compatible with recorded licences and permitted uses.

Plain-English glossary

#

Locally deployable model

A model that can run within privately controlled infrastructure under its applicable licence. This does not mean the Lintech project or its source code will be published.

Inference

Running a trained model to generate an answer.

Pre-training

Training a model from an initial parameter state on a broad governed dataset so that the model learns language patterns and representations.

Fine-tuning

Continuing training on a targeted dataset to adapt an existing model’s behaviour or specialised capability.

LoRA

A parameter-efficient fine-tuning method that trains smaller adapter matrices while leaving the base model largely unchanged.

Embedding

A numerical representation of text used for semantic search and retrieval.

Retrieval

Finding relevant approved documents at question time and supplying them to the model as evidence.

Tool

A fixed software function that can return live information or request a narrow action.

Shadow evaluation

Running a candidate model alongside the stable model for comparison without making the candidate user-facing.

Source availability

#

The publication and licensing model for the Lintech AI source code, datasets, adapters and trained artifacts has not yet been decided. This case study describes the engineering direction only and should not be interpreted as a commitment to release the project as open source.

Current status

#

In Development.

The existing homelab, GitOps, observability, recovery, security and Lintech Intelligence Platform foundations are operational. The dedicated AI processing virtual machine, model runtime, Lintech AI API and chatbot page are the next implementation milestones.

No live portal tools, persistent memory, custom fine-tuned model or infrastructure actions are currently claimed as complete.

Planned milestones

#

1. Provision the AI model-processing VM.
2. Install and benchmark the first open model.
3. Build the standalone Lintech AI API.
4. Deploy the API to Kubernetes through Flux.
5. Add the LIP chatbot interface.
6. Publish the first baseline evaluation report.
7. Build and review the first governed pre-training dataset.
8. Design and evaluate the first Lintech tokenizer.
9. Run small-scale transformer architecture and training experiments.
10. Train the first original Lintech model candidate.
11. Compare original candidates against the open-model baseline.
12. Explore LoRA as an optional comparison track.
13. Add read-only portal tools and approved retrieval.
14. Design, but do not yet enable, controlled action workflows.

Skills being developed

#

• Local LLM deployment
• Model-runtime benchmarking
• Python and FastAPI development
• Streaming chatbot interfaces
• MLOps and model versioning
• Dataset design and governance
• Evaluation engineering
• Tokenizer design
• Transformer architecture experiments
• Foundation-model pre-training
• Checkpointing and training recovery
• Optional LoRA fine-tuning
• Retrieval-augmented generation
• Tool and policy design
• Kubernetes AI orchestration
• Security boundaries for AI systems
• Human approval and auditable automation

Project outlook

#

The Lintech Local AI Platform is intended to become the intelligence layer above an already operational homelab. Starting with a proven open model provides immediate value and a measurable baseline. Developing a separate foundation-model track creates a structured path towards an original Lintech model, with project-owned architecture decisions, tokenizer experiments, governed training data, reproducible checkpoints and independent evaluation.

The objective is not to create an assistant that changes infrastructure without supervision. The objective is to build a trustworthy system that can explain evidence, support investigation, retrieve approved knowledge and eventually participate in carefully controlled operations where every action remains visible, validated and reversible.

Follow the project

#

The project will be documented as the model VM, chatbot, evaluation suite and first original Lintech model experiments are completed. Related case studies cover the Kubernetes platform, Flux delivery model, Lintech Intelligence Platform, security monitoring and recovery architecture that provide the foundation for this work.