An air gap is an operating discipline

Once an isolated AI appliance is installed, much of its security rests on routine. Updates do not arrive on their own, and remote assistance is not the default. Each change crosses the boundary through a deliberate step, and each step is an opportunity to do it well.

Every crossing is verified

An air-gapped enclave has no connection to external or untrusted networks, though its own devices may share an internal network. Updates, drivers, model files and documents enter on media. NIST’s OT security guide lists removable media among the paths that need policy and control, and cautions that air gaps alone are not enough.

An illustrative procedure:

1. Prepare the package on a connected staging machine that never touches production.
2. Verify the publisher’s signature there and record the file hashes.
3. Carry it on dedicated, labelled media.
4. Verify the signature and hashes again inside the enclave before installation.
5. Record who carried it, who installed it and what changed.

Signature checks can run offline. The Sigstore cosign documentation describes verification in an air-gapped environment, provided the signatures and the public keys or trusted root travel with the artifact. The NSA-led Deploying AI Systems Securely guidance recommends hashes and encrypted copies of each model release, with hashes stored apart from the files, and a tested route back to the last known good state.

Models have supply chains

A model file can carry code. Hugging Face’s pickle security documentation explains that loading pickle-based files can execute arbitrary code. ANSSI’s generative AI recommendations favour secure formats such as safetensors, advise against formats such as pickle, and recommend regular integrity checks on model files, scripts and binaries. The NSA guidance adds that imported models should be inspected in a separate zone before use.

Licences matter as well. Open weights come with terms. The Llama 3.3 Community License, for example, requires anyone who distributes the model, or makes available a product or service that contains it, to include a copy of the licence and display “Built with Llama” prominently. It also incorporates an acceptable use policy. Other models carry other terms. A register of each model’s name, version, source, hash and licence, reviewed before each upgrade, keeps this in view. Licence questions for a specific use are best confirmed with counsel.

Patching follows risk

Isolation reduces exposure; it does not remove the need to patch. NIST SP 800-40 Rev. 4 frames patching as preventive maintenance, a normal cost of operating technology. For an isolated system, a sound approach tracks advisories for the operating system, GPU drivers and inference software, and sets the response by severity and exposure. Critical fixes are tested and applied promptly through an expedited path. Routine updates follow a planned calendar agreed with the firm. The NSA guidance names hardware vendor notices, including GPU advisories, among the sources to follow.

A maintenance record completes the picture: date, person, change, verification result and rollback point. ANSSI also recommends logging processing on the AI system and administering it from dedicated accounts and workstations.

Backups proven by restoring them

A backup earns trust when it has been restored. CISA’s #StopRansomware Guide calls for offline, encrypted backups, tested for availability and integrity in a recovery scenario, and for maintained golden images of critical systems. For an AI appliance, that covers configuration, the document index, the permission map, logs and the exact model files. Restore tests on spare hardware, at a frequency set by the firm’s risk and change rate, show how long recovery takes in practice. Monaco’s Loi n° 1.565, Article 31 calls for the means to restore access to personal data in appropriate time after an incident, and for regular testing of security measures. This is general information, not legal advice.

GPU sizing is arithmetic

A model’s weights must fit in GPU memory, with room for the working cache that grows with context length and concurrent users. A 70-billion-parameter model at 16-bit precision needs about 140 GB for weights alone, before any cache. The vLLM documentation sets out the levers: split the model across GPUs with tensor parallelism, use quantized weights at some cost in precision, and limit context length and batch size.

For scale, an NVIDIA L40S carries 48 GB and an RTX PRO 6000 Blackwell Server Edition carries 96 GB. Neither holds a 16-bit 70B model on a single card. NVIDIA’s NIM 1.8.0 release notes state that its Llama 3.3 70B Instruct container needs at least 400 GB of CPU memory, and warn of out-of-memory errors when the KV cache runs short. A smaller model that answers well from retrieved documents often suits a firm’s needs. Testing real questions before selecting hardware grounds the decision.

Power, heat and noise

GPUs draw substantial power. NVIDIA lists the L40S at 350 W maximum and the RTX PRO 6000 Server Edition at up to 600 W, configurable. With processors, storage and fans added, a multi-GPU server produces steady heat through the working day. Planning typically covers a dedicated circuit, a UPS sized for a clean shutdown, and room cooling. Server-edition cards are often passively cooled and rely on chassis fans that are loud in an office; a ventilated technical room suits them better. Temperature monitoring with alerts completes the setup.

Every routine above works best with a named owner and a named deputy, alongside the fallback way of working that ANSSI recommends for periods when the AI system is unavailable. The schedules described here are illustrative and are agreed with each firm. To plan an isolated deployment, write to contact@desgarde.com.