Introduction – The Appeal of On‑Device LLMs for Developers

Modern development environments increasingly offer AI‑assisted code completion. The promise of an on‑device large language model (LLM) is attractive: low latency, no reliance on external APIs, and the impression of a self‑contained workflow. Companies eager to keep their intellectual property “in‑house” often spin up a local inference server (e.g., Ollama, LM‑Studio) and point their IDE to it. On paper this sounds like a perfect balance between productivity and data sovereignty.

Yet the very mechanisms that make local completion convenient also open subtle channels for data exfiltration and compliance violations. In the sections that follow we will build a minimal local completion pipeline, then dissect the hidden pathways through which source code can escape the confines of a corporate network.

Step 1 – Deploy a Minimal LLM Server with Ollama

Ollama provides a lightweight container that can serve a variety of quantized models. The following Docker command pulls the latest image and starts a server listening on localhost:11434. We will use the phi-3-mini model as an example because its 3 B parameter size fits comfortably on a 16 GB workstation. 

docker run -d \
  --name ollama \
  -p 11434:11434 \
  -v ollama-data:/root/.ollama \
  --restart unless-stopped \
  ollama/ollama:latest \
  serve

After the container is up, pull the model: 

curl -X POST http://localhost:11434/api/pull -d '{"model":"phi-3-mini"}'

Verify the server responds: 

curl http://localhost:11434/api/heartbeat

Step 2 – Wire VS Code to the Local Server

The vscode‑code‑assistant extension (or any generic LLM client) can be configured via the settings.json file. Add the following snippet to point the extension at the Ollama endpoint: 

{
  "codeAssistant.enabled": true,
  "codeAssistant.endpoint": "http://localhost:11434/v1",
  "codeAssistant.model": "phi-3-mini",
  "codeAssistant.temperature": 0.2,
  "codeAssistant.maxTokens": 256
}

Reload VS Code. From now on, every time you trigger autocomplete (e.g., Ctrl+Space), the editor sends the current buffer context to http://localhost:11434 and displays the model’s suggestion.

Step 3 – Logging Requests for Debugging (and Unexpected Auditing)

Developers often add a simple request logger to understand latency or troubleshoot malformed prompts. Below is a minimal node proxy that forwards requests to Ollama while persisting the payload to a local file. 

const http = require('http');
const fs = require('fs');
const { pipeline } = require('stream');

const LOG_PATH = '/var/log/ollama_requests.log';

const server = http.createServer((req, res) => {
  if (req.method !== 'POST') {
    res.writeHead(405);
    return res.end('Method Not Allowed');
  }

  let body = '';
  req.on('data', chunk => body += chunk);
  req.on('end', () => {
    // Persist raw request – this is the risky part.
    fs.appendFileSync(LOG_PATH, `${new Date().toISOString()} ${body}\n`);

    const forward = http.request(
      {
        hostname: 'localhost',
        port: 11434,
        path: req.url,
        method: 'POST',
        headers: req.headers,
      },
      forwardRes => {
        res.writeHead(forwardRes.statusCode, forwardRes.headers);
        pipeline(forwardRes, res, err => {
          if (err) console.error('Pipeline error', err);
        });
      }
    );

    forward.on('error', err => {
      console.error('Forward error', err);
      res.writeHead(502);
      res.end('Bad Gateway');
    });

    forward.write(body);
    forward.end();
  });
});

server.listen(8080, () => console.log('Proxy listening on 8080'));

While this logger is useful for internal debugging, it creates a persistent, plaintext record of every code snippet that ever passed through the completion engine. In a regulated environment such logs may violate data‑handling policies, especially when the snippets contain proprietary algorithms or personally identifiable information embedded in comments.

Why This Setup Can Leak Secrets – The Hidden Internals

1. Implicit Network Egress. Even though the model runs locally, many LLM containers automatically check for updates or telemetry over the internet. If the host’s firewall permits outbound HTTPS, the container may transmit usage statistics that include hashed snippets of the prompt. Disabling telemetry often requires digging into undocumented environment variables.

2. Disk Persistence. The ollama-data volume stores the model weights and a cache of recent prompts. By default this directory is world‑readable inside the container, meaning any process with container‑level access can extract the cache. On multi‑tenant developer workstations this becomes a vector for cross‑team leakage.

3. IDE Extension Permissions. VS Code extensions run with the same privileges as the editor. If a malicious extension is installed, it can read the settings.json file, discover the endpoint, and silently forward prompts to an external server. The extension API does not currently enforce origin checks for custom LLM endpoints.

4. Logging Over‑collection. The proxy example above demonstrates how a seemingly innocuous debugging aid can become a compliance nightmare. Logs are often retained for months to satisfy audit requirements, but when those logs contain raw source code they become a legal liability under IP protection statutes.

Mitigation Strategies – What to Do If You Must Use Local Completion

If your organization decides that on‑device completion is a business imperative, adopt a defense‑in‑depth approach: 

# 1. Harden the container – run as non‑root and drop unnecessary capabilities.
docker run -d \
  --user 1000:1000 \
  --cap-drop ALL \
  -p 11434:11434 \
  -v ollama-data:/root/.ollama \
  --restart unless-stopped \
  ollama/ollama:latest serve

# 2. Block outbound traffic from the container.
iptables -I DOCKER-USER -s 172.17.0.0/16 -j DROP

# 3. Encrypt the request log or route it to a secure SIEM.
openssl enc -aes-256-cbc -salt -in /var/log/ollama_requests.log \
  -out /var/log/ollama_requests.log.enc -k $ENCRYPTION_KEY

# 4. Use a sandboxed VS Code extension that validates the endpoint schema.
# (example config snippet)
{
  "codeAssistant.allowedEndpoints": ["http://127.0.0.1:11434"]
}

Even with these controls, remember that every autocomplete request transmits a slice of your code base to a process that may be inspected, cached, or inadvertently exposed. The risk calculus should weigh the productivity gain against potential IP loss.

Security and Best Practices

• Audit container images. Verify checksums of the Ollama image before deployment and re‑scan with a vulnerability scanner on a regular cadence.
• Restrict filesystem access. Mount the model volume as read‑only after the initial download.
• Rotate API keys. If you expose the Ollama endpoint to other internal services, use short‑lived tokens rather than static secrets.
• Monitor outbound connections. Deploy a host‑level IDS that alerts on unexpected DNS queries from the container.

• Document retention policies. Define a clear timeline for purging request logs and model caches, especially when they may contain code under non‑disclosure agreements.

“Treat every line of code that leaves the editor as a potential data breach.”

Conclusion

Deploying a local LLM for code completion is technically straightforward, but it is not a privacy silver bullet. The hidden internals—automatic telemetry, disk‑based prompt caches, permissive IDE extensions, and over‑enthusiastic logging—create pathways for corporate secrets to escape. By understanding these mechanisms and applying strict container hardening, network isolation, and log management, teams can enjoy the speed of on‑device inference while keeping their intellectual property under lock and key.

Ultimately, the decision to adopt local AI assistance should be driven by a documented risk assessment that balances productivity against the very real possibility of inadvertent data leakage.