Overview

Modern progressive web apps (PWAs) often showcase on‑device machine‑learning features to avoid round‑trips to the cloud. One popular demo is a TensorFlow.js model that doubles the resolution of user‑uploaded photos. While the demo looks impressive, the underlying design carries hidden performance, privacy, and maintenance costs that most developers overlook.

Building a Minimal Client‑Side Upscaler

The following sections walk through a bare‑bones implementation using TensorFlow.js and the esrgan super‑resolution model. The code is intentionally simple so readers can focus on the mechanics rather than on polishing UI. 

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Client‑Side Upscaler Demo</title>
  <script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]"></script>
  <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/esrgan"></script>
</head>
<body>
  <input type="file" id="uploader" accept="image/*">
  <canvas id="original" style="display:none;"></canvas>
  <canvas id="upscaled"></canvas>
  <script>
    const uploader = document.getElementById('uploader');
    const originalCanvas = document.getElementById('original');
    const upscaledCanvas = document.getElementById('upscaled');

    let model;

    async function loadModel() {
      // Load the pre‑trained ESRGAN model from TF Hub
      model = await tf.loadGraphModel(
        'https://tfhub.dev/captain-paul/esrgan/1',
        { fromTFHub: true }
      );
    }

    function drawImageToCanvas(img, canvas) {
      const ctx = canvas.getContext('2d');
      canvas.width = img.width;
      canvas.height = img.height;
      ctx.drawImage(img, 0, 0);
    }

    async function upscale() {
      const tfImg = tf.browser.fromPixels(originalCanvas);
      const expanded = tfImg.expandDims(0).toFloat().div(tf.scalar(255));
      const result = await model.executeAsync(expanded);
      const squeezed = result.squeeze().mul(tf.scalar(255)).clipByValue(0, 255).cast('int32');
      await tf.browser.toPixels(squeezed, upscaledCanvas);
      tfImg.dispose();
      expanded.dispose();
      result.dispose();
      squeezed.dispose();
    }

    uploader.addEventListener('change', async (e) => {
      const file = e.target.files[0];
      if (!file) return;
      const img = new Image();
      img.onload = async () => {
        drawImageToCanvas(img, originalCanvas);
        await upscale();
      };
      img.src = URL.createObjectURL(file);
    });

    loadModel();
  </script>
</body>
</html>

The script performs three essential steps: (1) load a TensorFlow.js graph model, (2) convert the uploaded image into a tensor, and (3) write the upscaled output back to a <canvas>. All of this occurs in the user's browser, eliminating any server‑side processing.

Why This Looks Attractive

Developers often cite three reasons for choosing this pattern: reduced backend load, instant feedback for the user, and a perception of stronger privacy because data never leaves the device. The demo indeed satisfies those criteria on paper, but each advantage masks a deeper issue. 

// Example of measuring memory usage before and after inference
console.log('Memory before:', performance.memory.usedJSHeapSize);
await upscale();
console.log('Memory after:', performance.memory.usedJSHeapSize);

Running the upscaler on a mid‑range smartphone can spike the JavaScript heap by 150 MB, trigger garbage‑collection storms, and cause noticeable UI jank. On older devices, the same model may cause the browser to kill the tab entirely, leading to a poor user experience.

Hidden Privacy Implications

While the image data stays on the device, the model itself is fetched from a public CDN at runtime. An attacker who controls the network path can replace the model with a malicious version that extracts fingerprints from the input image or embeds a hidden watermark. Moreover, the browser’s fetch request for the model is not covered by any CSP policy in the example, opening the door to supply‑chain compromise. 

// Adding Subresource Integrity (SRI) to protect the model URL
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/esrgan"
        integrity="sha384-abcdef..."
        crossorigin="anonymous"></script>

Adding SRI mitigates accidental tampering but does not protect against a compromised CDN. The safest approach is to self‑host the model, sign it, and serve it over HTTPS with strict CSP headers.

Maintenance and Version‑Lock Risks

TensorFlow.js evolves rapidly. A model that works with version 4.4 may break with a minor patch due to API deprecation. Because the upscaler is embedded directly in the PWA bundle, updating the dependency requires a full redeploy of the web app, which many teams postpone. Over time the app drifts into a security‑critical “unknown” state where the runtime library may contain undisclosed vulnerabilities. 

// Pinning exact TensorFlow.js version in package.json (if using a build step)
{
  "dependencies": {
    "@tensorflow/tfjs": "4.4.0",
    "@tensorflow-models/esrgan": "1.0.0"
  }
}

Even with exact version pinning, the underlying WebGL driver on the device can expose side‑channel leaks, especially when the GPU is shared with other tabs. The browser sandbox does not guarantee isolation of GPU memory across origins, a nuance that most developers miss.

Security and Best Practices

If a project still requires on‑device upscaling, follow these safeguards:

  • Host the model behind authenticated, signed URLs; verify the integrity client‑side.
  • Implement a fallback to server‑side processing for devices that report low GPU memory or high battery drain.
  • Limit the maximum input resolution; large images cause exponential memory growth.
  • Use the tf.tidy() helper to ensure tensors are released promptly.
  • Enforce a strict Content‑Security‑Policy that disallows inline scripts and unknown origins.
"A client‑side AI feature that looks sleek can become a silent drain on battery and a vector for supply‑chain attacks."

Conclusion

Embedding AI‑driven image upscaling directly into a PWA is tempting, yet it introduces a cascade of hidden liabilities: unpredictable performance on heterogeneous hardware, subtle privacy exposure through remote model loading, and a maintenance burden that can outpace the original business case. Before shipping such a feature, weigh the cost of a lightweight server endpoint against the perceived benefits of a fully client‑side experience.

In many scenarios, a hybrid approach—performing a quick low‑resolution preview on the client and delegating high‑quality upscaling to a hardened backend—delivers a smoother user journey while preserving control over model provenance and resource consumption.