Skip to main content

Upgrading

Release notes for each version are on the Edge Site releases page. Read them before upgrading.

warning

If your Edge Site runs edge-site Helm chart 0.0.32 or earlier, follow Upgrade from chart 0.0.x instead.

Upgrade the edge controller​

Download the latest package from the Edge Site releases page, saving it as /tmp/foxglove-edge-controller.deb, and install it:

sudo apt install /tmp/foxglove-edge-controller.deb

The package keeps your envfile and catalog, and restarts the service. Check the logs for the new version number:

journalctl -u foxglove-edge-controller

Upgrade from chart 0.0.x​

The edge-site Helm chart 0.1.0 and later run a reimplemented edge controller. It can keep your site's existing catalog, but it doesn't find it on its own. Chart 0.0.32 and earlier store the catalog at /data/index/foxglove.db on the index volume, while chart 0.1.0 and later default to /data/index/controller.db.

If you upgrade without pointing the edge controller at the existing catalog, it creates a new catalog at the default location and starts a new site epoch. Foxglove then treats the site's existing recordings as removed from the Edge Site.

Before you upgrade​

Check the following, and resolve any differences before upgrading:

  • Recording storage. If your values set recordingStorage.provider to s3_compatible, or set recordingStorage.bucketName, stay on chart 0.0.32 and contact support. The edge controller reads recordings only from the recording storage Persistent Volume Claim. With recordingStorage.provider set to anything other than local, chart 0.1.x exits at startup with invalid storage root /data/storage.
  • Recordings API clients. Requests to POST /v1/recordings must include projectId. deviceName must be up to 100 characters, using letters, digits, _, ., and -. Paths must be relative to the storage directory, and can't be absolute or contain ... GET /v1/recordings isn't available. See Recordings API.
  • Recording paths. On the Recordings page, check the paths of this site's recordings. If any path is absolute or contains .., contact support before upgrading. Retention resolves those paths as written, not inside the storage directory. It can delete a file outside the storage directory, or report a recording as deleted while its file stays in the storage directory.
  • Environment overrides. Review edgeController.env in your values file:
    • GARBAGE_COLLECTION_MAX_WORKERS, SLOW_QUERY_LOG_MIN_DURATION, SLOW_QUERY_LOG_PARAMETERS, DISABLE_DATABASE_STATISTICS_REPORTING, DISABLE_SERVER, and RECORDING_STORAGE_BUCKET_NAME are no longer read.
    • Chart 0.0.x keeps watch mode off when WATCH_MODE isn't set, and later chart versions might not. To keep submitting recordings only through the recordings API, add WATCH_MODE with the value '"false"'.
    • In WATCH_AUTO_IMPORT_PATTERN, * matches across directory separators, so *.mcap also matches recordings in subdirectories.
    • WATCH_IGNORE_SUFFIX is deprecated. The edge controller applies it only when no WATCH_INCLUDE_* or WATCH_EXCLUDE_* variable is set. Setting any of those variables also stops the default exclusion of .active and .json files. To filter recordings, use the variables in Recording filters.
  • Google Cloud Storage uploads. Verify that your existing credential file and bucket permissions support the reimplemented edge controller. Uploads at or above MULTIPART_PART_SIZE (8 MiB by default) use XML multipart uploads, which need additional permissions. See Google Cloud Storage credentials.
  • Azure uploads. Add STORAGE_AZURE_STORAGE_ACCOUNT_NAME to your cloud-credentials Secret, or set globals.upload.bucketName to <account>/<container>. See Configure cloud credentials.
  • Metrics. imports_gigabytes and recording_upload_ms are no longer exposed. recordings_gigabytes changes from a cumulative value updated during indexing and uploads to a catalog-size snapshot, using units of 1024³ bytes instead of decimal gigabytes. Use recordings_indexed_bytes and recordings_imported_bytes for indexing and upload rates. See Available metrics.

Upgrade steps​

  1. Save the values your chart 0.0.x release uses, and make a copy to edit:

    helm get values foxglove-edge-site --namespace foxglove --output yaml > values-0.0.x.yaml
    cp values-0.0.x.yaml values.yaml

    In values.yaml, add the existing catalog's location:

    edgeController:
      env:
        - name: DATABASE_CONNECTION_STRING
          value: sqlite://file:/data/index/foxglove.db

    If values.yaml already has an edgeController.env list, add the entry to that list instead of adding a second edgeController key.

    This is chart 0.0.x's connection string without its ?_journal_mode=WAL suffix. Don't copy the suffix: the edge controller exits at startup with unknown query parameter `_journal_mode` while parsing connection URL. Keep this setting in your values file for all later upgrades.

  2. Pause processes that submit recordings to the recordings API. Let in-flight imports finish, or record any unresolved import requests and their status for follow-up. Keep normal recording submissions paused until validation is complete.

  3. Stop the edge controller and wait for its pod to terminate:

    kubectl scale statefulset site-controller --replicas=0 -n foxglove
    kubectl wait --for=delete pod/site-controller-0 \
        --namespace foxglove --timeout=120s

    Scaling changes the desired replica count without waiting for shutdown. Continue only once the pod is absent; if the wait times out, resolve the shutdown before proceeding.

  4. Back up the index volume (the edge-controller-index-claim Persistent Volume Claim, unless you changed edgeController.indexClaim). With the controller stopped, take a snapshot with your storage provider's tools, or copy foxglove.db and any foxglove.db-wal and foxglove.db-shm files off the index volume as one backup set. Keep the backup until you've confirmed the upgrade.

    On the Recordings page, record the paths of a few recent recordings from this site.

  5. Upgrade the release to the latest chart version from helm search repo foxglove/edge-site --versions:

    helm repo update
    helm upgrade foxglove-edge-site foxglove/edge-site \
        --namespace foxglove \
        --version <version> \
        --values ./values.yaml

    The upgrade restores the StatefulSet to one replica. If the pod doesn't start, scale it up:

    kubectl scale statefulset site-controller --replicas=1 -n foxglove

  6. Check the logs:

    kubectl logs -n foxglove statefulset/site-controller

    On a site with existing recordings, the site initialized entry must show a site_epoch of 0. A legacy catalog with neither recordings nor reporting history gets a new epoch at first start, which is expected. A higher epoch on a site with existing recordings means the controller started with a new catalog instead of adopting the existing one. In that case, stop it and contact support.

  7. Confirm that the site shows as "Online" on the Sites settings page, and that the recordings from step 4 are still listed on the Recordings page.

  8. With normal recording submissions still paused, submit a completed test recording using your updated recordings API client. Confirm that the POST succeeds and the recording appears in Foxglove, then import it to your Primary Site and wait for the import to complete. Test with the site's existing credentials and network configuration, including any proxy. For self-managed uploads, use a recording at or above the configured MULTIPART_PART_SIZE (8 MiB by default) to validate multipart permissions.

  9. Resume normal recording submissions after these checks pass, and check any unresolved import requests from step 2.

Roll back​

To return to chart 0.0.32 before the chart 0.1.x edge controller has started, upgrade to chart 0.0.32 with your chart 0.0.x values file, without the DATABASE_CONNECTION_STRING override:

helm upgrade foxglove-edge-site foxglove/edge-site \
    --namespace foxglove \
    --version 0.0.32 \
    --values ./values-0.0.x.yaml

If the chart 0.1.x edge controller has started, it has already updated the catalog in place, and chart 0.0.32 can't reliably use it:

  1. Pause recording submissions, scale the StatefulSet to zero, and wait for site-controller-0 to terminate, as in upgrade step 3.
  2. Restore the index-volume snapshot, or replace foxglove.db and its -wal and -shm files with the matching backup set. Remove any post-upgrade catalog and sidecar files before restoring the file backup, including sidecars that aren't present in the backup. Keep the controller stopped throughout the restore.
  3. Run the chart 0.0.32 upgrade command above, confirm that the controller starts, and resume recording submissions.

Recordings cataloged after the backup aren't in the restored catalog.

If the chart 0.1.x edge controller started a new site epoch, restoring the backup doesn't bring back the recordings Foxglove treated as removed. Contact support.