Skip to content

Bulk Metadata Updates

Metadata methods — update_info(), add_tag(), update_file_classification(), and the rest — operate on one container at a time. Their full semantics are covered in Core Concepts: Metadata. Applying them across many containers is a client-side loop over a finder result. This page covers doing that loop at scale, validating the change set before committing it, and the per-container permission methods.

Tip

Use iter_find() rather than find() for the loops on this page. find() pulls every match into memory first; iter_find() streams one page at a time, so it works on a project with thousands of sessions. See Core Concepts: Finders.

Updating info across containers

Iterate a filtered set and call update_info() on each. update_info() merges at the top level, so unrelated keys are preserved — review the merge-versus-replace behavior in Core Concepts: Metadata before running this at scale.

Apply an info update across matching sessions
--8 < --"test_docs_advanced_bulk.py:bulk_update_info"

A sequential loop is the simplest approach. When the container count is large enough that round-trip latency dominates, dispatch the updates concurrently — see Performance Optimization: Async Calls.

Danger

replace_info() in a loop overwrites the entire info dict on every container it touches, discarding any keys you did not supply. Prefer update_info() for bulk work unless you have explicitly confirmed every target should be reset to the same value.

Bulk tag management

Tags are added and removed individually with add_tag() and delete_tag(). To tag a cohort, loop the finder result:

Add a tag to every matching session
--8 < --"test_docs_advanced_bulk.py:bulk_add_tags"

Removing a tag across a set is the same loop in reverse — and because tags are queryable, you can drive the removal loop directly from the tag itself:

Remove a tag from every container that carries it
--8 < --"test_docs_advanced_bulk.py:bulk_remove_tags"

Bulk file classification

File classification is set per file with update_file_classification() (merge) or replace_file_classification() (overwrite). The aspect/modality rules and the warning about invalid classifications are in Core Concepts: Metadata. To classify many files, loop the files on a reloaded container:

Classify every matching file on an acquisition
--8 < --"test_docs_advanced_bulk.py:bulk_file_classification"

To classify files across many acquisitions, wrap this in an outer project.acquisitions.iter_find(...) loop, reloading each acquisition before reading its .files. See Bulk File Operations for the memory-efficient iteration pattern.

Validate before you write

The defining risk of a bulk metadata update is scope: a finder query that matches more (or fewer) containers than you expect. Resolve the query into a concrete list, report what it matched, and validate that list before entering the write loop.

Build and validate the work list before applying
--8 < --"test_docs_advanced_bulk.py:validate_before_bulk_update"

This three-step shape — collect, validate, apply — is worth making a habit for every bulk write. It turns a silent "updated 4,000 containers instead of 40" into a printed count you can sanity-check, and it gives you a single place to abort if the match set is empty or implausibly large.

Batch permission changes

Container objects expose per-container permission methods: add_permission(), update_permission(), and delete_permission(). A "batch" permission change is a loop over the containers you want to affect.

Grant a user access across several projects
project_ids = ["000123456789abcdefABCDEF", "111123456789abcdefABCDEF"]
user_id = "researcher@example.com"

for project_id in project_ids:
    project = fw.get_project(project_id)
    project.add_permission({"id": user_id, "access": "rw"})

Permission payloads are instance-specific

Each instance can define its own roles with unique names, so the exact payload these methods expect varies. Confirm the available roles for your instance before running a permission loop — see Admin Concepts: Permissions.