Quick Reference¶

Finders¶

Each container type has it’s own finder in flywheel. It their most basic form, finders will return all possible matches. Finders are called with the .find() method.

To specify what container type you’re looking for, it can be called after a container type from the flywheel client:

• fw.projects.find()

• fw.subjects.find()

• fw.sessions.find()

etc.

In this case, each finder will return EVERY project/subject/session that you have access to on the site.

This is equivalent to simply calling:

• fw.projects()

• fw.subjects()

• fw.sessions()

Until you place filter arguments in find()

You can pass in filters to limit the containers returned. The filters can set conditions based off of the containers’ metadata. For example, if you want to only get subjects that belong to a certain project, you can filter for that property as follows:

fw.subjects.find(‘project=<PROJECT_ID>’)

Alternatively, you can search for all subjects DIRECTLY from that project’s container object:

project = fw.get_project('<PROJECT_ID>')
subjects = project.subjects.find()
# OR
subjects = project.subjects()

Further, finders can be applied here as well:

` subject = project.subjects.find(‘created>2018-10-31’)`

Will return all subjects created after October 31st, 2018, in the given project.

find_first & iter_find¶

There are two other varieties of finders:

1. find_first()

2. iter_find()

These two behave exactly as find() in that they can be called directly from the client:

• fw.subjects.find_first()

• fw.sessions.iter_find()

or from a container:

project = fw.get_project('<PROJECT_ID>')
subjects = project.subjects.find_first()
sessions = project.sessions.iter_find()

Additionally, they can take filter arguments just like find():

fw.subjects.find_first(“label=’My Subject’”)

They differ in what they return.

*find_first* stops searching flywheel when it finds the first match, and returns that match. This can be useful to simply see if your search is working, and if ANY data is matching to it.

*iter_find* returns an iterator. This can be useful for searches that return large numbers of containers. find() will retrieve the metadata of every container that matches the search filter, generate a list of that data for python, and return that to your terminal. This can take time when hundreds or thousands of search results need to be returned. iter_find() returns an iterator that sends the metadata for the matching containers one at a time, rather than all at once.

This can be useful when there is some sort of processing loop that you’re doing to your data.

For example, appending a keyword to all subjects created after a certain date:

project = fw.get_project('<PROJECT_ID>')

# INCORRECT - This method will be slow if there is a large number of subjects
subjects = project.subjects.find('created>2019-09-09')
for sub in subjects:
    label = sub.label
    sub.update({'label':f"old_cohort_{label}"})


# CORRECT - This method will be faster than the one above if there is a large number of subjects
subjects = project.subjects.iter_find('created>2019-09-09')
for sub in subjects:
    label = sub.label
    sub.update({'label':f"old_cohort_{label}"})


sessions = project.sessions.iter_find()

Queries¶

Queries are an extension of our advanced search function in the SDK.

You can generate queries just like you’d generate an advanced search, and passing the query into the fw.search() function:

age = 64

query = f'session.age >= 0 AND session.age <= {age} AND ' \
        f'project.label = Sample AND ' \
        f'file.classification.Measurement = T1 AND ' \
        f'file.info.PatientIdentityRemoved = YES'

results = fw.search({'structured_query': query, 'return_type': 'acquisition'}, size=10000)

you must specify the query and the return type. The return type specifies which container type it will return, regardless of what the query criteria is. This will return any containers who have a parent or child container that the query criteria satisfies.

the size parameter specifies the number of results to return, typically limited to 100, with a maximum of 10000.

As an example, the following query would return all subjects in the project labeled “Test Project”:

query = 'project.label = "Test Project"'
results = fw.search({'structured_query': query, 'return_type': 'subject'}, size=10000)

And likewise, the following query would return all subjects who have a child acquisition named “multi_echo”:

query = 'acquisition.label = multi_echo'
results = fw.search({'structured_query': query, 'return_type': 'subject'}, size=10000)

View All Groups¶

To view all the groups your account has access to on this flywheel instance:

all_groups = fw.groups()

print('Flywheel Groups:')
for group in all_groups:
      print(f"\t{group.id}")

Find a Specific Group¶

If you would like to examine a specific group in more detail, you can directly load that one group with the following:

group_name = 'scien'

group = fw.get(group_name)
# Equivalently, but possibly slower:
group = fw.lookup(group_name)
group = fw.groups.find(f"_id={group_id}")

View All Projects¶

To view all the projects your account has access to:

projects = fw.projects()

print('Flywheel Projects:')
for project in projects:
      print(f"\t{project.label}")

Using a Group ID¶

To view all projects in a specific group that your account has access to using a group id:

group_id = 'scien'
projects_in_group = fw.projects.find(f"group={group_id}")

print(f"Flywheel Projects in {group_id}:")
for project in projects_in_group:
      print(f"\t{project.label}")

The flywheel finder .find() can be usd to search for values in the Project Object Metadata. Only keywords found in the object can be used in the query. More details on flywheel finders can be found here [LLL].

Using a Group’s Project List¶

If you have already captured a group object (See Find a Specific Group), you can call projects() to return all projects in that group:

group_id = 'scien'
group = fw.get(group_id)

projects_in_group = group.projects()
print(f"Flywheel Projects in {group_id}:")
for project in projects_in_group:
      print(f"\t{project.label}")

Using the Client Projects Finder¶

To view one specific project in a group using the project label and group ID:

group_id = 'scien'
project_label = 'cmrr_deid'

one_project = fw.projects.find(f"group={group_id},label={project_label}")
# Or Equivalently:
one_project = fw.projects.find_first(f"group={group_id},label={project_label}")
one_project = fw.projects.find_one(f"group={group_id},label={project_label}")
one_project = fw.lookup(f"{group_id}/{project_label}")

The differences the finders/lookup functions are covered in more detail here

Note that if a group has multiple projects with the same name, it will be easier to access a project using it’s unique Flywheel ID. If you know a project’s flywheel ID, you can access it directly using the steps in the next section:

Using a Group’s Project Finder¶

If you already have the group object loaded that the project belongs to, it may be faster to locate the project using the group.projects.find() method:

group_id = 'scien'
group = fw.get(group_id)

project_label = 'cmrr_deid'
one_project = group.projects.find(f"label={project_label})
# Or Equivalently:
one_project = group.projects.find_first(f"label={project_label})
one_project = group.projects.find_one(f"label={project_label})

Finding All Subjects in a Project¶

Using a project’s subject list¶

To view all subjects in a specific project we use the project ID to get the project object, as it contains information on all subjects within itself. If you have the project object, or know the project ID, you can print all subjects within the project as follows:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

subjects_in_project = project.subjects()
print(f"{len(subjects_in_project)} Subjects found")
for subject in subjects_in_project:
      print(f"\t{subject.label}")

A project may also have a large number of subjects, so it may be desirable to use the subject iterator to loop through them to save on time waiting for the API to respond:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

subjects_in_project = project.subjects.iter()
print(f"{len(subjects_in_project)} Subjects found")
for subject in subjects_in_project:
      print(f"\t{subject.label}")

View All Subjects in a Project That Meet Filter Criteria¶

You can use the Flywheel finders to search for subjects with specific metadata values. For example, you can search for all male subjects under the age of 30:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

subjects_in_project = project.subjects.find("age<30,sex=male")
print(f"{len(subjects_in_project)} Subjects found")
for subject in subjects_in_project:
      print(f"\t{subject.label}")

If the subject has a custom metadata field in the “info” section, that too can be used with flywheel finders. For example:

subjects_in_project = project.subjects.find("info.custom_info_field=<desired_value>")

View one subject in a Project¶

Using a Project’s Subject Finder¶

Or if you have the project object, you can query that, using only the desired subject label:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

subject_label = 'ExampleSubject'
subject_in_project = project.subjects.find(f"label={subject_label}")

View All Sessions in a Project¶

Using a Project’s Session List¶

If you already have the project object, you can view all sessions in a specific project using the project’s session list:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

sessions_in_project = project.sessions()

print(f"Flywheel Sessions in {project_id}:")
for session in sessions_in_project:
      print(f"\t{session.label}")

to call the iterator from the project:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

sessions_in_project_iterator = project.sessions.iter()

print(f"Flywheel Sessions in {project_id}:")
for session in sessions_in_project_iterator:
      print(f"\t{session.label}")

View all Sessions for one Subject¶

Using a Subject’s Session List¶

If you already have the subject object, you can get the sessions as follows:

subject_id = '5ebd50b5bfda5102996aa0e5'
subject = fw.get(subject_id)
subject_sessions = subject.sessions()

View all Sessions in a project with a specific key¶

It’s sometimes useful to retrieve all sessions of a certain type (‘Baseline’, or ‘Resting’). If this in formation is stored in the session label, it can easily be retrieved as follows:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

sessions = project.sessions.find("label=baseline")
print(f"{len(sessions)} Sessions Found:")
for ses in sessions:
    print(f"\t{ses.label})

Regular expressions can also be used with the =~ symbol, in case for some reason each subject has a slightly different session name, but that always fits a pattern.

For example, assume that every subject has two resting state scans, each performed in a different session:

If we were only interested in all baseline sessions, we could perform the following search:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

sessions = project.sessions.find("label=~.*baseline")

And if we only wanted resting state sessions, we could perform the following:

project_id = '5ebd4ff8bfda5102856aa084'
project = fw.get(project_id)

sessions = project.sessions.find("label=~resting.*")

Read more about using finders here.

From the Flywheel Instance URL¶

A session ID can be found in the flywheel URL when in the “session” tab of a project, and in session view mode, as shown below:

The session can be loaded directly with fw.get(session_id).

From a Subject’s Session Finder¶

In this example, because the subject is specified with its unique flywheel string, there is no need to specify the group and project - that information is stored within the subject.

subject_id = '5efcdf14957ea0030bc8a8e7'
ses_label = 'Resting-baseline'
subject = fw.get(subject_id)

session = subject.sessions.find(f"label={ses_label}")

View All Acquisitions in a Session¶

Using a Sessions’s Acquisition List¶

Each Session object has an “Acquisition” finder object with every acquisition for that session. To view all Acquisitions in a Session, use the following:

session_id = '5efcdf14957ea0030bc8a8e7'
session = fw.get(session_id)
acquisitions = session.acquisitions()

print(f"{len(acquisitions)} acquisitions Found:")
for acq in acquisitions:
    print(f"\t{acq.label})

Finding Specific Acquisitions¶

Using a Sessions’s Acquisition Finder¶

In Flywheel, the only container with an Acquisition finder is the Session container. A Session’s Acquisition Finder can be used to locate all acquisitions in that Session, or to filter them using the finder. For example, looking for any Acquisitions with the word “resting” in the label can be done as follows:

session_id = '5efcdf14957ea0030bc8a8e7'
session = fw.get(session_id)
acquisitions = session.acquisitions.find('label=~.*resting.*')

print(f"{len(acquisitions)} acquisitions Found:")
for acq in acquisitions:
    print(f"\t{acq.label})

Using the Client’s Acquisition Finder¶

The only other way to search for acquisitions is using the client’s finder. This can be useful if you want to find all acquisitions of a certain type in a project. For example, if you wanted to locate all acquisitions with the label “BOLD_task_checkerboard” in the project “checkerboard_study”:

project = fw.projects.find_one('label=checkerboard_study')
acquisition_label = "BOLD_task_checkerboard"
acquisitions = fw.acquisitions.find(f"parents.project={project.id},label={acquisition_label}')

print(f"{len(acquisitions)} acquisitions Found:")
for acq in acquisitions:
    print(f"\t{acq.label})

Setting Acquisition Timestamps¶

Acquisition objects have timestamp fields that indicate the date of acquisition of the file. This is separate from the created and modified fields, which only pertain to the creation and modification of the file in Flywheel.

The timestamp object requires a dateteime object, with a timezone included. Working with timezones in any programming language can be…tricky to say the least. We recommend that you review some documentation on the subject, but a simple example will be provided below:

from datetime import datetime
import pytz


# Get a session object in flywheel:
session = fw.get_session('5f185ffd957ea0084cc8a85f')

# Create an acquisition
acq = ses.add_acquisition(label='MyNewAcquisition')

# get a timezone object, in this case UTC:
timezone = pytz.utc

# Create a timestamp for when the acquisition was taken (outside of flywheel)
# datetime can specify year, month, day, hour, minute, second, millisecond, and timezone:
# Let's set the date for January 1st, 4:30am, 2020.  Better times.
acq_time = datetime(2020, 1, 1, 4, 30, tzinfo=timezone)

# Now update the acquisition's timestamp
acq.update({'timestamp': acq_time})

Update¶

Update will modify any metadta fieds specified, and leave all unspecified fields unchanged.

For example:

subject.update(sex='male', info={'key1':'val1'})

This will change (or add) the values for sex and key1 in the custom info of this subject. All other values remain unchanged.

Update_info¶

update_info will allow you to pass in only a info metadata object. Any keys passed in will be updated if they exist, or added of they do not. All unspecified keys will remain untouched.

Note that this is equivalent to passing the metadata object to update() after the info key.

example:

session.update_info({'hair': 'black', 'group': 'control', 'IQ': 107})

# Equivalent to:
session.update(info={'hair': 'black', 'group': 'control', 'IQ': 107})

Replace_info()¶

This will completely replace ALL info with the metadata object passed into this argument. This means that all existing key/value pairs in the “info” section are erased, and reaplaced with this object.

For example, regardless of any data already present in acquisiton.info, after calling this command, the only information present will be 'key1':'val1'

acquisition.replace_info({'key1':'val1'})

EXAMPLE: Set permissions for a list of users across multiple projects¶

Let’s say you have a list of user ID’s, and a permission level that they should have for all projects in a group:

NOTE that this code will assign access to EVERY project within a given group. Extreme caution must be taken when assigning user permissions to ensure that ONLY the correct access is given to the correct projects.

Group Roles¶

Roles are different from group permissions - Roles apply to Project access within a given group. Roles and permissions is a detailed topic that we suggest you review. Here we will only demonstrate how to use them. Custom roles can be made, giving you granular control over a user’s access to a project. For example, you cna create a role where a user ONLY has permission to modify metadata, but not files.

Typically, you would have to create and name your own roles before assigning them here. However, flywheel does provide three default roles that you can always use: #. admin #. read/write #. read only

Below is a quick example of how to get these default roles and add them to your group. Remember that these roles define the permissions that we can assing users for PROJECTS within this group, and do not directly indicate what permissions a user has regarding what they can do to the GROUP itself.

admin_id = [role.id for role in fw.get_all_roles() if role.default_flywheel_role == 'admin'].pop()
rw_id = [role.id for role in fw.get_all_roles() if role.default_flywheel_role == 'rw'].pop()
ro_id = [role.id for role in fw.get_all_roles() if role.default_flywheel_role == 'ro'].pop()

fw.add_role_to_group(my_group.id, {'_id': admin_id})
fw.add_role_to_group(my_group.id, {'_id': rw_id})
fw.add_role_to_group(my_group.id, {'_id': ro_id})

These are now roles we can assign to users on individual projects within the group.

Any groups you can see here, you have access to or are part of. If you see a group you’d like to use for your new project, we can grab it using the fw.get() command, and pass in the group ID (printed using the command above):

my_group = fw.get('group-id')

Or you can create a new group (if you have the correct site permissions): Note that “id” must be a lowercase string with no spaces. It’s a good idea to make this relatively short, as it will be used to access your group programmatically, and no one wants to type out ‘my_very_long_descriptive_group_id’. The group “label” on the other hand can be a human friendly name with spaces.

new_group = flywheel.Group(id='my_id', label='My Label')
my_group_id = fw.add_group(new_group)  # Returns the group id as a string
my_group = fw.get(my_group_id)

Acquisition Timestamp¶

The acquisition timestamp is a time-zone specific timestamp for when the file within the acquisition was created, or when the scan took place. This is different from the Flywheel metadata tags Created and Modified, which refer to when the container was created and modified in Flywheel. To set the acquisition timestamp, you must know the time/date/timezome when the data was acquired:

Setting file Metadata¶

If we have metadata on this file (Such as dicom header fields, or any kind of BIDS sidecar .json file), we can add it to the info section of the flywheel metadata for that file. this info will be visible in the UI under the “Custom Information” tab. The metadata must be passed in as a python dictionary. The original format of this data can be arbitrary (csv, yaml, json, etc), as long as you can load the

# Load in metadata as a python dictionary as key/value pairs.
# This can be any format, but we'll assume we have it saved as a .json file

import json


with open('file_info.json') as json_file:
    metadata = json.load(json_file)

flywheel_file.update_info(metadata)

Uploading a file to any container with metadata¶

Files can also be uploaded to a container with just the container ID. File Metadata can be uploaded concurrently as a json string with this call as well:

import json

# Get a container ID:
container = fw.subjects.find_first()
container_id = container.id

# Create a Metadata object:
md = {'info':{'hair color':'brown','motion level':'low'}}

# Upload the file with metadata
file = '/path/to/file.nii.gz'
fw.upload_file_to_container(container_id, file, metadata=json.dumps(md))