Synchronizing study data

Synchronizing studies with Terra workspaces

What does synchronizing do?

If you add or edit data directly to your Terra (formerly, FireCloud) workspace outside of the portal, and would like to 'sync' the study with the workspace, you may do so in the portal. This will have several effects on your study in the portal:

• Any files found in the workspace that do not have a matching entry in your study will be added to your study (you will have a chance to choose which files to include and provide details about each one before saving).

• Your study permissions (i.e. people you have shared your study with) will be made to match the permissions of the Terra workspace. In other words, if you have shared your workspace with someone outside of the portal, a matching share will be added to the portal for your study. The converse is also true - if you remove a user's sharing of your workspace in Terra, then their share of your study in the portal will be removed as well.

How to synchronize a study

Option 1: Add a new study

After logging in, select '+ Add a study' from the profile menu.

Use an existing Terra workspace

After you enter all of the normal information for a study (name, description, embargo date, public/private), select 'Yes' for 'Use an existing workspace?' and enter the name of your Terra workspace in the text field.

NOTE: The workspace must be in the 'single-cell-portal' Terra project to be used with the portal. Please email [e-mail] for help creating a workspace in this project.

Save the new study

Click 'Create Study' to save your study.

Option 2: Synchronize an existing study

Log in and view the list of studies

After logging in, select 'My Studies' from the profile dropdown.

Synchronize your study

Click 'Sync Workspace' for the corresponding study.

You will see a dialog while the portal is synchronizing your study. Depending on how many files there are in your workspace, this may take a little while.

View list of discovered files

You will see a list of all files that were found in the workspace that the portal does not have a record of.

Fill out forms for new files

For every file found in the workspace, you will be prompted to provide details for each one. Depending on what type of file it is, the forms will update to match your selection.

NOTE: If you do not want a file to show up in your study, you can either leave the form blank or click "Don't Sync" to remove the form from the page. The portal will ignore that file for now. You can add it at a later date if you wish by re-running the 'Sync Workspace' function for this study.

Cluster Files

If you select 'Cluster', you will be prompted to provide a name (what the cluster will be labeled), a description, labels for each axis, and optionally the domain ranges of each axis. When you are finished, click 'Sync' to save your entry.

After you sync the file, you will see a dialog notifying you that the file is being loaded into the portal and that you will receive an email when it has been completed.

Expression matrices

If you select 'Expression Matrix', you will be prompted to provide a description and a label to be used when plotting expression scores (will be an axis label). When you are finished, click 'Sync' to save your entry. You will see the same kind of completion dialog as with cluster files regarding email notifications.

Metadata files

If you select 'Metadata', you will be prompted to provide a description of the file. When you are finished, click 'Sync' to save your entry. You will see the same kind of completion dialog as with cluster files regarding email notifications.

Gene lists

If you select 'Gene List', you will be prompted to provide a name for the list and a description. When you are finished, click 'Sync' to save your entry. You will see the same kind of completion dialog as with cluster files regarding email notifications.

Documentation or other files

If you select 'Documentation' or 'Other', you will be prompted to provide a description for the file. When you are finished, click 'Sync' to save your entry. Since these files are not parsed and loaded into the portal, there is no email notification as the synchronization is already complete.

Primary data directories

If you have uploaded multiple FASTQ files to your workspace, rather than entering information about each file individually, the portal will instead group files together by their directory path inside the Google storage bucket in your workspace. You will see a form for each group of files showing you the location where they were found and the number of files found there. You will be able to provide a description that will be used for all files found at that location.

In other words, if you had 3 directories at the locations /sample_a, /sample_b, and /sample_c, you would see forms for each location. NOTE: The portal currently only supports one level of directory nesting. Any subdirectories below the first will not be listed as separate entries.

Synchronizing orphaned files

If you have previously entered information about a file in your study that the portal cannot find a matching entry for in your workspace, you will see an 'orphaned' file form instead of the normal one. You can view these by clicking the 'Orphaned Study Files' header. Here, you will have the option of designating which new file in your workspace the old entry was supposed to point at. This is useful if you upload a new version of the file directly into the workspace and simply wish to update your entry in the portal to use that new version.

You will also have the option to 'reparse' the file (re-load the data from the file back into the portal) if the contents have changed. This will remove any of the old entries that were originally loaded from the previous file and replace them with the new ones from your new version. Once you click 'Sync', the entry will be updated (and re-loaded if necessary). You will see the normal confirmation dialog once that has been completed.

Review synchronized files

After you have completed entering information about all of your files, you can review the current information for each file as it is now by clicking the 'Synced Data' header. You can also use these forms to make any edits to the files (like names, descriptions, etc.) if you wish.