.md .pdf

repository open issue

Contents

Longitudinal microbiome analysis

In this section of the tutorial we’ll perform several analyses using QIIME 2’s q2-longitudinal [BDZ+18] plugin. These will allow us to track microbiome changes across time on a per-subject basis - something that was harder to do in the ordination plots that we viewed earlier in this tutorial.

Preparing our feature table for longitudinal analysis

Before applying these analyses, we’re going to perform some additional operations on the feature table that will make these analyses run quicker and make the results more interpretable.

First, we’re going to use the taxonomic information that we generated earlier to redefine our features as microbial genera. To do this, we group (or collapse) ASV features based on their taxonomic assignments through the genus level. This is achieved using the q2-taxa plugin’s collapse action.

genus_table, = taxa_actions.collapse(
    table=filtered_table_4,
    taxonomy=taxonomy,
    level=6,
)

qiime taxa collapse \
  --i-table filtered-table-4.qza \
  --i-taxonomy taxonomy.qza \
  --p-level 6 \
  --o-collapsed-table genus-table.qza

genus_table, = use.action(
    use.UsageAction(plugin_id='taxa', action_id='collapse'),
    use.UsageInputs(table=filtered_table_4, taxonomy=taxonomy, level=6),
    use.UsageOutputNames(collapsed_table='genus_table')
)

Using the qiime2 taxa collapse tool:

1. Set “table” to #: filtered-table-4.qza

2. Set “taxonomy” to #: taxonomy.qza

3. Set “level” to 6

4. Press the Execute button.

Once completed, for the new entry in your history, use the Edit button to set the name as follows:

(Renaming is optional, but it will make any subsequent steps easier to complete.)

History Name	“Name” to set (be sure to press Save)
#: qiime2 taxa collapse [...] : collapsed_table.qza	genus-table.qza

• genus-table.qza | view | download

Then, to focus on the genera that are likely to display the most interesting patterns over time (and to reduce the runtime of the steps that come next), we will perform even more filtering. This time we’ll apply prevalence and abudnance based filtering. Specifically, we’ll require that a genus’s abundance is at least 1% in at least 10% of the samples.

Note

The prevalence-based filtering applied here is fairly stringent. In your own analyses you may want to experiment with relaxed settings of these parameters. Because we need the commands below to run quickly, stringent filtering is helpful for the workshop.

filtered_genus_table, = feature_table_actions.filter_features_conditionally(
    table=genus_table,
    prevalence=0.1,
    abundance=0.01,
)

qiime feature-table filter-features-conditionally \
  --i-table genus-table.qza \
  --p-prevalence 0.1 \
  --p-abundance 0.01 \
  --o-filtered-table filtered-genus-table.qza

filtered_genus_table, = use.action(
    use.UsageAction(plugin_id='feature_table', action_id='filter_features_conditionally'),
    use.UsageInputs(table=genus_table, prevalence=0.1, abundance=0.01),
    use.UsageOutputNames(filtered_table='filtered_genus_table')
)

Using the qiime2 feature-table filter-features-conditionally tool:

1. Set “table” to #: genus-table.qza

2. Set “abundance” to 0.01

3. Set “prevalence” to 0.1

4. Press the Execute button.

Once completed, for the new entry in your history, use the Edit button to set the name as follows:

(Renaming is optional, but it will make any subsequent steps easier to complete.)

History Name	“Name” to set (be sure to press Save)
#: qiime2 feature-table filter-features-conditionally [...] : filtered_table.qza	filtered-genus-table.qza

• filtered-genus-table.qza | view | download

Finally, we’ll convert the counts in our feature table to relative frequencies. This is required for some of the analyses that we’re about to perform.

genus_rf_table, = feature_table_actions.relative_frequency(
    table=filtered_genus_table,
)

qiime feature-table relative-frequency \
  --i-table filtered-genus-table.qza \
  --o-relative-frequency-table genus-rf-table.qza

genus_rf_table, = use.action(
    use.UsageAction(plugin_id='feature_table', action_id='relative_frequency'),
    use.UsageInputs(table=filtered_genus_table),
    use.UsageOutputNames(relative_frequency_table='genus_rf_table')
)

Using the qiime2 feature-table relative-frequency tool:

1. Set “table” to #: filtered-genus-table.qza

2. Press the Execute button.

Once completed, for the new entry in your history, use the Edit button to set the name as follows:

(Renaming is optional, but it will make any subsequent steps easier to complete.)

History Name	“Name” to set (be sure to press Save)
#: qiime2 feature-table relative-frequency [...] : relative_frequency_table.qza	genus-rf-table.qza

• genus-rf-table.qza | view | download

Volatility plots

The first plots we’ll generate are volatility plots. We’ll generate these using two different time variables. First, we’ll plot based on week-relative-to-hct.

volatility_plot_1_viz, = longitudinal_actions.volatility(
    table=genus_rf_table,
    state_column='week-relative-to-hct',
    metadata=expanded_sample_metadata_md,
    individual_id_column='PatientID',
    default_group_column='autoFmtGroup',
)

qiime longitudinal volatility \
  --i-table genus-rf-table.qza \
  --p-state-column week-relative-to-hct \
  --m-metadata-file sample-metadata.tsv uu-umap.qza diversity-core-metrics-phylogenetic/faith_pd_vector.qza diversity-core-metrics-phylogenetic/evenness_vector.qza diversity-core-metrics-phylogenetic/shannon_vector.qza \
  --p-individual-id-column PatientID \
  --p-default-group-column autoFmtGroup \
  --o-visualization volatility-plot-1.qzv

use.action(
    use.UsageAction(plugin_id='longitudinal', action_id='volatility'),
    use.UsageInputs(table=genus_rf_table, state_column='week-relative-to-hct',
                    metadata=expanded_sample_metadata, individual_id_column='PatientID',
                    default_group_column='autoFmtGroup'),
    use.UsageOutputNames(visualization='volatility_plot_1'),
)

Using the qiime2 longitudinal volatility tool:

1. For “metadata”:

   1. Perform the following steps.

      1. Leave as Metadata from TSV

      2. Set “Metadata Source” to sample-metadata.tsv

   2. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to uu-umap.qza

   3. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : faith_pd_vector.qza

   4. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : evenness_vector.qza

   5. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : shannon_vector.qza

2. Set “state_column” to week-relative-to-hct

3. Expand the additional options section

   1. Set “table” to #: genus-rf-table.qza

   2. Set “individual_id_column” to PatientID

   3. Set “default_group_column” to autoFmtGroup

4. Press the Execute button.

Once completed, for the new entry in your history, use the Edit button to set the name as follows:

(Renaming is optional, but it will make any subsequent steps easier to complete.)

History Name	“Name” to set (be sure to press Save)
#: qiime2 longitudinal volatility [...] : visualization.qzv	volatility-plot-1.qzv

• volatility-plot-1.qzv | view | download

Next, we’ll plot based on week-relative-to-fmt.

volatility_plot_2_viz, = longitudinal_actions.volatility(
    table=genus_rf_table,
    state_column='week-relative-to-fmt',
    metadata=expanded_sample_metadata_md,
    individual_id_column='PatientID',
    default_group_column='autoFmtGroup',
)

qiime longitudinal volatility \
  --i-table genus-rf-table.qza \
  --p-state-column week-relative-to-fmt \
  --m-metadata-file sample-metadata.tsv uu-umap.qza diversity-core-metrics-phylogenetic/faith_pd_vector.qza diversity-core-metrics-phylogenetic/evenness_vector.qza diversity-core-metrics-phylogenetic/shannon_vector.qza \
  --p-individual-id-column PatientID \
  --p-default-group-column autoFmtGroup \
  --o-visualization volatility-plot-2.qzv

use.action(
    use.UsageAction(plugin_id='longitudinal', action_id='volatility'),
    use.UsageInputs(table=genus_rf_table, state_column='week-relative-to-fmt',
                    metadata=expanded_sample_metadata, individual_id_column='PatientID',
                    default_group_column='autoFmtGroup'),
    use.UsageOutputNames(visualization='volatility_plot_2'),
)

Using the qiime2 longitudinal volatility tool:

1. For “metadata”:

   1. Perform the following steps.

      1. Leave as Metadata from TSV

      2. Set “Metadata Source” to sample-metadata.tsv

   2. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to uu-umap.qza

   3. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : faith_pd_vector.qza

   4. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : evenness_vector.qza

   5. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : shannon_vector.qza

2. Set “state_column” to week-relative-to-fmt

3. Expand the additional options section

   1. Set “table” to #: genus-rf-table.qza

   2. Set “individual_id_column” to PatientID

   3. Set “default_group_column” to autoFmtGroup

4. Press the Execute button.

Once completed, for the new entry in your history, use the Edit button to set the name as follows:

(Renaming is optional, but it will make any subsequent steps easier to complete.)

History Name	“Name” to set (be sure to press Save)
#: qiime2 longitudinal volatility [...] : visualization.qzv	volatility-plot-2.qzv

• volatility-plot-2.qzv | view | download

Feature volatility

The last plots we’ll generate in this section will come from a QIIME 2 pipeline called feature-volatility. These use supervised regression to identify features that are most associated with changes over time, and add plotting of those features to a volatility control chart.

Again, we’ll generate the same plots but using two different time variables on the x-axes. First, we’ll plot based on week-relative-to-hct.

action_results = longitudinal_actions.feature_volatility(
    table=filtered_genus_table,
    metadata=expanded_sample_metadata_md,
    state_column='week-relative-to-hct',
    individual_id_column='PatientID',
)
important_genera_table_1 = action_results.filtered_table
genus_importances_1 = action_results.feature_importance
genus_volatility_plot_1_viz = action_results.volatility_plot
accuracy_results_1_viz = action_results.accuracy_results
sample_estimator_1 = action_results.sample_estimator

qiime longitudinal feature-volatility \
  --i-table filtered-genus-table.qza \
  --m-metadata-file sample-metadata.tsv uu-umap.qza diversity-core-metrics-phylogenetic/faith_pd_vector.qza diversity-core-metrics-phylogenetic/evenness_vector.qza diversity-core-metrics-phylogenetic/shannon_vector.qza \
  --p-state-column week-relative-to-hct \
  --p-individual-id-column PatientID \
  --output-dir longitudinal-feature-volatility

use.action(
    use.UsageAction(plugin_id='longitudinal', action_id='feature_volatility'),
    use.UsageInputs(table=filtered_genus_table, metadata=expanded_sample_metadata,
                    state_column='week-relative-to-hct', individual_id_column='PatientID'),
    use.UsageOutputNames(filtered_table='important_genera_table_1',
                         feature_importance='genus_importances_1',
                         volatility_plot='genus_volatility_plot_1',
                         accuracy_results='accuracy_results_1',
                         sample_estimator='sample_estimator_1')
)

Using the qiime2 longitudinal feature-volatility tool:

1. Set “table” to #: filtered-genus-table.qza

2. For “metadata”:

   1. Perform the following steps.

      1. Leave as Metadata from TSV

      2. Set “Metadata Source” to sample-metadata.tsv

   2. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to uu-umap.qza

   3. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : faith_pd_vector.qza

   4. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : evenness_vector.qza

   5. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : shannon_vector.qza

3. Set “state_column” to week-relative-to-hct

4. Expand the additional options section

   • Set “individual_id_column” to PatientID

5. Press the Execute button.

Once completed, for each new entry in your history, use the Edit button to set the name as follows:

(Renaming is optional, but it will make any subsequent steps easier to complete.)

History Name	“Name” to set (be sure to press Save)
#: qiime2 longitudinal feature-volatility [...] : filtered_table.qza	important-genera-table-1.qza
#: qiime2 longitudinal feature-volatility [...] : feature_importance.qza	genus-importances-1.qza
#: qiime2 longitudinal feature-volatility [...] : volatility_plot.qzv	genus-volatility-plot-1.qzv
#: qiime2 longitudinal feature-volatility [...] : accuracy_results.qzv	accuracy-results-1.qzv
#: qiime2 longitudinal feature-volatility [...] : sample_estimator.qza	sample-estimator-1.qza

• longitudinal-feature-volatility/filtered_table.qza | view | download

• longitudinal-feature-volatility/feature_importance.qza | view | download

• longitudinal-feature-volatility/volatility_plot.qzv | view | download

• longitudinal-feature-volatility/accuracy_results.qzv | view | download

• longitudinal-feature-volatility/sample_estimator.qza | view | download

Next, we’ll plot based on week-relative-to-fmt.

action_results = longitudinal_actions.feature_volatility(
    table=filtered_genus_table,
    metadata=expanded_sample_metadata_md,
    state_column='week-relative-to-fmt',
    individual_id_column='PatientID',
)
important_genera_table_2 = action_results.filtered_table
genus_importances_2 = action_results.feature_importance
genus_volatility_plot_2_viz = action_results.volatility_plot
accuracy_results_2_viz = action_results.accuracy_results
sample_estimator_2 = action_results.sample_estimator

qiime longitudinal feature-volatility \
  --i-table filtered-genus-table.qza \
  --m-metadata-file sample-metadata.tsv uu-umap.qza diversity-core-metrics-phylogenetic/faith_pd_vector.qza diversity-core-metrics-phylogenetic/evenness_vector.qza diversity-core-metrics-phylogenetic/shannon_vector.qza \
  --p-state-column week-relative-to-fmt \
  --p-individual-id-column PatientID \
  --output-dir longitudinal-feature-volatility-2

use.action(
    use.UsageAction(plugin_id='longitudinal', action_id='feature_volatility'),
    use.UsageInputs(table=filtered_genus_table, metadata=expanded_sample_metadata,
                    state_column='week-relative-to-fmt', individual_id_column='PatientID'),
    use.UsageOutputNames(filtered_table='important_genera_table_2',
                         feature_importance='genus_importances_2',
                         volatility_plot='genus_volatility_plot_2',
                         accuracy_results='accuracy_results_2',
                         sample_estimator='sample_estimator_2')
)

Using the qiime2 longitudinal feature-volatility tool:

1. Set “table” to #: filtered-genus-table.qza

2. For “metadata”:

   1. Perform the following steps.

      1. Leave as Metadata from TSV

      2. Set “Metadata Source” to sample-metadata.tsv

   2. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to uu-umap.qza

   3. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : faith_pd_vector.qza

   4. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : evenness_vector.qza

   5. Press the + Insert metadata button to set up the next steps.

      1. Change to Metadata from Artifact

      2. Set “Metadata Source” to qiime2 diversity core-metrics-phylogenetic [...] : shannon_vector.qza

3. Set “state_column” to week-relative-to-fmt

4. Expand the additional options section

   • Set “individual_id_column” to PatientID

5. Press the Execute button.

Once completed, for each new entry in your history, use the Edit button to set the name as follows:

(Renaming is optional, but it will make any subsequent steps easier to complete.)

History Name	“Name” to set (be sure to press Save)
#: qiime2 longitudinal feature-volatility [...] : filtered_table.qza	important-genera-table-2.qza
#: qiime2 longitudinal feature-volatility [...] : feature_importance.qza	genus-importances-2.qza
#: qiime2 longitudinal feature-volatility [...] : volatility_plot.qzv	genus-volatility-plot-2.qzv
#: qiime2 longitudinal feature-volatility [...] : accuracy_results.qzv	accuracy-results-2.qzv
#: qiime2 longitudinal feature-volatility [...] : sample_estimator.qza	sample-estimator-2.qza

• longitudinal-feature-volatility-2/filtered_table.qza | view | download

• longitudinal-feature-volatility-2/feature_importance.qza | view | download

• longitudinal-feature-volatility-2/volatility_plot.qzv | view | download

• longitudinal-feature-volatility-2/accuracy_results.qzv | view | download

• longitudinal-feature-volatility-2/sample_estimator.qza | view | download

Wrapping up: what did you do five days ago?

Exercise 6

We’re now getting close to the end of our analysis. Over the course of the tutorial we’ve applied multiple filtering steps to our feature table, and if we were writing this work up for publication it would be essential to accurately describe the filtering steps that we applied, the order in which we applied them, and the parameters that were used for each filter.

Without referring back to the earlier sections of the tutorial, but rather using your memory or information from the results that you’ve generated, describe all filtering steps that were applied to the feature table in the order that we applied them.

Solution to Exercise 6

This information is contained in the provenance of the artifacts (.qza files) and visualizations (.qzv files) that were generated in this tutorial. To get a comprehensive view of the analyses that we ran, you can load one of the most recent visualizations that were created (e.g., a volatility or feature volatility plot from the last section of the tutorial) with QIIME 2 View, and then click the Provenance tab in QIIME 2 View to review the steps that led to that particular visualization.

We ran the following filtering steps in this tutorial:

1. qiime2 feature-table filter-samples was applied to remove samples that were not assigned to an autoFmtGroup.

2. qiime2 feature-table filter-samples was applied to remove samples that were collected outside the range of -10 through +70 days, relative to the nearest HCT event.

3. qiime2 feature-table filter-features was applied to remove features that were present in fewer than two samples.

4. qiime2 taxa filter-table was applied to remove features that were not assigned to a named phylum and sequences with annotations that included the word Chloroplast or Mitochondria.

5. qiime2 feature-table filter-samples was applied to remove samples that had a total frequency of fewer than 10,000 sequences.

6. qiime2 feature-table filter-features-conditionally was applied to remove features that were not present in at least 1% abundance in at least 10% of the samples.

That’s a lot of filtering! Thankfully QIIME 2’s data provenance tracking system keeps track of it all for us in case we forget.

previous

Beta diversity visualizations

next

Other approaches for exploring the impact of FMT on the microbiome