AGAT

Another Gtf/Gff Analysis Toolkit

Suite of tools to handle gene annotations in any GTF/GFF format.

Documentation >>here<<
Previous documentation until v1.4.0 (readthedocs) here

Table of Contents

• What can AGAT do for you?
• Installation
  • Using Docker
  • Using Singularity
  • Using Bioconda
    • Install AGAT
    • Update AGAT
    • Uninstall AGAT
  • Old school - Manually
    • Install prerequisites
    • Install AGAT
    • Update AGAT
    • Change to a specific version
    • Uninstall AGAT
• Usage
• List of tools
• More about the tools
• The AGAT parser - Standardisation to create GXF files compliant to any tool
  • The data structure
  • How does the AGAT parser work
  • What can the AGAT parser do for you
  • examples
• How to cite?
• Publication using AGAT
• Troubleshooting

What can AGAT do for you?

AGAT has the power to check, fix, pad missing information (features/attributes) of any kind of GTF and GFF to create complete, sorted and standardised gff3 format. Over the years it has been enriched by many many tools to perform just about any tasks that is possible related to GTF/GFF format files (sanitizing, conversions, merging, modifying, filtering, FASTA sequence extraction, adding information, etc). Comparing to other methods AGAT is robust to even the most despicable GTF/GFF files.

• Standardize/sanitize any GTF/GFF file into a comprehensive GFF3 format (script with _sp_ prefix)

  
  

  See standardization/sanitization tool

  
  
  | task | tool |
  | —- | —- |
  | check, fix, pad missing information into sorted and standardised gff3 | agat_convert_sp_gxf2gxf.pl |
  
  add missing parent features (e.g. gene and mRNA if only CDS/exon exists). add missing features (e.g. exon and UTR).
  add missing mandatory attributes (i.e. ID, Parent). fix identifiers to be uniq.
  fix feature locations. remove duplicated features.
  group related features (if spread in different places in the file). sort features (tabix optional).
  * merge overlapping loci into one single locus (only if option activated).
  

• Convert many formats

  
  

  See conversion tools

  
  
  | task | tool |
  | —- | —- |
  | convert any GTF/GFF into BED format | agat_convert_sp_gff2bed.pl |
  | convert any GTF/GFF into GTF format | agat_convert_sp_gff2gtf.pl |
  | convert any GTF/GFF into tabulated format | agat_sp_gff2tsv.pl |
  | convert any BAM from minimap2 into GFF format | agat_convert_sp_minimap2_bam2gff.pl |
  | convert any GTF/GFF into ZFF format | agat_sp_gff2zff.pl |
  | convert any GTF/GFF into any GTF/GFF (bioperl) format | agat_convert_sp_gxf2gxf.pl |
  | convert BED format into GFF3 format | agat_convert_bed2gff.pl |
  | convert EMBL format into GFF3 format | agat_convert_embl2gff.pl |
  | convert genscan format into GFF3 format | agat_convert_genscan2gff.pl |
  | convert mfannot format into GFF3 format | agat_convert_mfannot2gff.pl |
  

• Perform numerous tasks (Just about anything that is possible)

  
  

  See tools

  
  
  | task | tool |
  | —- | —- |
  | make feature statistics | agat_sp_statistics.pl |
  | make function statistics | agat_sp_functional_statistics.pl |
  | extract any type of sequence | agat_sp_extract_sequences.pl |
  | extract attributes | agat_sp_extract_attributes.pl |
  | complement annotations (non-overlapping loci) | agat_sp_complement_annotations.pl |
  | merge annotations | agat_sp_merge_annotations.pl |
  | filter gene models by ORF size | agat_sp_filter_by_ORF_size.pl |
  | filter to keep only longest isoforms | agat_sp_keep_longest_isoform.pl |
  | create introns features | agat_sp_add_introns.pl |
  | fix cds phases | agat_sp_fix_cds_phases.pl |
  | manage IDs | agat_sp_manage_IDs.pl |
  | manage UTRs | agat_sp_manage_UTRs.pl |
  | manage introns | agat_sp_manage_introns.pl |
  | manage functional annotation | agat_sp_manage_functional_annotation.pl |
  | specificity sensitivity | agat_sp_sensitivity_specificity.pl |
  | fusion / split analysis between two annotations | agat_sp_compare_two_annotations.pl |
  | analyze differences between BUSCO results | agat_sp_compare_two_BUSCOs.pl |
  | … and much more …| … see here …|
  

About the GTF/GFF fromat
The GTF/GFF formats are 9-column text formats used to describe and represent genomic features.
The formats have quite evolved since 1997, and despite well-defined specifications existing nowadays they have a great flexibility allowing holding wide variety of information.
This flexibility has a drawback aspect, there is an incredible amount of flavour of the formats, that can result in problems when using downstream programs.
For a complete overview of the GTF/GFF formats have a look here.

Installation

Using Docker

Using Singularity

Using Bioconda

Old school - Manually

Usage

  script_name.pl -h

List of tools

As AGAT is a toolkit, it contains a lot of tools. The main one is agat_convert_sp_gxf2gxf.pl that allows to check, fix, pad missing information (features/attributes) of any kind of gtf and gff to create complete, sorted and standardised gff3 format.
All the installed scripts have the agat_ prefix.

To have a look to the available tools you have several approaches:

• agat --tools
• Typing agat_ in your terminal followed by the key to activate the autocompletion will display the complete list of available tools.
• The documentation.

More about the tools

with _sp_ prefix => Means SLURP

The gff file will be charged in memory in a specific data structure facilitating the access to desired features at any time.
It has a memory cost but make life smoother. Indeed, it allows to perform complicated tasks in a more time efficient way.
Moreover, it allows to fix all potential errors in the limit of the possibilities given by the format itself.
See the AGAT parser section for more information about it.

with _sq_ prefix => Means SEQUENTIAL

The gff file is read and processed from its top to the end line by line without sanity check (e.g. relationship between the features). This is memory efficient.

The AGAT parser - Standardisation to create GXF files compliant to any tool

All tools with agat_sp_ prefix will parse and slurps the entire data into a specific data structure.
Below you will find more information about peculiarity of the data structure,
and the parsing approach used.

the data structure

How does the AGAT parser work

The AGAT parser phylosophy:

• 1) Parse by Parent/child relationship or gene_id/transcript_id relationship.
• 2) ELSE Parse by a common tag (an attribute value shared by feature that must be grouped together. By default we are using locus_tag but can be set by parameter).
• 3) ELSE Parse sequentially (mean group features in a bucket, and the bucket change at each level2 feature, and bucket are join in a common tag at each new L1 feature).

/!\ Cases with only level3 features (i.e rast or some prokka files), sequential parsing may not work as expected if Parent/ID gene_id/transcript_id attributes are missing. Indeed all features will be the child of only one newly created Parent. To create a parent per feature or group of features, a common tag must be used to group them correctly (by default gene_id and locus_tag but you can set up the ones of your choice)

To resume by priority of way to parse: Parent/child relationship > locus_tag > sequential.
The parser may used only one or a mix of these approaches according of the peculiarity of the gtf/gff file you provide.

What can the AGAT parser do for you

• It creates missing parental features. (e.g if a level2 or level3 feature do not have parental feature(s) we create the missing level2 and/or level1 feature(s)).
• It creates missing mandatory attributes (ID and/or Parent).
• It fixes identifier to be uniq.
• It removes duplicated features (same position, same ID, same Parent).
• It expands level3 features sharing multiple parents (e.g if one exon has list of multiple parent mRNA in its Parent attribute, one exon per parent with uniq ID will be created.
• It fixes feature location errors (e.g an mRNA spanning over its gene location, we fix the gene location).
• It adds UTR if possible (CDS and exon present).
• It adds exon if possible (CDS has to be present).
• It groups features together (if related features are spread at different places in the file).

examples

AGAT was tested on 42 different types of GTF/GFF of different flavours or/and containing errors.
Below few are listed but you can find the full list of them into the t/gff_syntax directory.

example 8 - only CDS defined

agat_convert_sp_gxf2gxf.pl --gff 8_test.gff

example 9 - level2 feature missing (mRNA) and level3 features missing (UTRs)

agat_convert_sp_gxf2gxf.pl --gff 8_test.gff

example 18 - related features spread within the file

agat_convert_sp_gxf2gxf.pl --gff 18_test.gff

How to cite?

This work has not been published (I will think about it) but you can cite it as follow:

Dainat J. 2022. Another Gtf/Gff Analysis Toolkit (AGAT): Resolve interoperability issues and accomplish more with your annotations. Plant and Animal Genome XXIX Conference. https://github.com/NBISweden/AGAT.

or/and (Adapt the AGAT version to the one you used):

Dainat J. AGAT: Another Gff Analysis Toolkit to handle annotations in any GTF/GFF format.  
(Version v1.4.1). Zenodo. https://www.doi.org/10.5281/zenodo.3552717

Publication using AGAT

See here for examples of publications using AGAT.

Troubleshooting

See Troubleshooting section form the doc here.