'\" t
.TH samtools-markdup 1 "12 September 2024" "samtools-1.21" "Bioinformatics tools"
.SH NAME
samtools markdup \- mark duplicate alignments in a coordinate sorted file
.\"
.\" Copyright (C) 2008-2011, 2013-2024 Genome Research Ltd.
.\" Portions copyright (C) 2010, 2011 Broad Institute.
.\"
.\" Author: Heng Li <lh3@sanger.ac.uk>
.\" Author: Joshua C. Randall <jcrandall@alum.mit.edu>
.\"
.\" Permission is hereby granted, free of charge, to any person obtaining a
.\" copy of this software and associated documentation files (the "Software"),
.\" to deal in the Software without restriction, including without limitation
.\" the rights to use, copy, modify, merge, publish, distribute, sublicense,
.\" and/or sell copies of the Software, and to permit persons to whom the
.\" Software is furnished to do so, subject to the following conditions:
.\"
.\" The above copyright notice and this permission notice shall be included in
.\" all copies or substantial portions of the Software.
.\"
.\" THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
.\" IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
.\" FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
.\" THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
.\" LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
.\" FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
.\" DEALINGS IN THE SOFTWARE.
.
.\" For code blocks and examples (cf groff's Ultrix-specific man macros)
.de EX

.  in +\\$1
.  nf
.  ft CR
..
.de EE
.  ft
.  fi
.  in

..
.
.SH SYNOPSIS
.PP
samtools markdup
.RB [ -l
.IR length ]
.RB [ -r ]
.RB [ -T ]
.RB [ -S ]
.RB [ -s ]
.RB [ -f
.IR file ]
.RB [ --json ]
.RB [ -d
.IR distance ]
.RB [ -c ]
.RB [ -t ]
.RB [ ---duplicate-count ]
.RB [ -m ]
.RB [ --mode ]
.RB [ --include-fails ]
.RB [ --no-PG ]
.RB [ -u ]
.RB [ --no-multi-dup ]
.RB [ --read-coords ]
.RB [ --coords-order ]
.RB [ --barcode-tag ]
.RB [ --barcode-name ]
.RB [ --barcode-rgx ]
.RB [ --use-read-groups ]
.I in.algsort.bam out.bam

.SH DESCRIPTION
.PP
Mark duplicate alignments from a coordinate sorted file that
has been run through \fBsamtools fixmate\fR with the \fB-m\fR option.  This
program relies on the MC and ms tags that fixmate provides.

Duplicates are found by using the alignment data for each read (and its mate for
paired reads).  Position and orientation (which strand it aligns against and in
what direction) are used to to compare the reads against one another. If two (or
more) reads have the same values then the one with the highest base qualities is
held to be the original and the others are the duplicates.

It should be noted that \fBsamtools markdup\fR looks for duplication first and
then classifies the type of duplication afterwards. If your process does not
care whether duplication is PCR or optical then it is faster if you do not use
the optical duplicate option.

Duplicates are marked by setting the alignment's DUP flag.

For more details please see:

<http://www.htslib.org/algorithms/duplicate.html>

.SH OPTIONS
.TP 11
.BI "-l " INT
.RI "Expected maximum read length of " INT " bases."
[300]
.TP
.B -r
Remove duplicate reads.
.TP
.BI "-T " PREFIX
Write temporary files to
.IB PREFIX . samtools . nnnn . mmmm . tmp
.TP
.B -S
Mark supplementary reads of duplicates as duplicates.
.TP
.B -s
Print some basic stats. See STATISTICS.
.TP
.BI "-f " file
Write stats to named file.
.TP
.B --json
Output stats in JSON format.
.TP
.BI "-d " distance
The optical duplicate distance.  Suggested settings of 100 for HiSeq style
platforms or about 2500 for NovaSeq ones.  Default is 0 to not look for
optical duplicates.  When set, duplicate reads are tagged with \fBdt:Z:SQ\fR for
optical duplicates and \fBdt:Z:LB\fR otherwise.  Calculation of distance depends
on coordinate data embedded in the read names produced by the Illumina
sequencing machines.  Optical duplicate detection will not work on non standard
names without the use of \fB--read-coords\fR.
.TP
.B -c
Clear previous duplicate settings and tags.
.TP
.B -t
Mark duplicates with the name of the original in a \fBdo\fR tag.
.TP
.B --duplicate-count
Record the original primary read duplication count (including itself) in a
\fBdc\fR tag.
.TP
.BI "-m, --mode " TYPE
Duplicate decision method for paired reads.  Values are \fBt\fR or \fBs\fR.
Mode \fBt\fR measures positions based on template start/end (default).
Mode \fBs\fR measures positions based on sequence start.
While the two methods identify mostly the same reads as duplicates, mode 
\fBs\fR tends to return more results.  Unpaired reads are treated identically
by both modes.
.TP
.B -u
Output uncompressed SAM, BAM or CRAM.
.TP
.B --include-fails
Include quality checked failed reads.
.TP
.B --no-multi-dup
Stop checking duplicates of duplicates for correctness.  While still marking
reads as duplicates further checks to make sure all optical duplicates are found
are not carried out.  Also operates on \fB-t\fR tagging where reads may tagged
with a better quality read but not necessarily the best one.  Using this option
can speed up duplicate marking when there are a great many duplicates for each
original read. 
.TP
.BI "--read-coords " REGEX
This takes a POSIX regular expression for at least x and y to be used in
optical duplicate marking It can also include another part of the read name
to test for equality, eg lane:tile elements. Elements wanted are captured with
parentheses.  Examples below.
.TP
.BI "--coords-order " ORDER
The order of the elements captured in the regular expression. Default is txy
where t is a part of the read name selected for string comparison and x/y the
coordinates used for optical duplicate detection.  Valid orders are: txy, tyx,
xyt, yxt, xty, ytx, xy and yx.
.TP
.BI "--barcode-tag " TAG
Duplicates must now also match the barcode tag.
.TP
.B --barcode-name
Use the UMI/barcode embedded in the read name (eigth colon delimited part).
.TP
.BI "--barcode-rgx " REGEX
Regex for barcode in the readname (alternative to --barcode-name).
.TP
.B --use-read-groups
The @RG tags must now also match to be a duplicate.
.TP
.B --no-PG
Do not add a PG line to the output file.
.TP
.BI "-@, --threads " INT
Number of input/output compression threads to use in addition to main thread [0].

.SH STATISTICS
Entries are:
.br
\fBCOMMAND\fR: the command line.
.br
\fBREAD\fR: number of reads read in.
.br
\fBWRITTEN\fR: reads written out.
.br
\fBEXCLUDED\fR: reads ignored.  See below.
.br
\fBEXAMINED\fR: reads examined for duplication.
.br
\fBPAIRED\fR: reads that are part of a pair.
.br
\fBSINGLE\fR: reads that are not part of a pair.
.br
\fBDUPLICATE PAIR\fR: reads in a duplicate pair.
.br
\fBDUPLICATE SINGLE\fR: single read duplicates.
.br
\fBDUPLICATE PAIR OPTICAL\fR: optical duplicate paired reads.
.br
\fBDUPLICATE SINGLE OPTICAL\fR: optical duplicate single reads.
.br
\fBDUPLICATE NON PRIMARY\fR: supplementary/secondary duplicate reads.
.br
\fBDUPLICATE NON PRIMARY OPTICAL\fR: supplementary/secondary optical
duplicate reads.
.br
\fBDUPLICATE PRIMARY TOTAL\fR: number of primary duplicate reads.
.br
\fBDUPLICATE TOTAL\fR: total number of duplicate reads.
.br
\fBESTIMATED LIBRARY SIZE\fR: estimate of the number of unique fragments in the
sequencing library.


Estimated library size makes various assumptions e.g. the library consists of
unique fragments that are randomly selected (with replacement) with equal
probability.
This is unlikely to be true in practice.
However it can provide a useful guide into how many unique read pairs are likely 
to be available.
In particular it can be used to determine how much more data might be obtained
by further sequencing of the library.

Excluded reads are those marked as secondary, supplementary or unmapped.
By default QC failed reads are also excluded but can be included as an option.
Excluded reads are not used for calculating duplicates.
They can optionally be marked as duplicates if they have a primary that is also
a duplicate.    
.SH EXAMPLES
This first collate command can be omitted if the file is already name ordered or collated:
.EX 4
samtools collate -o namecollate.bam example.bam
.EE

Add ms and MC tags for markdup to use later:
.EX 4
samtools fixmate -m namecollate.bam fixmate.bam
.EE

Markdup needs position order:
.EX 4
samtools sort -o positionsort.bam fixmate.bam
.EE

Finally mark duplicates:
.EX 4
samtools markdup positionsort.bam markdup.bam
.EE

Typically the fixmate step would be applied immediately after sequence
alignment and the markdup step after sorting by chromosome and
position.  Thus no \fIadditional\fR sort steps are normally needed.

To use the regex to obtain coordinates from reads, two or three values have to
be captured.  To mimic the normal behaviour and match a read name of the format
\fImachine:run:flowcell:lane:tile:x:y\fR use:
.EX 4
--read-coords '([!-9;-?A-~]+:[0-9]+:[0-9]+:[0-9]+:[0-9]+):([0-9]+):([0-9]+)'
--coords-order txy
.EE

To match only the coordinates of \fIx:y:randomstuff\fR use:
.EX 4
--read-coords '^([[:digit:]]+):([[:digit:]]+)'
--coords-order xy
.EE

To use a barcode from the read name matching the Illumina example of
\fINDX550136:7:H2MTNBDXX:1:13302:3141:10799:AAGGATG+TCGGAGA\fR use:

.EX 4
--barcode-rgx '[0-9A-Za-z]+:[0-9]+:[0-9A-Za-z]+:[0-9]+:[0-9]+:[0-9]+:[0-9]+:([!-?A-~]+)'
.EE

It is possible that complex regular expressions may slow the running of the
program.  It would be best to keep them simple. 

.SH AUTHOR
.PP
Written by Andrew Whitwham from the Sanger Institute.

.SH SEE ALSO
.IR samtools (1),
.IR samtools-sort (1),
.IR samtools-collate (1),
.IR samtools-fixmate (1)
.PP
Samtools website: <http://www.htslib.org/>