**************************
*****BioNano Genomics*****
******BNG-NGS Merge*******
**************************

ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang

Running on host: 69aa0c745d62

Process ID (PID): 997
Start time: 2023-01-26 18:36:57

COMMAND
perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n 9128-3.hap2.hic.filtered.fasta -u CTTAAG -z cur_results_10236012_10236025.zip -w status_10236012_10236025.txt -M conflicts_cut_status_CTTAAG_10236012_10236025.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236025/output

NGS file: /home/bionano/access/local/jobs/10236025/output/fa2cmap/9128-3.hap2.hic.filtered.fasta 
BNG file: /home/bionano/access/local/jobs/10236025/EXP_REFINEFINAL1.cmap 
Configuration file: /home/bionano/access/local/jobs/10236025/hybridScaffold_DLE1_config_082522.xml 
Output folder: /home/bionano/access/local/jobs/10236025/output
User-defined conflict status file: /home/bionano/access/local/jobs/10236025/conflicts_cut_status_CTTAAG_10236012_10236025.txt
RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner

Beginning cut_conflicts based on input a conflict file...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236025/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236025/output/fa2cmap/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236025/conflicts_cut_status_CTTAAG_10236012_10236025.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
cut_conflicts complete in 4 seconds.
32 BNG contigs are found after the cut-conflict step
91 NGS contigs are found after the cut-conflict step

Beginning MergeNGS_BN...
*Using user-defined conflict-cut BioNano and sequence CMAP files*
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236025/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236025/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236025/hybridScaffold_DLE1_config_082522.xml
MergeNGS_BN complete in 25 seconds.

Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236025/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of NGS contigs complete in 1 second.

Beginning alignment of NGS cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236025/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236025/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236025/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236025/output/align_final_M1/align_final_seq_script_log.txt
align sequences to hybrid scaffolds completed  in 56 seconds.

Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236025/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner
Extraction of BNG contigs complete in 1 second.

Beginning alignment of BNG cmap to Hybrid CMAP...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236025/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236025/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236025/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236025/output/align_final_M1/align_final_bng_log.txt
align Bionano genome maps to hybrid scaffolds completed  in 38 seconds.

Merging Hybrid CMAP with NGS not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236025/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236025/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr
Merging Hybrid CMAP with naive NGS CMAP complete in 1 second.

Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold...
Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236025/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236025/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr
Merging Hybrid CMAP with naive BNG CMAP complete in no time.

Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236025/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236025/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236025/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236025/output/fa2cmap/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236025/output/fa2cmap/9128-3.hap2.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t
AGP and FASTA generation complete in 43 seconds.

Calculating statistics...

Original BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/EXP_REFINEFINAL1.cmap

Count  = 32
Min length (Mbp) = 0.434
Median length (Mbp) = 7.045
Mean length (Mbp) = 15.233
N50 length (Mbp) = 40.227
Max length (Mbp) = 50.244
Total length (Mbp) = 487.453

Bpp-adjusted BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap

Count  = 32
Min length (Mbp) = 0.434
Median length (Mbp) = 7.044
Mean length (Mbp) = 15.230
N50 length (Mbp) = 40.220
Max length (Mbp) = 50.235
Total length (Mbp) = 487.367

Original NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/output/fa2cmap/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels.cmap

Count  = 92
Min length (Mbp) = 0.072
Median length (Mbp) = 0.248
Mean length (Mbp) = 5.486
N50 length (Mbp) = 37.411
Max length (Mbp) = 51.408
Total length (Mbp) = 504.738

Conflict Resolution from BNG-NGS alignment:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt

Number of conflict cuts made to Bionano maps: 0
Number of conflict cuts made to NGS sequences: 0
Number of Bionano maps to be cut: 0
Number of NGS sequences to be cut: 0

Before merge: BioNano Genome Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap

Count  = 32
Min length (Mbp) = 0.434
Median length (Mbp) = 7.044
Mean length (Mbp) = 15.230
N50 length (Mbp) = 40.220
Max length (Mbp) = 50.235
Total length (Mbp) = 487.367

Before merge: NGS sequences statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/output/assignAlignType/cut_conflicts_M1/9128-3.hap2.hic.filtered_CTTAAG_0kb_0labels_cut.cmap

Count  = 91
Min length (Mbp) = 0.072
Median length (Mbp) = 0.255
Mean length (Mbp) = 5.545
N50 length (Mbp) = 37.411
Max length (Mbp) = 51.408
Total length (Mbp) = 504.551

BNG Genome Map in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 30
Min length (Mbp) = 0.870
Median length (Mbp) = 7.231
Mean length (Mbp) = 16.200
N50 length (Mbp) = 40.220
Max length (Mbp) = 50.235
Total length (Mbp) = 486.005

NGS sequences in hybrid scaffold (CMAP) statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap

Count  = 27
Min length (Mbp) = 0.122
Median length (Mbp) = 10.619
Mean length (Mbp) = 18.220
N50 length (Mbp) = 37.411
Max length (Mbp) = 51.408
Total length (Mbp) = 491.929

Hybrid scaffold Map statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236025/output/align_final_M1/step2.hybrid.cmap

Count  = 13
Min length (Mbp) = 1.376
Median length (Mbp) = 43.861
Mean length (Mbp) = 37.936
N50 length (Mbp) = 44.031
Max length (Mbp) = 51.478
Total length (Mbp) = 493.163

NGS FASTA sequence in hybrid scaffold statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236025/output/agp_fasta_M1/9128-3.hap2.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236025/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 27
Min length (Mbp) = 0.122
Median length (Mbp) = 10.580
Mean length (Mbp) = 18.209
N50 length (Mbp) = 37.411
Max length (Mbp) = 51.408
Total length (Mbp) = 491.647

Hybrid scaffold FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236025/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta

Count  = 13
Min length (Mbp) = 1.111
Median length (Mbp) = 43.814
Mean length (Mbp) = 37.829
N50 length (Mbp) = 43.883
Max length (Mbp) = 51.408
Total length (Mbp) = 491.777

Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics:
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236025/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236025/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta

Count  = 86
Min length (Mbp) = 0.004
Median length (Mbp) = 0.166
Mean length (Mbp) = 5.871
N50 length (Mbp) = 43.883
Max length (Mbp) = 51.408
Total length (Mbp) = 504.869


Calculating CMAP statistics complete in 16 seconds.

Calculating XMAP statistics...
Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236025/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
100055	1375999.1	1104460.4	1104460.4	80.27
100006	40220293.4	40085071.4	40077330.9	99.64
5	42949030.0	42810284	42810284	99.68
100001	50235015.3	49959698.1	49943860.2	99.42
100002	44030622.2	43871278.6	43859683.9	99.61
100003	43860864.8	43702468.9	43702468.9	99.64
1	51477689.9	51331967.9	51331967.9	99.72
24	36962823.8	36854736.9	36854736.9	99.71
9	49474425.9	49331829.1	49271215.3	99.59
20	5529252.1	5289310.9	5289310.9	95.66
100008	34348945.7	34227318.3	34219299.9	99.62
2	47061807.1	46969875.4	46969875.4	99.80
6	45635961.3	45567225.7	45551988.3	99.82

Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236025/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_9128-3_hap2_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats

refContig	refContig_size	query_cov_bp	query_unique_cov_bp	query_unique_cov_percent
6	45635961.3	44656297.3	44656297.3	97.85
100006	40220293.4	40077330.9	40077330.9	99.64
20	5529252.1	1788395.3	1788395.3	32.34
9	49474425.9	48769535.5	48761402.5	98.56
1	51477689.9	51144025.9	51144025.9	99.35
24	36962823.8	35481796.4	35481796.4	95.99
100055	1375999.1	1370471.8	1370471.8	99.60
2	47061807.1	46382933.4	46382933.4	98.56
100001	50235015.3	50084170.9	50084170.9	99.70
100008	34348945.7	34219299.9	34219299.9	99.62
100002	44030622.2	43859683.9	43859683.9	99.61
5	42949030.0	42810284	42810284	99.68
100003	43860864.8	43702468.9	43702468.9	99.64

XMAP statistics calculation complete in 1 second.

Beginning archive and zip of result files...
