************************** *****BioNano Genomics***** ******BNG-NGS Merge******* ************************** ExportAGP.pl 11203 2020-06-18 21:28:10Z jwang Running on host: 6d5379c5276e Process ID (PID): 997 Start time: 2023-01-31 19:05:03 COMMAND perl /home/bionano/tools/access/1.0/../../pipeline/1.0/HybridScaffold/1.0/hybridScaffold.pl -c hybridScaffold_DLE1_config_082522.xml -b EXP_REFINEFINAL1.cmap -n SH3142.hap1.hic.filtered.fasta -u CTTAAG -z cur_results_10236018_10236353.zip -w status_10236018_10236353.txt -M conflicts_cut_status_CTTAAG_10236018_10236353.txt -g -B 2 -N 2 -r /home/bionano/tools/access/1.0/../../pipeline/1.0/RefAligner/1.0/RefAligner -p /home/bionano/tools/access/1.0/../../pipeline/1.0/Pipeline/1.0 -o /home/bionano/access/local/jobs/10236353/output NGS file: /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered.fasta BNG file: /home/bionano/access/local/jobs/10236353/EXP_REFINEFINAL1.cmap Configuration file: /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml Output folder: /home/bionano/access/local/jobs/10236353/output User-defined conflict status file: /home/bionano/access/local/jobs/10236353/conflicts_cut_status_CTTAAG_10236018_10236353.txt RefAligner binary: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner Beginning cut_conflicts based on input a conflict file... Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/cut_conflicts.pl -oriGMFile /home/bionano/access/local/jobs/10236353/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap -oriSeqFile /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap -outDir /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1 -outGMFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -outSeqFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -modBkptStatusFile /home/bionano/access/local/jobs/10236353/conflicts_cut_status_CTTAAG_10236018_10236353.txt -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner cut_conflicts complete in 3 seconds. 21 BNG contigs are found after the cut-conflict step 124 NGS contigs are found after the cut-conflict step Beginning MergeNGS_BN... *Using user-defined conflict-cut BioNano and sequence CMAP files* Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/MergeNGS_BN.pl -outputDir /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1 -refaligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -logFile mergeNGS_BN_script.log -ngs_cmap_fn /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -bng_cmap_fn /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -id_shift 100000 -readparameters /home/bionano/access/local/jobs/10236353/output/align1/align1.errbin -xmlFile /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml MergeNGS_BN complete in 22 seconds. Beginning extraction of used and not used NGS contigs in hybrid scaffold pairmerge step... Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_ngs.pl /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner Extraction of NGS contigs complete in 1 second. Beginning alignment of NGS cmap to Hybrid CMAP... Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_seq_two_passes.pl -outDir /home/bionano/access/local/jobs/10236353/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml -seqFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/align_final_seq_script_log.txt align sequences to hybrid scaffolds completed in 51 seconds. Beginning extraction of used and not used BNG contigs in hybrid scaffold pairmerge step... Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/find_used_not_used_bn.pl /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1 100000 /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner Extraction of BNG contigs complete in 1 second. Beginning alignment of BNG cmap to Hybrid CMAP... Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/align_final_bng.pl -outDir /home/bionano/access/local/jobs/10236353/output/align_final_M1 -outFilePrefix EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD -refAligner /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -xmlFile /home/bionano/access/local/jobs/10236353/hybridScaffold_DLE1_config_082522.xml -bngFile /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap -scaffoldFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -logFile /home/bionano/access/local/jobs/10236353/output/align_final_M1/align_final_bng_log.txt align Bionano genome maps to hybrid scaffolds completed in 34 seconds. Merging Hybrid CMAP with NGS not participated in the hybrid scaffold... Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1/all_not_used_ngs.cmap -o HYBRID_SCAFFOLD_notUsedNGS_merged -merge -minsites 0 -stdout -stderr Merging Hybrid CMAP with naive NGS CMAP complete in 1 second. Merging Hybrid CMAP with BNG CMAP not participated in the hybrid scaffold... Running command: /home/bionano/tools/pipeline/Solve3.7_03302022_283/RefAligner/12432.12642rel/RefAligner -f -i /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap -i /home/bionano/access/local/jobs/10236353/output/mergeNGS_BN_M1/all_not_used_bng.cmap -o HYBRID_SCAFFOLD_notUsedBNG_merged -merge -stdout -stderr Merging Hybrid CMAP with naive BNG CMAP complete in 1 second. Beginnning construction of AGP and FASTA file of the scaffolded and unscaffolded sequences... Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/ExportAGP.pl -i /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap -c /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_r.cmap -o /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1 -m /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_key.txt -s /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered.fasta -e CTTAAG 1 -r /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/auto_cut_NGS_coord_translation.txt -t AGP and FASTA generation complete in 44 seconds. Calculating statistics... Original BioNano Genome Map statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/EXP_REFINEFINAL1.cmap Count = 21 Min length (Mbp) = 0.121 Median length (Mbp) = 20.793 Mean length (Mbp) = 22.985 N50 length (Mbp) = 45.663 Max length (Mbp) = 50.281 Total length (Mbp) = 482.679 Bpp-adjusted BioNano Genome Map statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align0/EXP_REFINEFINAL1_bppAdjust.cmap Count = 21 Min length (Mbp) = 0.121 Median length (Mbp) = 20.783 Mean length (Mbp) = 22.974 N50 length (Mbp) = 45.641 Max length (Mbp) = 50.257 Total length (Mbp) = 482.448 Original NGS sequences statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/fa2cmap/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels.cmap Count = 128 Min length (Mbp) = 0.071 Median length (Mbp) = 0.192 Mean length (Mbp) = 3.868 N50 length (Mbp) = 24.395 Max length (Mbp) = 49.057 Total length (Mbp) = 495.040 Conflict Resolution from BNG-NGS alignment: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_conflicts_cut_stats.pl /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/conflicts_cut_status.txt Number of conflict cuts made to Bionano maps: 0 Number of conflict cuts made to NGS sequences: 0 Number of Bionano maps to be cut: 0 Number of NGS sequences to be cut: 0 Before merge: BioNano Genome Map statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/EXP_REFINEFINAL1_bppAdjust_cut.cmap Count = 21 Min length (Mbp) = 0.121 Median length (Mbp) = 20.783 Mean length (Mbp) = 22.974 N50 length (Mbp) = 45.641 Max length (Mbp) = 50.257 Total length (Mbp) = 482.448 Before merge: NGS sequences statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/assignAlignType/cut_conflicts_M1/SH3142.hap1.hic.filtered_CTTAAG_0kb_0labels_cut.cmap Count = 124 Min length (Mbp) = 0.071 Median length (Mbp) = 0.187 Mean length (Mbp) = 3.982 N50 length (Mbp) = 24.395 Max length (Mbp) = 49.057 Total length (Mbp) = 493.730 BNG Genome Map in hybrid scaffold statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD_q.cmap Count = 15 Min length (Mbp) = 1.784 Median length (Mbp) = 36.761 Mean length (Mbp) = 31.756 N50 length (Mbp) = 46.663 Max length (Mbp) = 50.257 Total length (Mbp) = 476.335 NGS sequences in hybrid scaffold (CMAP) statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_q.cmap Count = 36 Min length (Mbp) = 0.094 Median length (Mbp) = 7.646 Mean length (Mbp) = 13.257 N50 length (Mbp) = 24.395 Max length (Mbp) = 49.057 Total length (Mbp) = 477.256 Hybrid scaffold Map statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl ./scripts/calc_cmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1/step2.hybrid.cmap Count = 14 Min length (Mbp) = 1.941 Median length (Mbp) = 38.286 Mean length (Mbp) = 34.154 N50 length (Mbp) = 46.678 Max length (Mbp) = 50.257 Total length (Mbp) = 478.162 NGS FASTA sequence in hybrid scaffold statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/SH3142.hap1.hic.filtered.fasta.cut.fasta /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta Count = 36 Min length (Mbp) = 0.046 Median length (Mbp) = 7.635 Mean length (Mbp) = 13.252 N50 length (Mbp) = 24.391 Max length (Mbp) = 49.057 Total length (Mbp) = 477.060 Hybrid scaffold FASTA statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_fasta_stats.pl /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta Count = 14 Min length (Mbp) = 1.914 Median length (Mbp) = 38.262 Mean length (Mbp) = 34.084 N50 length (Mbp) = 46.656 Max length (Mbp) = 50.211 Total length (Mbp) = 477.176 Hybrid scaffold FASTA plus not scaffolded NGS FASTA statistics: Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_hybrid_scaffold_not_scaffolded_seq_fasta_stats.pl /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.fasta /home/bionano/access/local/jobs/10236353/output/agp_fasta_M1/EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD_NOT_SCAFFOLDED.fasta Count = 119 Min length (Mbp) = 0.001 Median length (Mbp) = 0.133 Mean length (Mbp) = 4.161 N50 length (Mbp) = 45.603 Max length (Mbp) = 50.211 Total length (Mbp) = 495.156 Calculating CMAP statistics complete in 17 seconds. Calculating XMAP statistics... Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_NGScontigs_HYBRID_SCAFFOLD.xmap.stats refContig refContig_size query_cov_bp query_unique_cov_bp query_unique_cov_percent 3 39649759.7 39606637.4 39606637.4 99.89 14 46798738.2 46733828.4 46733828.4 99.86 100013 5301110.5 5059281.3 5059281.3 95.44 10 20790859.0 20700702.3 20700702.3 99.57 39 1941468.7 1920697.2 1835078.8 94.52 15 18085687.3 17846394.6 17846394.6 98.68 100009 35704529.1 35440046.7 35440046.7 99.26 4 47088199.1 47037292.4 47028708.4 99.87 12 46678111.9 46640736.8 46638028.4 99.91 11 45660567.5 45607206.4 45607206.4 99.88 100001 50256983.3 50114569.6 50107939.8 99.70 100002 49122649.6 49046225.2 49046225.2 99.84 8 36922738.7 36845612.3 36845612.3 99.79 100010 34160807.9 34093388 34085334.9 99.78 Running command: /home/bionano2/.conda/envs/bionano3/bin/perl scripts/calc_xmap_stats.pl /home/bionano/access/local/jobs/10236353/output/align_final_M1 EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap step2.hybrid.cmap EXP_REFINEFINAL1_bppAdjust_cmap_SH3142_hap1_hic_filtered_fasta_BNGcontigs_HYBRID_SCAFFOLD.xmap.stats refContig refContig_size query_cov_bp query_unique_cov_bp query_unique_cov_percent 100001 50256983.3 50188664.3 50188664.3 99.86 100002 49122649.6 49082889.7 49082889.7 99.92 39 1941468.7 1773716.8 1773716.8 91.36 15 18085687.3 16545972.3 16395671.3 90.66 12 46678111.9 46649368.8 46649368.8 99.94 10 20790859.0 20773370 20773370 99.92 11 45660567.5 45609950 45609950 99.89 100009 35704529.1 35658498.2 35658498.2 99.87 100013 5301110.5 5263815.4 5263815.4 99.30 100010 34160807.9 34104728.5 34104728.5 99.84 14 46798738.2 46761286 46761286 99.92 8 36922738.7 36897697.7 36897697.7 99.93 3 39649759.7 39606637.4 39606637.4 99.89 4 47088199.1 47046456.3 47046456.3 99.91 XMAP statistics calculation complete in 1 second. Beginning archive and zip of result files...