Memory error for metaSNV

[nishijima-suguru]

Hi, I would like to use metaSNV for my study.
However, I had a following error when I tried to use metaSNV.

[nishijima@tn1 metaSNV]$ python metaSNV.py test.metaSNV file.list db/freeze9.genomes.RepGenomesv9.fna --threads 10
Printing details in test.metaSNV/cov/3080.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Printing details in test.metaSNV/cov/3082.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Printing details in test.metaSNV/cov/3085.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Printing details in test.metaSNV/cov/3078.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Printing details in test.metaSNV/cov/3077.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Printing details in test.metaSNV/cov/3079.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Printing details in test.metaSNV/cov/3083.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Printing details in test.metaSNV/cov/3081.prinseq_1.fastq.cutad.fastq.fs.fastq.exHuman.exPhix.fastq.gz.bwa.sam.bam.sort.bam.cov.detail!
Probably really good coverage, since variables overrun!
That's a big sum: 3457...
Probably really good coverage, since variables overrun!
That's a big sum: 113618...
Probably really good coverage, since variables overrun!
That's a big sum: 89440...
*** Error in `/home/nishijima/tool/metaSNV/src/qaTools/qaCompute': munmap_chunk(): invalid pointer: 0x00000000029e8310 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7ab54)[0x2ab74cf6fb54]
/home/nishijima/tool/metaSNV/src/qaTools/qaCompute[0x4024a1]
/home/nishijima/tool/metaSNV/src/qaTools/qaCompute[0x403724]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x2ab74cf16c05]
/home/nishijima/tool/metaSNV/src/qaTools/qaCompute[0x4017d9]
======= Memory map: ========
00400000-00409000 r-xp 00000000 00:2a 1074273 /home/nishijima/tool/metaSNV/src/qaTools/qaCompute
00608000-00609000 r--p 00008000 00:2a 1074273 /home/nishijima/tool/metaSNV/src/qaTools/qaCompute
00609000-0060a000 rw-p 00009000 00:2a 1074273 /home/nishijima/tool/metaSNV/src/qaTools/qaCompute
01d24000-029f7000 rw-p 00000000 00:00 0 [heap]
2ab74bdc9000-2ab74bdea000 r-xp 00000000 08:03 134352079 /usr/lib64/ld-2.17.so
2ab74bdea000-2ab74bdf1000 rw-p 00000000 00:00 0
2ab74bdff000-2ab74bf2e000 rw-p 00000000 00:00 0
2ab74bfea000-2ab74bfeb000 r--p 00021000 08:03 134352079 /usr/lib64/ld-2.17.so
2ab74bfeb000-2ab74bfec000 rw-p 00022000 08:03 134352079 /usr/lib64/ld-2.17.so
2ab74bfec000-2ab74bfed000 rw-p 00000000 00:00 0
2ab74bfed000-2ab74c0a1000 r-xp 00000000 00:2a 6142517 /home/nishijima/tool/htslib/libhts.so
2ab74c0a1000-2ab74c2a1000 ---p 000b4000 00:2a 6142517 /home/nishijima/tool/htslib/libhts.so
2ab74c2a1000-2ab74c2a2000 r--p 000b4000 00:2a 6142517 /home/nishijima/tool/htslib/libhts.so
2ab74c2a2000-2ab74c2a4000 rw-p 000b5000 00:2a 6142517 /home/nishijima/tool/htslib/libhts.so
2ab74c2a4000-2ab74c2b9000 r-xp 00000000 08:03 134352310 /usr/lib64/libz.so.1.2.7
2ab74c2b9000-2ab74c4b8000 ---p 00015000 08:03 134352310 /usr/lib64/libz.so.1.2.7
2ab74c4b8000-2ab74c4b9000 r--p 00014000 08:03 134352310 /usr/lib64/libz.so.1.2.7

Could you please help me solving this problem?
I deeply appreciate.

Suguru

[Fedorov113]

I experience the same issue too.

[Laurenz0908]

I also have this problem, help would be much appreciated.

[CosteaPaul]

Hi @Laurenz0908 ,

Can you please try again with the latest version?
I had a counter variable overrun. That should be fixed now.

[Laurenz0908]

Hi @CosteaPaul ,
thank you for the quick reply - unfortunately it doesn't compile on my machine

I get:

g++ -I/usr/bin -I/usr/include/htslib -c -Wall removeUnmapped.c
removeUnmapped.c: In function ‘bool is_mapped(const bam1_t*, int, double)’:
removeUnmapped.c:39:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i=0; i< core->n_cigar; ++i) {
                 ~^~~~~~~~~~~~~~~
removeUnmapped.c:40:10: error: ‘bam1_cigar’ was not declared in this scope
     if ((bam1_cigar(b)[i] & BAM_CIGAR_MASK) == BAM_CMATCH) {
          ^~~~~~~~~~
removeUnmapped.c:40:10: note: suggested alternative: ‘bam_cigar_op’
     if ((bam1_cigar(b)[i] & BAM_CIGAR_MASK) == BAM_CMATCH) {
          ^~~~~~~~~~
          bam_cigar_op
removeUnmapped.c: At global scope:
removeUnmapped.c:74:1: error: ‘samfile_t’ does not name a type; did you mean ‘samFile’?
 samfile_t * open_alignment_file(std::string path, void* aux = NULL)
 ^~~~~~~~~
 samFile
removeUnmapped.c: In function ‘bool isJunction(const bam1_t*)’:
removeUnmapped.c:98:14: error: ‘bam1_cigar’ was not declared in this scope
     int op = bam1_cigar(entry)[k] & BAM_CIGAR_MASK; //operation
              ^~~~~~~~~~
removeUnmapped.c:98:14: note: suggested alternative: ‘bam_cigar_op’
     int op = bam1_cigar(entry)[k] & BAM_CIGAR_MASK; //operation
              ^~~~~~~~~~
              bam_cigar_op
removeUnmapped.c: In function ‘void print_bam_to_fastq(bam1_t*, FILE*, int)’:
removeUnmapped.c:117:9: warning: statement has no effect [-Wunused-value]
   for (i; (i<core->l_qseq) && (i<cutoff); ++i) {
         ^
removeUnmapped.c:118:14: error: ‘bam_nt16_rev_table’ was not declared in this scope
     seq[i] = bam_nt16_rev_table[bam1_seqi(bam1_seq(b), i)];
              ^~~~~~~~~~~~~~~~~~
removeUnmapped.c:118:14: note: suggested alternative: ‘seq_nt16_table’
     seq[i] = bam_nt16_rev_table[bam1_seqi(bam1_seq(b), i)];
              ^~~~~~~~~~~~~~~~~~
              seq_nt16_table
removeUnmapped.c:118:43: error: ‘bam1_seq’ was not declared in this scope
     seq[i] = bam_nt16_rev_table[bam1_seqi(bam1_seq(b), i)];
                                           ^~~~~~~~
removeUnmapped.c:118:43: note: suggested alternative: ‘bam_seqi’
     seq[i] = bam_nt16_rev_table[bam1_seqi(bam1_seq(b), i)];
                                           ^~~~~~~~
                                           bam_seqi
removeUnmapped.c:118:33: error: ‘bam1_seqi’ was not declared in this scope
     seq[i] = bam_nt16_rev_table[bam1_seqi(bam1_seq(b), i)];
                                 ^~~~~~~~~
removeUnmapped.c:118:33: note: suggested alternative: ‘bam_seqi’
     seq[i] = bam_nt16_rev_table[bam1_seqi(bam1_seq(b), i)];
                                 ^~~~~~~~~
                                 bam_seqi
removeUnmapped.c:120:15: error: ‘bam1_qual’ was not declared in this scope
     qual[i] = bam1_qual(b)[i]+64;
               ^~~~~~~~~
removeUnmapped.c:120:15: note: suggested alternative: ‘bam_dup1’
     qual[i] = bam1_qual(b)[i]+64;
               ^~~~~~~~~
               bam_dup1
removeUnmapped.c:125:36: error: ‘bam1_qname’ was not declared in this scope
   fprintf(fastq,"@%s\n%s\n+\n%s\n",bam1_qname(b),seq,qual);
                                    ^~~~~~~~~~
removeUnmapped.c:125:36: note: suggested alternative: ‘bam_get_qname’
   fprintf(fastq,"@%s\n%s\n+\n%s\n",bam1_qname(b),seq,qual);
                                    ^~~~~~~~~~
                                    bam_get_qname
removeUnmapped.c: In function ‘int main(int, char**)’:
removeUnmapped.c:133:5: error: ‘samfile_t’ was not declared in this scope
     samfile_t *fp;
     ^~~~~~~~~
removeUnmapped.c:133:5: note: suggested alternative: ‘samFile’
     samfile_t *fp;
     ^~~~~~~~~
     samFile
removeUnmapped.c:133:16: error: ‘fp’ was not declared in this scope
     samfile_t *fp;
                ^~
removeUnmapped.c:134:16: error: ‘out’ was not declared in this scope
     samfile_t *out;
                ^~~
removeUnmapped.c:134:16: note: suggested alternative: ‘putw’
     samfile_t *out;
                ^~~
                putw
removeUnmapped.c:135:16: error: ‘out_rem’ was not declared in this scope
     samfile_t *out_rem = NULL;
                ^~~~~~~
removeUnmapped.c:162:24: error: ‘strlen’ was not declared in this scope
  good_file = new char[ strlen(optarg) ];
                        ^~~~~~
removeUnmapped.c:162:24: note: suggested alternative: ‘mbrlen’
  good_file = new char[ strlen(optarg) ];
                        ^~~~~~
                        mbrlen
removeUnmapped.c:163:2: error: ‘strcpy’ was not declared in this scope
  strcpy(good_file, optarg);
  ^~~~~~
removeUnmapped.c:163:2: note: suggested alternative: ‘strtoq’
  strcpy(good_file, optarg);
  ^~~~~~
  strtoq
removeUnmapped.c:192:10: error: ‘open_alignment_file’ was not declared in this scope
     fp = open_alignment_file(argv[optind]);
          ^~~~~~~~~~~~~~~~~~~
removeUnmapped.c:230:12: error: ‘samread’ was not declared in this scope
     while (samread(fp,b) >= 0) {
            ^~~~~~~
removeUnmapped.c:230:12: note: suggested alternative: ‘sam_read1’
     while (samread(fp,b) >= 0) {
            ^~~~~~~
            sam_read1
removeUnmapped.c:232:2: error: ‘samwrite’ was not declared in this scope
  samwrite(out,b);
  ^~~~~~~~
removeUnmapped.c:232:2: note: suggested alternative: ‘sam_write1’
  samwrite(out,b);
  ^~~~~~~~
  sam_write1
removeUnmapped.c:238:4: error: ‘samwrite’ was not declared in this scope
    samwrite(out_rem,b);
    ^~~~~~~~
removeUnmapped.c:238:4: note: suggested alternative: ‘sam_write1’
    samwrite(out_rem,b);
    ^~~~~~~~
    sam_write1
removeUnmapped.c:245:12: error: ‘samread’ was not declared in this scope
     while (samread(fp, b) >= 0) {
            ^~~~~~~
removeUnmapped.c:245:12: note: suggested alternative: ‘sam_read1’
     while (samread(fp, b) >= 0) {
            ^~~~~~~
            sam_read1
removeUnmapped.c:254:18: error: ‘bam1_qname’ was not declared in this scope
  string b_name = bam1_qname(b);
                  ^~~~~~~~~~
removeUnmapped.c:254:18: note: suggested alternative: ‘b_name’
  string b_name = bam1_qname(b);
                  ^~~~~~~~~~
                  b_name
removeUnmapped.c:256:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  if (b_name.find('/') != -1) {
      ~~~~~~~~~~~~~~~~~^~~~~
removeUnmapped.c:261:2: error: ‘samwrite’ was not declared in this scope
  samwrite(out,b);
  ^~~~~~~~
removeUnmapped.c:261:2: note: suggested alternative: ‘sam_write1’
  samwrite(out,b);
  ^~~~~~~~
  sam_write1
removeUnmapped.c:284:4: error: ‘samwrite’ was not declared in this scope
    samwrite(out,c);
    ^~~~~~~~
removeUnmapped.c:284:4: note: suggested alternative: ‘sam_write1’
    samwrite(out,c);
    ^~~~~~~~
    sam_write1
removeUnmapped.c:285:54: error: ‘bam1_qname’ was not declared in this scope
    fprintf(stderr,"Writing c: %s\t%s\t%d\t%d\t%d\n", bam1_qname(c),fp->header->target_name[c->core.tid], b->core.qual, b->core.flag&BAM_FUNMAP, c->core.flag&BAM_FUNMAP);
                                                      ^~~~~~~~~~
removeUnmapped.c:285:54: note: suggested alternative: ‘bam_get_qname’
    fprintf(stderr,"Writing c: %s\t%s\t%d\t%d\t%d\n", bam1_qname(c),fp->header->target_name[c->core.tid], b->core.qual, b->core.flag&BAM_FUNMAP, c->core.flag&BAM_FUNMAP);
                                                      ^~~~~~~~~~
                                                      bam_get_qname
removeUnmapped.c:288:54: error: ‘bam1_qname’ was not declared in this scope
    fprintf(stderr,"Writing b: %s\t%s\t%d\t%d\t%d\n", bam1_qname(b), fp->header->target_name[b->core.tid] ,c->core.qual, b->core.flag&BAM_FUNMAP, c->core.flag&BAM_FUNMAP);
                                                      ^~~~~~~~~~
removeUnmapped.c:288:54: note: suggested alternative: ‘bam_get_qname’
    fprintf(stderr,"Writing b: %s\t%s\t%d\t%d\t%d\n", bam1_qname(b), fp->header->target_name[b->core.tid] ,c->core.qual, b->core.flag&BAM_FUNMAP, c->core.flag&BAM_FUNMAP);
                                                      ^~~~~~~~~~
                                                      bam_get_qname
removeUnmapped.c:289:4: error: ‘samwrite’ was not declared in this scope
    samwrite(out,b);
    ^~~~~~~~
removeUnmapped.c:289:4: note: suggested alternative: ‘sam_write1’
    samwrite(out,b);
    ^~~~~~~~
    sam_write1
removeUnmapped.c:296:4: error: ‘samwrite’ was not declared in this scope
    samwrite(out_rem, b);
    ^~~~~~~~
removeUnmapped.c:296:4: note: suggested alternative: ‘sam_write1’
    samwrite(out_rem, b);
    ^~~~~~~~
    sam_write1
removeUnmapped.c:307:3: error: ‘samclose’ was not declared in this scope
   samclose(fp);
   ^~~~~~~~
removeUnmapped.c:307:3: note: suggested alternative: ‘sam_close’
   samclose(fp);
   ^~~~~~~~
   sam_close
Makefile:32: recipe for target 'removeUnmapped.o' failed
make: *** [removeUnmapped.o] Error 1

[CosteaPaul]

Have you stashed the changes made to Makefile before updating?
It looks like now the compiler can't find the paths to samtools and htslib.

Checkout the readme again ;). You have to set these paths for things to compile.

Maybe i should push some binaries too. Guess it would make things easier.

[CosteaPaul]

@Laurenz0908, i added a bin directory with a compiled version of qaCompute.
If that runs for you, then you don't have to bother with compiling.

[CosteaPaul]

@Laurenz0908 , can i consider this issue closed?

[Laurenz0908]

@CosteaPaul thank you for your help, I'm sorry I couldn't get back to you so far. I have tried with your new version but could not get it to compile, also just exchanging the binaries did not work. Unfortunately I haven't had the time yet to work on it but I will let you know as soon as I can.

[alimayy]

see metasnv-tool/metaSNV#3

@Laurenz0908 @Fedorov113 @nishijima-suguru @CosteaPaul