Run time of Virmap

[bioinfonewbie-kiki]

Hi, I have two 11Gb paired end fastq file and am trying to run virmap with it, but it has been taking more than 36 hours for the job to run and it still hasn't finished (with 48 cores). I would like to know what is the estimated run time for virmap?

[IgnatiusPang]

Hi, I will suggest subsampling to a much smaller size (e.g. less than 200 Mb) and having a trial run with the smaller file, to check if the pipeline runs properly. The https://github.com/lh3/seqtk will have tools for subsampling your reads. You could also do some mapping to first subtract any contaminant reads (e.g. from the host or environmental bacteria.) Run time is highly dependent on the size of the file and the complexity of sequences in the sample, also the memory and CPU cores you've used, so it is very difficult to give you an estimate. It can be around 24 minutes for 24 cores and 182 Gb ram for a file single-ended file around 120 Mb and using the VirCapSeq enrichment technique ( https://pubmed.ncbi.nlm.nih.gov/26396248/), where the majority of reads, at least 90% of them, are human or bacterial reads.... the rest e.g. less than 10% are virome reads. Hope this helps. Kind regards, Igy

…

On Fri, Aug 27, 2021 at 10:18 AM bioinfonewbie-kiki < ***@***.***> wrote: Hi, I have two 11Gb paired end fastq file and am trying to run virmap with it, but it has been taking more than 36 hours for the job to run and it still hasn't finished (with 48 cores). I would like to know what is the estimated run time for virmap? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARWXVVCJ32IHXEYRQCFAZLT63KV7ANCNFSM5C4OJANA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[IgnatiusPang]

I was referring to fastq.gz compressed files when I was referring to file sizes. On Fri, Aug 27, 2021 at 12:49 PM Ignatius Pang ***@***.***> wrote:

…

Hi, I will suggest subsampling to a much smaller size (e.g. less than 200 Mb) and having a trial run with the smaller file, to check if the pipeline runs properly. The https://github.com/lh3/seqtk will have tools for subsampling your reads. You could also do some mapping to first subtract any contaminant reads (e.g. from the host or environmental bacteria.) Run time is highly dependent on the size of the file and the complexity of sequences in the sample, also the memory and CPU cores you've used, so it is very difficult to give you an estimate. It can be around 24 minutes for 24 cores and 182 Gb ram for a file single-ended file around 120 Mb and using the VirCapSeq enrichment technique ( https://pubmed.ncbi.nlm.nih.gov/26396248/), where the majority of reads, at least 90% of them, are human or bacterial reads.... the rest e.g. less than 10% are virome reads. Hope this helps. Kind regards, Igy On Fri, Aug 27, 2021 at 10:18 AM bioinfonewbie-kiki < ***@***.***> wrote: > Hi, I have two 11Gb paired end fastq file and am trying to run virmap > with it, but it has been taking more than 36 hours for the job to run and > it still hasn't finished (with 48 cores). I would like to know what is the > estimated run time for virmap? > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#4>, or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AARWXVVCJ32IHXEYRQCFAZLT63KV7ANCNFSM5C4OJANA> > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> > or Android > <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. > >

[bioinfonewbie-kiki]

Hi, thanks for your reply. I will give that a try.

[bioinfonewbie-kiki]

Hi, I tried a small sample (36mb) and it seemed to have finished successfully. But I noticed the iterative improvement step took the longest, and all of the samples that took very long to run seems to stop at the iterative improvement step, as I checked the log and the log only recorded up to the filter contigs step. Do you have any suggestions on why is this the case?