Error whilst querying BigWig file: "got an error in bwStatsFromZoom"

[andrewyatz]

Whilst querying for stats from a BigWig file from the Ensembl FTP site I got an error from libBigWig. I found it first in my Perl bindings but I've been able to replicate this in C below:

#include "bigWig.h"
#include <stdio.h>
#include <inttypes.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    bigWigFile_t *fp = NULL;
    double *stats = NULL;
    
    char file[] = "http://ftp.ensembl.org/pub/data_files/homo_sapiens/GRCh38/rnaseq/GRCh38.illumina.sk_muscle.1.bam.bw";
    char chrom[] = "21";
    int start = 33031597;
    int end = 34041570;
    int full = 0;
    int bins = 100;
    
    if(bwInit(1<<17) != 0) {
        fprintf(stderr, "Received an error in bwInit\n");
        return 1;
    }
    
    fp = bwOpen(file, NULL, "r");
    if(!fp) {
        fprintf(stderr, "An error occured while opening %s\n", file);
        return 1;
    }
    if(full) 
      stats = bwStatsFromFull(fp, chrom, 33031597, 34041570, bins, mean);
    else 
      stats = bwStats(fp, chrom, 33031597, 34041570, bins, mean);

    if(stats)
      free(stats);

    bwClose(fp);
    bwCleanup();
    return 0;
}

This was compiled using cc -g -Wall -O3 -Wsign-compare -o query query.c libBigWig.a -lcurl -lm -lz. When run the error I currently get back is

got an error in bwStatsFromZoom in the range 33031597-33041696: Operation now in progress

When I switch to requesting full stats the message goes away. The BigWig file in question is a coverage plot from a BAM file and was created using the kent utils. I've also checked with the kent binaries and it's able to report statistics for the requested region.

Any help is appreciated.

[dpryan79]

Weird, I'll have too have a look at this. Note that it might take me a bit of time as I'm off for vacation tomorrow.

[andrewyatz]

Thanks for looking into it. Don't worry about speed of response. Have a nice vacation!

[dpryan79]

Please check the fix20 branch, which at least fixes this exact example.

[andrewyatz]

That seems to work. I'm not sure of any other problematic BigWig files yet but I think you can call this squished

…

On 15 March 2017 at 22:30, Devon Ryan ***@***.***> wrote: Please check the fix20 branch, which at least fixes this exact example. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAK5GUh9lAo2qCfRQe-PIrZN5feTalIuks5rmGaQgaJpZM4MeGNt> .

[dpryan79]

Cool, when I have time I'll try to figure out what the heck is up with the other issue you reported (for whatever reason, curl was only asking for 8 bytes at a time, which made everything take ages). Once those are both fixed I'll push out a new release.

[andrewyatz]

That's great news thank you

…

On Thu, 16 Mar 2017 at 07:25, Devon Ryan ***@***.***> wrote: Cool, when I have time I'll try to figure out what the heck is up with the other issue you reported (for whatever reason, curl was only asking for 8 bytes at a time, which made everything take ages). Once those are both fixed I'll push out a new release. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAK5GbWnpzr8thpoFO7zoQmjFh7RgaK2ks5rmOP6gaJpZM4MeGNt> .

[andrewyatz]

I'll close this down as well since 33d436b seems to fix everything

[lidaof]

I also have similar issue while trying to view a large region, but not small region...but sometimes I don't get this error. Is it possible to print out the fail reason? Thanks.

got an error in bwStatsFromZoom in the range 27053199-27069217: Operation now in progress

[dpryan79]

"Operation now in progress" is the full error message and corresponds to libcurl setting errno to EINPROGRESS. In the python wrapper for this, I tended to see this most often when using multithreading where each thread was querying the remote server (you have to limit your thread numbers when doing that).

[lidaof]

Thank you. I am not using the python wrapper. Just single C binary file, but it didn't happen every time. Thanks again.