It’s not always accurate.

[saas-pro]

I’m running the same command, but I’m getting different output results. How can I overcome this issue? The output isn’t accurate each time. For example, before adding the URLscan API key, the output was 134707.

After adding the URLscan API key to “provider-config.yaml,” I’m receiving even fewer URLs. Why is this happening?

[dogancanbakir]

@dark-aura At first glance, there appear to be problems with the statistics. Could you compare the results according to the number of lines each run generates?

[zy9ard3]

@dogancanbakir

I confirm this is indeed an issue with urlscan implementation and not with statistics

┌──(root🍀zy9ard3)-[~]
└─# wc -l withapikey withoutapikey 
   95766 withapikey
  171594 withoutapikey
  267360 total

[dogancanbakir]

@zy9ard3 Are you suggesting that there is no issue with urlfinder ?

[zy9ard3]

@dogancanbakir

@zy9ard3 Are you suggesting that there is no issue with urlfinder ?

No, what I meant is that the issue arises from the urlscan implementation and no issue with the statistics

I verified both the outputs ( with and without the urlscan API key ) and found that some URLs are being missed when using the urlscan API key

[dogancanbakir]

Thanks, I'll look into it.

[gwatai]

Hello folks, I hope you are doing great, Any update on the issue? Thank you