Maintain dynamic count of URLs per crawl ID

[klockla]

This PR introduces a new API call: GetCrawlStats which will return the number of completed and total URLs per crawl.

[jnioche]

sorry don't have time to look at it in details but how is it different from

     /** Return stats for a specific queue or an entire crawl. Does not aggregate the stats across different crawlids. **/
     rpc GetStats(QueueWithinCrawlParams) returns (Stats) {}

[klockla]

sorry don't have time to look at it in details but how is it different from

     /** Return stats for a specific queue or an entire crawl. Does not aggregate the stats across different crawlids. **/
     rpc GetStats(QueueWithinCrawlParams) returns (Stats) {}

yes, I see your point now :) What I really needed was the total nb of URLs per crawl, I got a bit confused by GetStats and the concepts of active URLs (I mixed it at some point with "in process") and I didn't realize that the total number of URLs can be retrieved with Getstats: active + completed URLs....so I guess we can conclude that there is no need for this PR.