Why am I getting more hits for "Go-http-client"? I want to learn more about this bot.

Question

We have a website, I have placed robots.txt file, in the file I have disallowed "go-http-client". But even then I can see more hits from this crawler. I want to understand How the hits are calculated, and why is this go-http-client crawler is used for?
I have 8K visitor on my website, 32K page views, but 2 lakhs hit and over all it is consuming 48GB bandwidth.
Can you please help me in understanding why this crawler "go-http-client" is used for.

JWP (John) · Accepted Answer

Hi Venkatesh
Just in case it helps, here are the 'official' Google bots...
https://developers.google.com/search/docs/advanced/crawling/overview-google-crawlers

barryhunter · Accepted Answer

Not sure why you expect us to know. You would need to find the author ot controller of this bot to see what it doing. 
robots.txt is a honorary protocol. You provide rules to bots, but it's unto the bots to actually honour them. 
It would be upto the developer to implement logic for honouring robots.txt. And if they were going to do that, they would probably set a better User-Agent header, so that can find more information about their bot, and in particular find out what 'token' from robots.txt they would be willing to honour.