Excessive usage of queries to github.com, Pi-hole v6, Docker, development tag

[akordowski]

Versions

Platform

• OS and version:
  Raspbian

• Platform:
  Raspberry Pi, Docker

Actual behavior / bug

After a long time I checked today the Pi-hole dashboard and have seen an excessive usage of queries to github.com, in the last 24h it was over 94k. I run the development version of Pi-hole v6 in a Docker container on a Raspberry Pi.

My investigation have shown that the queries are triggered by the Pi-hole Docker container network. As the screenshot below shows this queries are triggered every 3 seconds or so.

What causes this behaviour? Is there a way to fix it? Any help is much appreciated. Thank you in advance.

[rdwebdesign]

This is not normal.

What causes this behaviour?

Usually Pi-hole container only connect to Github to check for updates, but only once a day.
Maybe a device using Pi-hole as DNS server is constantly trying to access github.com.

We need more information.
Please generate a debug log, upload it and post here only the Token.

EDIT:
Please, also post your compose file or docker run command used to start the container.

[akordowski]

Thank you for your response. I attached the files.

Here also some screenshots how the dashboard looks like after 24h of running. I hope you can find something. I'm bit concerned as I have not observed such behavior in previous versions.

pihole_debug.log
I redacted the Clients and Adlists segments.

docker-compose.yml

version: "3.8"
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:development
    hostname: pi.hole
    environment:
      - TZ=${TZ}
      - FTLCONF_webserver_api_password=${PASSWORD}
      - FTLCONF_dns_upstreams=${DNS_UPSTREAMS}
      - DNSMASQ_LISTENING=all
    ports:
      - "53:53/tcp"
      - "192.168.178.2:53:53/udp"
      - "80:80/tcp"
      - "443:443/tcp"
    volumes:
      - ./etc/dnsmasq.d:/etc/dnsmasq.d
      - ./etc/pihole:/etc/pihole
    restart: unless-stopped

[akordowski]

Maybe a device using Pi-hole as DNS server is constantly trying to access github.com.

I don't have any device that would do that. I also disabled the notifications for the GithHub app, but the queries with the same frequency are logged. I also disabled other containers to exclude other possible sources. No change. The queries seem to come from Pi-hole as the 172.20.0.1 IP is the of the Pi-hole's Docker bridge.

[akordowski]

@rdwebdesign I don't know what the team has done but the image was updated on 2024-11-10 and since then the issue is gone. Therefore, I will close this issue. Thank you!

[akordowski]

@rdwebdesign Today I noticed that the issue is back again with the latest update of the development docker image. Any idea what can cause it?

[rdwebdesign]

Any idea what can cause it?

On one else is complaining about this.
It seems to be a local issue in your network.

The IP 172.20.0.1 is not Pi-hole container IP.
This is actually docker gateway IP used by the network called pihole_default. (The container IP is probably 172.20.0.2).

This means some app, service or another container in your network is generating this DNS queries and docker is passing them to Pi-hole.

[akordowski]

Thank you for the quick response.

It seems to be a local issue in your network.

The only GitHub related app I have is the iOS GitHub app. I turned off all notifications and devices and the github.com DNS queries are still comming every 2-3 seconds. And the same pattern is seen over 24 hours unchanged, even with devices turned off. The strange thing is that with the image update from Nov 10 the issue was gone and is back again with the latest update from Nov 22. So my conclusion is that is somehow related with the PiHole developmen container.

But I will keep eye on it and see how it behaves with the next update.
Thank you for your work!

[akordowski]

Do you have maybe an hint how to download the previous image, so I could verify my assumption?

[rdwebdesign]

This is really looking like a local issue in your network or in your host machine, unrelated to Pi-hole.

The most recent pihole/pihole:development image was released 10 days ago. This image only changes when we change something in the way the docker image is created.

If you want to use the most recent v6 image, please try pihole/pihole:nightly.
If the issue persists with nightly, please generate a new debug log, upload it and post here only the Token.

[akordowski]

This is really looking like a local issue in your network or in your host machine, unrelated to Pi-hole.

Running the latest pihole/pihole:development container I disabled all devices, docker containers (except PiHole) and other sources, and the queries were still comming every 2-3 seconds.

I just updated the image to the pihole/pihole:nightly and the queries stoped. So my conclusion is that it must be PiHole related, in which way I can not say.

But thank you for the hint, I will leave the container on the last nightly build for some time.

[akordowski]

I could provide a screen recording if it would help you finding/analyzing the issue.

[rdwebdesign]

It's not necessary.

As explained nightly has the most current v6 branches for core, web and FTL.

The development image is only updated when we change Docker Pi-hole code. All Pi-hole components (core, web and FTL) inside this image are using slightly outdated branches, so there is no value in debugging this image if nightly is working.

[akordowski]

Ok, thank you. But it is very strange that this issue is present on one build and on another it is absent. There must be some root cause for it.

I will close the issue for now.

[akordowski]

Hi @rdwebdesign. I am reopening this issue as I made some new observations regarding this issue.

I had a PiHole container in use with a nightly build image created on 2025-01-05, which worked fine. On the 2025-01-11 I noticed that at the time 4:00 UTC+1 the excessive queries to github.com was triggered out of nowhere, as you can see in the image below.

Then around 19:00 UTC+1 I rebuilded the container with the latest nightly build image and the issue was gone and the queries stayed stable.

Over the last month I noticed that this issue is recurring every 6 days. So my conclusion is that the issue is not related to a specific Docker image, nevertheless there must be something in the codebase which triggers this behavior. With that I can exclude that this is a network problem on my side.

Maybe the team have an idea what cause this issue. Looking forward to your feedback.

[PromoFaux]

Silly question - do any of your other containers share the IP address of 172.20.0.1 (or use that network? I'm a bit rusty when it comes to Docker networking, personally I use a MACVLAN network so that each container has a real IP on my home network)?

There is nothing in the container that cause quite so many lookups of github.com, so I'm quite stumped. I've also not seen this in either of the instances that I run, nor has anyone else reported something similar - which leads me to believe it's something unique to your setup... maybe!

According to your previously attached debug log, the IP address of your Pi-hole container is 172.20.0.2, but the queries are coming from172.20.0.1, to re-quote RD's response before:

The IP 172.20.0.1 is not Pi-hole container IP.
This is actually docker gateway IP used by the network called pihole_default. (The container IP is probably 172.20.0.2).
This means some app, service or another container in your network is generating this DNS queries and docker is passing them to Pi-hole.

Are there other containers that are using the same bridge network, and therefore passing queries through the gateway and onto Pi-hole?

[akordowski]

@PromoFaux Thank you for the response.

do any of your other containers share the IP address of 172.20.0.1

No, I have only 6 containers running (including PiHole) and PiHole is the only one which uses the IP 172.20.0.1 as the Network Gateway.

I already have tested it with all disabled containers and devices and could still observe the queries. I also find it strange that the queries are triggered out of nowhere middle in the night, every 6 days and that they are gone when I rebuild the container with the latest image.

There is nothing in the container that cause quite so many lookups of github.com

As @rdwebdesign wrote here:

Usually Pi-hole container only connect to Github to check for updates, but only once a day.

Maybe it is related with this feature in some way?

To be clear, this issue appears only with the v6 development version of the PiHole container. With the v5 I never experienced an issue like that.

I also don't have any idea what could be the cause. I guess the only way is to monitor it and rebuild the container on regular basis.

[rdwebdesign]

Maybe it is related with this feature in some way?

Pi-hole v6 checks for updates exactly the same way v5 does... only once a day.

What is the output of cat /etc/crontabs/root inside your v6 container?

[akordowski]

57 4 * * 6 PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole updateGravity >/var/log/pihole/pihole_updateGravity.log || cat /var/log/pihole/pihole_updateGravity.log
00 00 * * * PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole flush once quiet
25 19 * * * PATH="$PATH:/usr/sbin:/usr/local/bin/" pihole updatechecker

[rdwebdesign]

Pi-hole only makes requests to Github:

• during installation;
• when it checks for updates;
• on (automatic or manual) gravity updates.

The crontab output shows your container updates gravity once a week, at 04:57 every Saturday. It also checks for updates daily, at 19:25. Nothing automatically happens at 4:00.

There is something else in your network making the requests you are seeing.

[akordowski]

There is something else in your network making the requests you are seeing.

If that were a network problem, then I should see

• an effect on the query rate when disabling containers and devices
• no changes in the query rate after rebuilding the container with a new image

and that is not the case.

I tired everything to narrow down the issue and all points to the PiHole container.

Do you maybe have an idea what I could do to inspect the network? I will also observe the container the next days to see if the issue occurs after the 6 days.

[yubiuser]

Are you still using the docker-compose you posted above?

Could you try to remove the following lines and see what happens

      - DNSMASQ_LISTENING=all
      - "192.168.178.2:53:53/udp"

If the issue persists, would you be able to run a package capture (e.g. wireshark) on the host and bind to the docker network bridge to inspect the traffic ? This might give us a clue where the queries come from

https://stackoverflow.com/questions/39362730/how-to-capture-packets-for-single-docker-container

Add

Are you using a reverse proxy?

[akordowski]

Are you still using the docker-compose you posted above?

I used docker-compose for the initial build, the container rebuild is done by Portainer.

Could you try to remove the following lines and see what happens

Are these not required to run PiHole properly?

If the issue persists, would you be able to run a package capture

Thanks for the link, I will try to go with tcpdump when the issue occurs again. Maybe this will give some hints.

Are you using a reverse proxy?

No.

[rdwebdesign]

Are these not required to run PiHole properly?

• DNSMASQ_LISTENING=all is just an option.
• "192.168.178.2:53:53/udp" - You can remove the IP and just use 53:53/udp

[akordowski]

Thanks, I will try.