Docker, iptables and csf restarts

[ibertrix]

Hi,

I have been trying to configure CSF and Docker under a Plesk server. There are many posts in forums reporting that when Docker creates a NAT redirect to certain port, that port is exposed to the entire world.

I tried to use this csfpost tool https://github.com/juli3nk/csf-post-docker but apparently It hasn´t worked.

In some way, installing netfilters tool for saving iptables rules I have managed to store a set of iptables rules under /etc/iptables/rules.v4 that work.

Apparently Docker is setting up a DOCKER-USER chain where so I managed to isolate Docker only within the server using this iptables rule:

-A DOCKER-USER -i ethernet_external_interface -j DROP

Now If I restart the server or only the iptables service, the configuration works, Docker works and port is not exposed publicly.

If I restart csf or run csf -r and then some of the DOCKER rules are removed so then I have to restart iptables again, so that way I have csf running and Docker rules are back in place.

If I grep under /etc/iptables/rules.v4 all docker related entries I get:

:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A INPUT -i docker0 -j ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o docker0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8443 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -i ens18 -j DROP
-A DOCKER-USER -j RETURN
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 32743 -j DNAT --to-destination 172.17.0.2:8443
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 32780 -j DNAT --to-destination 172.17.0.2:80

I have tried adding this to /etc/csf/csfpost.sh

iptables-restore < /etc/iptables/rules.v4

apparently it works but I am wondering if this is the way to go?

[panomitrius]

This is pretty much the only hack I've manage to find anywhere that seems to solve the issue of having docker work with CSF without exposing all containers to global connections. Could you please describe how you went about to distill and save the iptables rules?

[alsur]

We are trying to clarify a similar setup. We want to limit via CSF all incoming external traffic to docker unless allowed in the configuration.

With a personalised csfpost.sh generating our own docker rules (disabled in docker server) we've managed it to work but involves reloading every time we create any new container in docker.

With "DOCKER=1" in csf we have not managed for it to work

Currently testing this solution: https://serverfault.com/questions/10204 ... f-firewall

I must admit that the CSF documentation for docker is not at all clear of helpfull.

[alsur]

No help here but after long research hours we found our solution via 2 scripts in csfpre and csfpost.sh.

Most solutions talk only about docker0 but that is only the default bridge network, any other private bridge created will get a new name and require additional rules.

We tested the great setup refered at viewtopic.php?t=12046 but that solution is intended for docker to work as-is together with CSF, this is... all docker exposed ports are automatically exposed and CSF has no control over them. It is basically a DNAT solution and like in standard CSF in NAT there are no filter rules applied.

As said we solved it so far, pending some additional testing by creating standar masquerade rules on NAT so the docker bridge networks can have external traffic, forward rules for each bridge network on docker together with an accept rule on the output for each of those too. Like this we have managed for CSF to rule over whatever we setup in docker. So if you want a port publicly exposed it will require an open port or valid source or dynamic IP on CSF.

[barart]

No help here but after long research hours we found our solution via 2 scripts in csfpre and csfpost.sh.

Most solutions talk only about docker0 but that is only the default bridge network, any other private bridge created will get a new name and require additional rules.

We tested the great setup refered at viewtopic.php?t=12046 but that solution is intended for docker to work as-is together with CSF, this is... all docker exposed ports are automatically exposed and CSF has no control over them. It is basically a DNAT solution and like in standard CSF in NAT there are no filter rules applied.

As said we solved it so far, pending some additional testing by creating standar masquerade rules on NAT so the docker bridge networks can have external traffic, forward rules for each bridge network on docker together with an accept rule on the output for each of those too. Like this we have managed for CSF to rule over whatever we setup in docker. So if you want a port publicly exposed it will require an open port or valid source or dynamic IP on CSF.

Did you manage how to make this work?

[alsur]

I reopen this old thread as we are back to trying to find a correct working solution for CSF with docker.
We've had docker servers behind csf for a while by creating various sets of complicated bash scripts that analyze docker networks and creates required rules. Obviouslly csf needs reloading for every new docker container network created.

Our only problem so far has been that no matter how source IP never reaches the container... for some reason our rulling (masquerade I guess) is not working as expected.

As we are trying to redo the whole setup now it would be great to know the current situation of CSF before proceeding. I have the impression there is currently a lack of updates and I am uncertain about the future of this great tool.

[mpiber]

I'm also waiting for the same solution, when activating CSF on my server I can't make internal communication between them. I tried to configure "csfpre.sh" but without success!

[Sergio]

@mpiber,
You wrote "when activating CSF on my server I can't make internal communication between them".

Check on CSF logs what is blocking the IP and post it here.

Sergio