I am facing a strange problem with logging when I set up a network of three docker containers each running a small flask app.
My python application looks as follows:
from flask import Flask, requestimport sysimport loggingimport logging.handlersimport osapp = Flask(__name__)logger = logging.getLogger('Logger')logger.setLevel(logging.DEBUG)log_handler = logging.StreamHandler(sys.stdout)logger.addHandler(log_handler)service_name=""if 'SERVICE_NAME' in os.environ: service_name = os.environ['SERVICE_NAME']logger = logging.LoggerAdapter(logger, {"serviceName" : service_name})@app.route("/ping")def ping(): logger.info(f'answering ping request') return f'{service_name} OK'if __name__ == "__main__": logger.info("Started") app.run(debug=True, host='0.0.0.0')
I start three instances of this in a docker network behind nginx:
docker network create tNetdocker run -it -d -p 24224:24224 -v `pwd`:/fluentd/etc -e FLUENTD_CONF=fluentd.conf --name=fluentd_logging fluent/fluentddocker run -d --log-driver=fluentd --name ms1 -e SERVICE_NAME="ms1" --network tNet testflaskdocker run -d --log-driver=fluentd --name ms2 -e SERVICE_NAME="ms2" --network tNet testflaskdocker run -d --log-driver=fluentd --name ms3 -e SERVICE_NAME="ms3" --network tNet testflaskdocker run -d --name=nginx --network tNet --volume `pwd`/nginx.conf:/etc/nginx/conf.d/default.conf -p 2000:2000 -p 3000:3000 nginx
The nginx configuration is as follows:
upstream app { server ms1:5000; server ms2:5000; server ms3:5000;}server { listen 2000; location / { proxy_pass http://app; }}
My fluent configuration looks as follows:
<source> @type forward</source><match> @type stdout</match>
When I use curl I can see that each instance of the service is responding:
curl localhost:2000/pingms1 OKcurl localhost:2000/pingms2 OKcurl localhost:2000/pingms3 OK
When I however inspect the logs I can only see logs from the 3rd instance of the service:
docker logs <fluent container ID> | grep -Eis "answering ping"19-08-06 15:34:46.000000000 +0000 be338a3d5aa0: {"container_id":"be338a3d5aa0b8920366fdf1ba2a267082cee47e681adcea16b6c51c9c82cf0d","container_name":"/ms3","source":"stdout","log":"answering ping request"}
Has anyone an idea? Thank you in advance.