[Fuego] Fuego docker container size keeps growing and hits maximum storage capacity

[Tim.Bird at sony.com]

> -----Original Message-----
> From: Dhinakar Kalyanasundaram 
> 
> Dear Fuego Users,
> 
> I am using Fuego version 1.3.0 (see below) in one of my Linux system
> (Ubuntu 16.04 LTS).
> 
> # This project adheres to Semantic Versioning.  See http://semver.org/
> # The codename for this release is "Draft"
> VERSION=1.3.0

Can you do: "git describe" in your fuego-core directory, and tell me what
it says?

> 
> I noticed that the fuego container size keeps growing continuously and hits
> the maximum storage capacity in a day or two.
> 
This is a serious problem.  I think it's likely that there's some runaway logging
going on inside the container.  It would be good to isolate this and see what's
going on.  It is possible some recent Docker changes have introduced a
new program that logs more than it should.
Here are some questions and some advice for finding the problem.

If you exit the container and restart it, does the size reset?
Do any files outside the container get bigger (e.g. in fuego-ro or fuego-rw 
on your host system)?

Please try to find the file inside the container that is getting big.
First look at logging, and if it's not something obvious in there, then
look other places.

Inside the container, do:
 $ cd /var/log
 $ du -shx * | sort -h

You can use the 'du | sort' sequence in various directories to try to track
down where the size is accumulating.  Since your container is at 45G, this will take
a long time if you run it at the root of the container filesystem, but that's probably
the best place to start.

> 
> It grows at the rate of 1GB per minute
> 
> root at userl2:/ <mailto:root at userl2:/> # docker ps -s
> CONTAINER ID        IMAGE                        COMMAND                  CREATED
> STATUS              PORTS               NAMES                                         SIZE
> 632433hg24324        fuego-docker-image   "/bin/sh -c 'serviceâ¦"   19 hours
> ago        Up 2 hours                              fuego-docker-container-12thNov2018
> 40.9GB (virtual 42.7GB)
> 
> 
> In the next 30 mins I see below:
> root at userl2:/ <mailto:root at userl2:/> # docker ps -s
> CONTAINER ID        IMAGE                        COMMAND                  CREATED
> STATUS              PORTS               NAMES                                         SIZE
> 6bdacdafd2343        fuego-docker-image   "/bin/sh -c 'serviceâ¦"   19 hours
> ago        Up 3 hours                              fuego-docker-container-12thNov2018
> 45.9GB (virtual 47.7GB)
> 
> 
> I am not sure if this issue is related to current version of Fuego framework
> but I don't see this issue in previous Fuego versions installed on my other
> Ubuntu systems.
> 
> I am using Ubuntu 16.04 LTS version across all my Linux systems.

I'm using Ubuntu 16.04 LTS on my Fuego host, and the latest Fuego version,
and I haven't seen this.  My guess is that some error is triggering some repeated
message to a log.
Once we identify the file or directory that is growing inside the container,
we should be able to pinpoint the issue.
 -- Tim