[Fuego] Fuego docker container size keeps growing and hits maximum storage capacity

Tim.Bird at sony.com Tim.Bird at sony.com
Fri Nov 16 23:44:43 UTC 2018


Wow. I didn't know you could copy a file to a non-running container! That's handy to know. I'm glad you were able to find a workaround. We should add that to our docket tips page: http://fuegotest.org/wiki/Docker_Tips

- - Tim


-------- Original Message --------
Subject: RE: RE: RE: [Fuego] Fuego docker container size keeps growing and hits maximum storage capacity
From: Dhinakar Kalyanasundaram <dhinakar.k at samsung.com>
Date: Nov 16, 2018, 3:43 AM
To: "Bird, Timothy" <Tim.Bird at sony.com>,fuego at lists.linuxfoundation.org

Hi Tim,



Just an update to the issue that I mentioned at the end of my email.

I was able to find a way to fix the issue that I made because of which the container was not starting.

I got the container id via 'docker ps -a' command and replaced the 'etc/default/jenkins' file with the working one using 'docker cp' command as shown below.

Then I was able to start the docker container successfully.



$ sudo docker cp jenkins 6b4e6e63rfg7:/etc/default/



Regards,

Dhinakar





--------- Original Message ---------

Sender : Dhinakar Kalyanasundaram <dhinakar.k at samsung.com> Staff Engineer/FDS SW /SSIR/Samsung Electronics

Date : 2018-11-14 18:57 (GMT+5:30)

Title : RE: RE: [Fuego] Fuego docker container size keeps growing and hits maximum storage capacity

To : null<Tim.Bird at sony.com>, null<fuego at lists.linuxfoundation.org>

CC : Dhinakar Kalyanasundaram<dhinakar.k at samsung.com>



Hi Tim,



I have disabled 'DNSMultiCast' temporarily via Manage Jenkins -> System Log -> Log Levels (on the left)
and also deleted the /var/log/jenkins/jenkins.log file.

I tried a few things to permanently disable 'DNSMultiCast' inside the container (/etc/default/jenkins file)

Add the below JAVA_ARGS and restarted fuego but did not see it getting added under http://107.108.34.124:8080/fuego/systemInfo

JAVA_ARGS="-Djava.awt.headless=true -Dhudson.DNSMultiCast.disabled=true"



I also tried -> JAVA_ARGS="-Dhudson.DNSMultiCast.disabled=true" but did not work. Only Manage Jenkins -> System Log -> Log Levels (on the left) works.

Is there any way to permanently disable it within the container, so that each time I restart the container I don't have to do this -> Manage Jenkins -> System Log -> Log Levels (on the left)

Also, I tried setting the timezone to 'Asia/Kolkata' in /etc/default/jenkins file (add below arguments) but it also never gets updated

JAVA_ARGS="-Dorg.apache.commons.jelly.tags.fmt.timeZone=Asia/Kolkata"

Please let me know how to set timezone permanent as well.



Basically I want to keep everything updated in the container so that I can easily port it across machines.



During the whole process there was an interesting observation detailed below.

To edit timezone to 'Asia/Kolkata' I tried many things in /etc/default/jenkins file.

In one such case, I appended the below phrase to 'JENKINS_ARGS' in /etc/default/jenkins file, which seems erroneous,

JENKINS_ARGS="--webroot=/var/cache/jenkins/war --httpPort=8080 --prefix=/storm --Dorg.apache.commons.jelly.tags.fmt.timeZone=Asia/Kolkata"

saved the file, exited fuego, and tried to restart the container but it failed to start as shown below,



dhinakar at userl2:~/fuego$<mailto:dhinakar at userl2:~/fuego$> ./fuego-host-scripts/docker-start-container.sh fuego-docker-container-12thNov2018

Starting Fuego container (fuego-docker-container-12thNov2018)
[FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
Please create Fuego docker container via docker-create-container.sh script



So the question is, if we make such mistake to /etc/default/jenkins file and not able to start the container, is there a way to fix it from outside the container?

Because in my case I had to create a new container, so lost all changes done inside my previous container.

Regards,

Dhinakar





--------- Original Message ---------

Sender : Tim.Bird at sony.com <Tim.Bird at sony.com>

Date : 2018-11-14 13:13 (GMT+5:30)

Title : RE: [Fuego] Fuego docker container size keeps growing and hits maximum storage capacity

To : Tim.Bird at sony.com, Dhinakar Kalyanasundaram<dhinakar.k at samsung.com>, fuego at lists.linuxfoundation.org



This seems like a band-aid rather than a fix, but here's some information
about turning down the logging for Jenkins.  A proper fix would be to figure
out what error in your networking setup is causing the DNS errors, and fix
that.  But disabling logging might help.

Here's the logging level fix:
https://stackoverflow.com/questions/31719756/how-to-stop-jenkins-log-from-becoming-huge

See the last answer in that thread.  It has a workaround to turn off DNSMultiCast,
which might be the cause of the problem.
 --  Tim



> > -----Original Message-----
...

>
> Looks like it could be related to this bug in Jenkins, having to do with
> multicast:
> https://issues.jenkins-ci.org/browse/JENKINS-25369
>
> I would truncate the log file, stop the container, and see if there's some way
> to
> disable DNSMultiCast.  Something appears to have gone wrong with your
> network
> configuration.  It could be the docker virtual network, or maybe something
> has
> changed about your physical networks.
>
>  -- Tim
>
> _______________________________________________
> Fuego mailing list
> Fuego at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego


[cid:XOK0LK7CT9SZ at namo.co.kr]



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/fuego/attachments/20181116/9c00bfb6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 201602111742151_N3WZA6X7.png
Type: image/png
Size: 33527 bytes
Desc: 201602111742151_N3WZA6X7.png
URL: <http://lists.linuxfoundation.org/pipermail/fuego/attachments/20181116/9c00bfb6/attachment-0001.png>


More information about the Fuego mailing list