[Fuego] selftest instructions

Bird, Tim Tim.Bird at sony.com
Wed Jul 15 03:12:12 UTC 2020


Hey Laszlo,

Sorry for the slow response.  I was out of the office on some personal business over the weekend.

Thanks very much for looking into this.  We definitely need to document how to run this 
Fuego release test.  I have a vague recollection that  sometime after this test was added, something
broke that was hard to fix.  I don't recall what it was, but the test ended up bitrotting, as
you have seen from the out-of-date repository references in the spec.json file.

See http://fuegotest.org/wiki/Issue_0074

The way that Fuego is installed by default has been refactored since the 1.3 release.
The fuego-core directory is now installed as a sub-directory under fuego.
It looks like the 'git clone' commands in tests/Functional.fuego_release_test/fuego_test.sh are
out of date.  I have made a commit to update fuego_test.sh to put fuego-core
underneath the 'fuego' directory, and pushed this to the master branch.

The 0074 bug was related to a package problem between jessie and jessie-updates package
repositories.  We have, since then, switched Fuego to use Debian stretch (slim) as the
base distribution.  So it's possible all this is resolved now.  But I'm pretty sure any of the hardcoded
package versions in the Dockerfile.test  are now out-of-date.
There may be some anachronisms in Functional.fuego_release_test/test_run.sh as well. 

The stuff in Dockerfile.test used to be incorporated into Dockerfile, but it was split out
as a separate set of statements in 2019.  So I think your set of commands to build 
a fuego container first, then a fuego-test container, are correct.

I'm not sure about the port usage when building the fuego-test-container.  I would think
it was better to put it on it's own port (maybe 8095), so as not to conflict with a possible
already-existing fuego-container on port 8090.

I started a new page on the wiki: http://fuegotest.org/wiki/Functional.fuego_release_test
I'll try to collect information about the failure modes and issues with the test.

Maybe we can dust this thing off and make it work again!  That would be nice.
 -- Tim

> -----Original Message-----
> From:  Laszlo Sitzer
> 
> Apologies for me spamming the mailing list : ((
> 
> After digging more .... I came to understand I was on the wrong track when it came to patching the Dockerfile.
> There is a Dockerfile.test that I had never looked into before that that builds and image on top of the fuego image.
> 
> So please ignore my patches for the Dockerfile related to making the release test work.
> My current set of commands to build and run the self test looks like this
> 
> >>>
> ./fuego-host-scripts/docker-build-image.sh fuego 8090 Dockerfile
> ./fuego-host-scripts/docker-build-image.sh fuego-test 8090 Dockerfile.test
> ./fuego-host-scripts/docker-create-container.sh fuego-test fuego-test-container
> ./start.sh fuego-test-container
> ftc add-nodes -b fuego-test
> ftc add-jobs  -b fuego-test -t Functional.fuego_release_test
> ftc run-test  -b fuego-test -t Functional.fuego_release_test
> 
> >>>
> 
> I would be very grateful if someone could point out how I am supposed to setup up and run the release test.
> 
> The current sequence of commands gets me so far that I noticed that fuego-core/tests/Functional.fuego_release_test/spec.json contains
> URLs to outdated git repositories (which I have a patch for but can be fixed by anyone).
> 
> 
> Best,
> 
> Lazlo
> 
> On Tue, Jul 14, 2020 at 12:27 PM Laszlo Sitzer <laszlo.sitzer at native-instruments.de <mailto:laszlo.sitzer at native-instruments.de> > wrote:
> 
> 
> 	So, I found the reason why the release test builds an ancient (debian jessie) docker image.
> 
> 	The fuego_release_test are referencing git repos from https://bitbucket.org/tbirdd20/fuego instead of
> https://bitbucket.org/fuegotest/fuego
> 	I will prepare a branch for fuego-core where this will be fixed as well.
> 
> 	On Mon, Jul 13, 2020 at 8:00 PM Laszlo Sitzer <laszlo.sitzer at native-instruments.de <mailto:laszlo.sitzer at native-instruments.de>
> > wrote:
> 
> 
> 		So I am still trying to run the Functional.fuego_release_test from within the fuego container (as root).
> 
> 		Is there something totally wrong?
> 
> 		>>>
> 
> 		docker exec -ti fuego-container /bin/bash
> 		ftc add-nodes -b fuego-test
> 		ftc add-jobs    -b fuego-test -t Functional.fuego_release_test
> 		ftc run-test     -b fuego-test -t Functional.fuego_release_test
> 
> 		>>>
> 
> 		I managed to get the release test itself to work. It runs and fails as it times out waiting for the Jenkins in the fuego-release-
> container (i suspect).
> 
> 		But I wanted to give an update on what I did in the meantime to get there.
> 		So I had to:
> 		  - bind-mount  /usr/bin/docker into the container (the socket was already there)
> 		  - add jenkins to sudoers that need no password
> 		  - install python3-pip (because test_run.py wants python3) and a couple of other python3 deb and pips
> 
> 		But along the way I tend to run into lots of unexpected curiosities like: when I ran the fuego_relase_test
> 		and it is building the docker image for fuego-release-container (expected) but with debian jessie!? I am still puzzled.
> 
> 		The string "jessie" I could only find in fuego-ro/conf/fuego.conf in the line "server_squad_project=jessie".
> 		I suspect the release test somehow managed to pick up either an outdated git repo, branch, tar archive or alike.
> 
> 		Enough for today ; )
> 
> 		Would be nice to hear if anyone can execute release tests successfully.
> 
> 
> 		Best,
> 
> 		Lazlo
> 
> 		PS: I am very tempted to start a CI discussion ... but not today
> 
> 		On Sun, Jul 12, 2020 at 8:00 PM Laszlo Sitzer <laszlo.sitzer at native-instruments.de <mailto:laszlo.sitzer at native-
> instruments.de> > wrote:
> 
> 
> 			Found the documentation in Functional.fuego_release_test/README.md.
> 			So I run the following from the docker host
> 
> 
> 			>>>
> 			docker exec -ti fuego-container /bin/bash
> 			ftc add-nodes -b fuego-test
> 			ftc add-jobs    -b fuego-test -t Functional.fuego_release_test
> 			ftc run-test     -b fuego-test -t Functional.fuego_release_test
> 			>>>
> 
> 			Next problem I run into is that "sudo" is called at some point either by the test script or by the fuego host scripts
> from the fuego under test.
> 			I hotfixed by allowing the "jenkins" user inside the docker container to use sudo without a password.
> 
> 			>>>
> 			echo "jenkins   ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/no-passwd
> 			>>>
> 
> 
> 			So I run the 'run-test' command again .... and end up with another problem.
> 
> 			>>>
> 			Traceback (most recent call last):
> 			  File "/fuego-core/tests/Functional.fuego_release_test/test_run.py", line 15, in <module>
> 			    import docker
> 			ImportError: No module named 'docker'
> 			>>>
> 
> 			Okay. So I install python-docker in the container (apt-get -q install -y python-docker) ... and run again.
> 			Same "No module named 'docker'" error. Which leaves me a bit puzzled as I installed python-docker
> 			as root using apt in the container ... and whoever is using it should be able to pick it up (assuming the
> 			jenkins user is executing the tests when I run ftc as root).
> 
> 			I will keep on digging and fixing. Just wanted to check if that is expected or I am doing something fundamentally
> wrong here.
> 			Just to be sure, I am trying to run the self-tests against the current master of Fuego (not the branch that I plan to
> actually verify).
> 			If someone could confirm Fuego self-tests are working for the out of the box?
> 
> 
> 
> 			Best,
> 
> 			Lazlo
> 
> 
> 
> 			On Sun, Jul 12, 2020 at 6:43 PM Laszlo Sitzer <laszlo.sitzer at native-instruments.de <mailto:laszlo.sitzer at native-
> instruments.de> > wrote:
> 
> 
> 				Hello!
> 
> 
> 				I am working on updating the Fuego docker image and want to make sure I don't break anything. I notice
> in Fuego v1.3 a self-test was introduced.
> 
> 				I wanted to check if the following sequence is sufficient to test Fuego works correctly:
> 
> 				>>>
> 				ftc add-nodes -b docker
> 				ftc add-jobs -b docker -t Functional.fuego_release_test
> 				ftc run-test -b docker -t Functional.fuego_release_test
> 				>>>
> 
> 				Best,
> 
> 				Lazlo



More information about the Fuego mailing list