[Fuego] Fuego v1.4 not able to add nodes

Tim.Bird at sony.com Tim.Bird at sony.com
Fri Aug 9 23:25:35 UTC 2019



> -----Original Message-----
> From: Dhinakar Kalyanasundaram 

> 
> Hi Tim,
> 
> 
> 
> Thanks for your prompt response.
> 
> Please find my response below.
> 
> 
> 
> Here are some things you can try:
> 1) in ftc, remove the try:/except block around the call to do_add_nodes,
> try the 'ftc add-nodes' command, and see what the traceback looks like.
> I'm guessing it's going to look a lot like the traceback for the list-nodes call.
> The first call to the server inside the do_add_nodes() function is to
> server.get_nodes(), to check for a duplicate.  I'm guessing this is throwing
> the same exception.
> 
> 
> 
> Yes you are right, after removing the try:/except block, I get the same
> exception as shown below:
> 
> root at linux-2030047407:/fuego-rw <mailto:root at linux-2030047407:/fuego-
> rw> # ftc list-nodes
> Traceback (most recent call last):
>   File "/usr/local/bin/ftc", line 5275, in <module>
>     main()
>   File "/usr/local/bin/ftc", line 5231, in main
>     do_list_nodes(conf)
>   File "/usr/local/bin/ftc", line 2062, in do_list_nodes
>     node_list = [node['name'] for node in server.get_nodes()]
>   File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line 1447,
> in get_nodes
>     requests.Request('GET', self._build_url(NODE_LIST, locals()))))
>   File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line 564,
> in jenkins_open
>     return self.jenkins_request(req, add_crumb, resolve_auth).text
>   File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line 596,
> in jenkins_request
>     raise NotFoundException('Requested item could not be found')
> jenkins.NotFoundException: Requested item could not be found
> 
> 
> 2) try manually adding a node, just to see if that clears the error.
> ftc list-nodes should show your manually created node.
> 
> 
> Actually I tried this yesterday itself, I get the same exception as mentioned
> above even after adding a 'Test' node (as shown in the picture below).
> 
> 
> 
> root at linux-2030047407:/fuego-rw <mailto:root at linux-2030047407:/fuego-
> rw> # ftc list-nodes -> exception is same as above.

Just out of curiosity - do you have more than one instance of Jenkins running?
Is it possible that ftc is trying to contact the wrong jenkins instance?

> 
> If it is installation of fuego or Jenkins issue then it is fine I will redo it.

I looked to see if there was maybe a file you could replace in /var/lib/jenkins
(like under the 'nodes' directory, or something), that might have
gotten corrupted.

I couldn't see anything obvious.  The master node doesn't even show
up as a directory under /var/lib/jenkins/nodes, so it must be handled
differently from the other nodes.

> 
> Only thing is I have updated jenkins and installed a lot of plugins related to
> git, gerrit, pipeline etc.
> 
> Is there is a way I can retain those plugin changes while I reinstall/overwrite
> the existing installation? or I need to do a 'clean install' only.
> 
> If it is Jenkins installation issue, is it possible to reinstall Jenkins alone?


Well, this might be tedious, but you might try a new installation of Jenkins,
and then grabbing the contents of /var/lib/jenkins (the xml files at the root directory)
in the new container, tar-ring them up and putting them in the old container,
and doing a diff, to see what's different about the new xml files.

You might be able to figure out the problem if you see something different 
in the files for the new jenkins instance (the broken one).

In particular, I'd look at /var/lib/jenkins/config.xml, and examine the
content of the <nodeProperties> entity.

If you find a difference in config.xml, or some other top-level xml file,
you may be able to repair just that file.  My experience is that when
Jenkins is restarted, if you haven't touched the plugins directory, none
of your plugins will be affected.

By the way - you have restarted jenkins in the container haven't you?

If not, try this inside the container:
  # service jenkins stop
  # service jenkins start

That's about it for ideas from me, I'm afraid.
 -- Tim


> 
> 
> Regards,
> 
> Dhinakar
> 
> 
> 
> 
> 
> --------- Original Message ---------
> 
> Sender : Tim.Bird at sony.com <Tim.Bird at sony.com>
> 
> Date : 2019-08-09 07:02 (GMT+5:30)
> 
> Title : RE: Fuego v1.4 not able to add nodes
> 
> 
> 
> > -----Original Message-----
> > From: Dhinakar Kalyanasundaram
> >
> > Dear Fuego Users,
> >
> > I am setting up test environment with fuego v1.4
> >
> > dhinakar at linux-2030047407:~/fuego$ <mailto:dhinakar at linux-
> > 2030047407:~/fuego$>  cat VERSION
> > # This project adheres to Semantic Versioning.  See http://semver.org/
> > # The codename for this release is "Ember"
> > VERSION=1.4.0
> >
> >
> >
> > I am able to list the boards but not able to add nodes (please check error
> > below).
> >
> >
> >
> > root at linux-2030047407:/fuego-rw <mailto:root at linux-2030047407:/fuego-
> > rw> # ftc list-boards
> > Boards in this system:
> >    ARM64-SSH-Test
> >    ARM64-SSH-Test1
> >    ARM64-Serial-Test
> >    X86-SSH-Test
> >    X86-Serial-Test
> >    bbb
> >    beaglebone-serial
> >    docker
> >    ftc-test
> >    fuego-test
> >    lager
> >    local
> >    localhost
> >    minnow
> >    qemu-arm
> >    raspberrypi3
> >    template-dev
> >
> >
> > root at linux-2030047407:/fuego-rw <mailto:root at linux-2030047407:/fuego-
> > rw> # ftc add-nodes -b ARM64-SSH-Test
> > Requested item could not be found
> > Usage: ftc add-nodes [-f] -b <board1>[,<board2>...]
> >   By convention the name of the node and board is the same. Also,
> >   the corresponding board file (<board_name>.board) must exist.
> >   Use list-boards to see available boards.
> >
> >   Use -f for "force" mode. This tries to remove the node before adding it
> >
> > One strange thing I noticed is that, I am not able to see 'master' node in
> > fuego dashboard.
> >
> > Please check the image below:
> >
> >
> >
> > But it is listed when I navigate to
> > http://101.102.103.104:8090/fuego/computer/
> >
> >
> >
> > The output of 'ftc list-nodes' command is given below.
> >
> > Should it list 'master node' atleast or it is the correct output since no nodes
> > are added yet via 'ftc add-nodes' command?
> >
> > root at linux-2030047407:/fuego-rw <mailto:root at linux-2030047407:/fuego-
> > rw> # ftc list-nodes
> > Traceback (most recent call last):
> >   File "/usr/local/bin/ftc", line 5275, in <module>
> >     main()
> >   File "/usr/local/bin/ftc", line 5231, in main
> >     do_list_nodes(conf)
> >   File "/usr/local/bin/ftc", line 2062, in do_list_nodes
> >     node_list = [node['name'] for node in server.get_nodes()]
> >   File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line
> 1447,
> > in get_nodes
> >     requests.Request('GET', self._build_url(NODE_LIST, locals()))))
> >   File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line
> 564,
> > in jenkins_open
> >     return self.jenkins_request(req, add_crumb, resolve_auth).text
> >   File "/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py", line
> 596,
> > in jenkins_request
> >     raise NotFoundException('Requested item could not be found')
> > jenkins.NotFoundException: Requested item could not be found
> >
> >
> > Is it a problem with my installation of fuego or any missing plugin?
> 
> I think this might be a Fuego or Jenkins installation problem.
> There is code in 'ftc' to skip the 'master' node when it is returned
> from a call to server.get_nodes() internally.
> 
> It looks like this call is having problems.
> 
> The photo you supplied looks normal.  There should be no problem with the
> python jenkins module retrieving the list of nodes from  your system.
> 
> Here are some things you can try:
> 1) in ftc, remove the try:/except block around the call to do_add_nodes,
> try the 'ftc add-nodes' command, and see what the traceback looks like.
> I'm guessing it's going to look a lot like the traceback for the list-nodes call.
> The first call to the server inside the do_add_nodes() function is to
> server.get_nodes(), to check for a duplicate.  I'm guessing this is throwing
> the same exception.
> 2) try manually adding a node, just to see if that clears the error.
> ftc list-nodes should show your manually created node.
> 
> I don't think this will resolve the issue, but maybe some information
> will surface that will help us track down the problem.
>  -- Tim
> 
> >
> > If anyone experienced similar issue please help me out.
> >
> >
> >
> > Regards,
> >
> > Dhinakar
> >
> >
> >
> >
> >
> <http://ext.samsung.net/mail/ext/v1/external/status/update?userid=dhina
> > kar.k&do=bWFpbElEPTIwMTkwODA4MTU1NTQ0ZXBjbXM1cDdkY2E3YThlM
> >
> WIzZmRkMTc2NDNiYTVhNjkwMDFkMmFmMyZyZWNpcGllbnRBZGRyZXNzP
> > VRpbS5CaXJkQHNvbnkuY29t>
> 
> 
> 
> 
> 
> 
> 
> <http://ext.samsung.net/mail/ext/v1/external/status/update?userid=dhina
> kar.k&do=bWFpbElEPTIwMTkwODA5MTIxMjI5ZXBjbXM1cDIzNWFmMzE0M2
> FiNWMxYmYyYjQyNzEzY2NhNTYwNGM2MiZyZWNpcGllbnRBZGRyZXNzPVRp
> bS5CaXJkQHNvbnkuY29t>


More information about the Fuego mailing list