[Fuego] [PATCH 4/4] ftc: add remove-logs flag to rm-jobs
Daniel Sangorrin
daniel.sangorrin at toshiba.co.jp
Fri Aug 4 03:59:34 UTC 2017
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird at sony.com]
> Sent: Friday, August 04, 2017 12:14 PM
> To: Daniel Sangorrin; fuego at lists.linuxfoundation.org
> Subject: RE: [Fuego] [PATCH 4/4] ftc: add remove-logs flag to rm-jobs
>
> See comment below.
>
> > -----Original Message-----
> > From: fuego-bounces at lists.linuxfoundation.org [mailto:fuego-
> > bounces at lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > Sent: Thursday, August 03, 2017 7:17 PM
> > To: fuego at lists.linuxfoundation.org
> > Subject: [Fuego] [PATCH 4/4] ftc: add remove-logs flag to rm-jobs
> >
> > When removing jobs from jenkins this flag additionally allows
> > deleting the corresponding log files.
> >
> > For developers, ftc rm-jobs --remove-logs, will leave Fuego
> > in a pristine state without jobs nor old logs.
> >
> > Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> > ---
> > engine/scripts/ftc | 45 +++++++++++++++++++++++++++++++++++++++---
> > ---
> > 1 file changed, 39 insertions(+), 6 deletions(-)
> >
> > diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> > index e885c5f..d6cede5 100755
> > --- a/engine/scripts/ftc
> > +++ b/engine/scripts/ftc
> > @@ -44,7 +44,7 @@
> > import os, sys, string, re
> > import time
> > import commands
> > -from shutil import copy2
> > +from shutil import copy2, rmtree
> > from lxml import etree
> > import io
> > import subprocess
> > @@ -57,6 +57,7 @@ import shutil
> > import tempfile
> > import yaml
> > import jenkins
> > +import glob
> >
> > # MAJOR, MINOR, REVISION
> > VERSION = (1,2,0)
> > @@ -94,15 +95,17 @@ command_help = {
> > This interface may change in the future."""),
> >
> > "rm-jobs":("Removes jobs from Jenkins.",
> > - """Usage: ftc rm-jobs <target>.<testspec>.<testtype>.<testcase>
> > + """Usage: ftc rm-jobs [--remove-logs]
> > <target>.<testspec>.<testtype>.<testcase>
> > Use list-jobs to see the existing jobs. A wildcard can be used to
> > specify which jobs to remove (just make sure you have 4 words):
> > Example: ftc rm-jobs "docker.testplan_docker.*.*"
> > Example: ftc rm-jobs "docker.*.F*.*stress"
> > Multiple combinations of the <target>.<testspec>.<testtype>.<testcase>
> > - pattern can be passed as well.
> > -
> > - If no option is provided all existing jobs will be removed."""),
> > + pattern can be passed as well. The option --remove-logs will additionally
> > + remove the corresponding log files in the log directory. If no job is
> > + provided all existing jobs will be removed.
> > + Example: ftc rm-jobs --remove-logs
> > + """),
> >
> > "add-nodes":("Adds new nodes to Jenkins.",
> > """Usage: ftc add-nodes [-f] <board1> <board2> ...
> > @@ -1077,8 +1080,25 @@ def do_build_jobs(conf, options):
> >
> > def do_rm_jobs(conf, options):
> > count = 0
> > +
> > + if '--remove-logs' in options:
> > + remove_logs = True
> > + options.remove('--remove-logs')
> > + raw_input("WARNING: this will remove logs. Press ENTER to
> > confirm...")
>
> I don't think this is needed. If you really want it, I can leave it,
> but I think the user adding '--remove-logs' is specific enough to
> not warrant asking for confirmation. Let me know. If you're
> OK with me taking it out, I'll do so when I add the patch to
> my next branch. If you want to keep it, though, just let
> me know.
Maybe I was being too conservative. Please remove the line if you want.
> > + else:
> > + remove_logs = False
> > +
> > if len(options) == 0:
> > - # FIXTHIS: warn user that all jobs will be removed
> > + if remove_logs:
> > + # remove files and folders in LOGDIR
> > + path = conf.FUEGO_RW + "/logs/*"
>
> I can't tell, but I assume this is in the 'remove all jobs' path?
Yes, that's right.
>
> > + files = glob.glob(path)
> > + for f in files:
> > + if os.path.isfile(f):
> > + os.remove(f)
> > + else:
> > + shutil.rmtree(f, ignore_errors=True)
> > + # remove jenkins jobs
> > jobs = [job['name'] for job in server.get_jobs()]
> > for job in jobs:
> > server.delete_job(job)
> > @@ -1097,6 +1117,19 @@ def do_rm_jobs(conf, options):
> > else:
> > for match in matches:
> > server.delete_job(match)
> > + if remove_logs:
> > + # remove the per-build_number log folders for the job
> > + path = conf.FUEGO_RW + '/logs/' + '.'.join(match.split('.')[2:]) +
> > '/' + '.'.join(match.split('.')[:2]) + '*'
> > + folders = glob.glob(path)
> > + for folder in folders:
> > + print 'Removing folder ' + folder
> > + shutil.rmtree(folder, ignore_errors=True)
> > + # remove the log folder itself in case it became empty
> > + folders = glob.glob(path)
> > + if not folders:
> > + path = conf.FUEGO_RW + '/logs/' + '.'.join(match.split('.')[2:])
> > + print 'Removing folder ' + path
> > + shutil.rmtree(path, ignore_errors=True)
> > count = count + 1
> > if not quiet:
> > print "Deleted %d jobs." % count
> > --
> > 2.7.4
>
> Ack.
> -- Tim
>
Thanks for the quick review of the patches.
Daniel
More information about the Fuego
mailing list