[Fuego] [PATCH 4/4] ftc: add remove-logs flag to rm-jobs

Bird, Timothy Tim.Bird at sony.com
Fri Aug 4 03:14:07 UTC 2017


See comment below.

> -----Original Message-----
> From: fuego-bounces at lists.linuxfoundation.org [mailto:fuego-
> bounces at lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Thursday, August 03, 2017 7:17 PM
> To: fuego at lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 4/4] ftc: add remove-logs flag to rm-jobs
> 
> When removing jobs from jenkins this flag additionally allows
> deleting the corresponding log files.
> 
> For developers, ftc rm-jobs --remove-logs, will leave Fuego
> in a pristine state without jobs nor old logs.
> 
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin at toshiba.co.jp>
> ---
>  engine/scripts/ftc | 45 +++++++++++++++++++++++++++++++++++++++---
> ---
>  1 file changed, 39 insertions(+), 6 deletions(-)
> 
> diff --git a/engine/scripts/ftc b/engine/scripts/ftc
> index e885c5f..d6cede5 100755
> --- a/engine/scripts/ftc
> +++ b/engine/scripts/ftc
> @@ -44,7 +44,7 @@
>  import os, sys, string, re
>  import time
>  import commands
> -from shutil import copy2
> +from shutil import copy2, rmtree
>  from lxml import etree
>  import io
>  import subprocess
> @@ -57,6 +57,7 @@ import shutil
>  import tempfile
>  import yaml
>  import jenkins
> +import glob
> 
>  # MAJOR, MINOR, REVISION
>  VERSION = (1,2,0)
> @@ -94,15 +95,17 @@ command_help = {
>    This interface may change in the future."""),
> 
>  "rm-jobs":("Removes jobs from Jenkins.",
> -    """Usage: ftc rm-jobs <target>.<testspec>.<testtype>.<testcase>
> +    """Usage: ftc rm-jobs [--remove-logs]
> <target>.<testspec>.<testtype>.<testcase>
>    Use list-jobs to see the existing jobs. A wildcard can be used to
>    specify which jobs to remove (just make sure you have 4 words):
>      Example: ftc rm-jobs "docker.testplan_docker.*.*"
>      Example: ftc rm-jobs "docker.*.F*.*stress"
>    Multiple combinations of the <target>.<testspec>.<testtype>.<testcase>
> -  pattern can be passed as well.
> -
> -  If no option is provided all existing jobs will be removed."""),
> +  pattern can be passed as well. The option --remove-logs will additionally
> +  remove the corresponding log files in the log directory. If no job is
> +  provided all existing jobs will be removed.
> +    Example: ftc rm-jobs --remove-logs
> +  """),
> 
>  "add-nodes":("Adds new nodes to Jenkins.",
>      """Usage: ftc add-nodes [-f] <board1> <board2> ...
> @@ -1077,8 +1080,25 @@ def do_build_jobs(conf, options):
> 
>  def do_rm_jobs(conf, options):
>      count = 0
> +
> +    if '--remove-logs' in options:
> +        remove_logs = True
> +        options.remove('--remove-logs')
> +        raw_input("WARNING: this will remove logs. Press ENTER to
> confirm...")

I don't think this is needed.  If you really want it, I can leave it,
but I think the user adding '--remove-logs' is specific enough to
not warrant asking for confirmation.  Let me know.  If you're
OK with me taking it out, I'll do so when I add the patch to
my next branch.  If you want to keep it, though, just let
me know.

> +    else:
> +        remove_logs = False
> +
>      if len(options) == 0:
> -        # FIXTHIS: warn user that all jobs will be removed
> +        if remove_logs:
> +            # remove files and folders in LOGDIR
> +            path = conf.FUEGO_RW + "/logs/*"

I can't tell, but I assume this is in the 'remove all jobs' path?

> +            files = glob.glob(path)
> +            for f in files:
> +                if os.path.isfile(f):
> +                    os.remove(f)
> +                else:
> +                    shutil.rmtree(f, ignore_errors=True)
> +        # remove jenkins jobs
>          jobs = [job['name'] for job in server.get_jobs()]
>          for job in jobs:
>              server.delete_job(job)
> @@ -1097,6 +1117,19 @@ def do_rm_jobs(conf, options):
>              else:
>                  for match in matches:
>                      server.delete_job(match)
> +                    if remove_logs:
> +                        # remove the per-build_number log folders for the job
> +                        path = conf.FUEGO_RW + '/logs/' + '.'.join(match.split('.')[2:]) +
> '/' + '.'.join(match.split('.')[:2]) + '*'
> +                        folders = glob.glob(path)
> +                        for folder in folders:
> +                            print 'Removing folder ' + folder
> +                            shutil.rmtree(folder, ignore_errors=True)
> +                        # remove the log folder itself in case it became empty
> +                        folders = glob.glob(path)
> +                        if not folders:
> +                            path = conf.FUEGO_RW + '/logs/' + '.'.join(match.split('.')[2:])
> +                            print 'Removing folder ' + path
> +                            shutil.rmtree(path, ignore_errors=True)
>                      count = count + 1
>      if not quiet:
>          print "Deleted %d jobs." % count
> --
> 2.7.4

Ack.
 -- Tim



More information about the Fuego mailing list