[1/2] task.py: allow expected failures

From: Kim Højgaard-Hansen <kiho@prevas.dk>

Make it possible to have test cases which are expected to fail
as part of a bake build e.g. oe bake world. When a task is marked
as expected to fail, the bake will continue when the task actually
fails, and abort if the task should succeed.
---
 lib/oelite/task.py | 29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

<kim.hansen@prevas.dk> writes:

> From: Kim Højgaard-Hansen <kiho@prevas.dk>
>
> Make it possible to have test cases which are expected to fail
> as part of a bake build e.g. oe bake world. When a task is marked
> as expected to fail, the bake will continue when the task actually
> fails, and abort if the task should succeed.
> ---
>  lib/oelite/task.py | 29 ++++++++++++++++++++++-------
>  1 file changed, 22 insertions(+), 7 deletions(-)
>
> diff --git a/lib/oelite/task.py b/lib/oelite/task.py
> index 2dd08c8..49bcd20 100644
> --- a/lib/oelite/task.py
> +++ b/lib/oelite/task.py
> @@ -223,23 +223,38 @@ class OEliteTask:
>  
>          try:
>              for prefunc in self.get_prefuncs():
> -                print "running prefunc", prefunc
> +                debug("running prefunc: %", prefunc)

Please do not convert print statement to debug() calls.  I hope to
introduce a nicer way of controlling debug/info/warning/error messages,
which will most likely be based on the logging module API.  But I very
much plan on keeping support for all print statements being treated as
debug messages, as I believe it is a very nice KISS approach :)

>                  self.do_cleandirs(prefunc)
>                  wd = self.do_dirs(prefunc)
>                  if not prefunc.run(wd or cwd):
> -                    return False
> +                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                        return True

Returning True here will terminate the task and not continue.  But do we
really want to apply TASK_EXPECTED_FAILURE to prefuncs and postfuncs?  I
am not sure how to do that in a sane way.

> +                    else:
> +                        return False
>              try:
>                  if not function.run(cwd):
> -                    return False
> +                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                        return True

Again, this will abort without running the postfuncs. Is that intended?

> +                    else:
> +                        return False
>              except oebakery.FatalError:
> -                return False
> +                if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                    return True
> +                else:
> +                    return False

We might want to still see FatalError as errors when
TASK_EXPECTED_FAILURE is set.

>              for postfunc in self.get_postfuncs():
> -                print "running postfunc", postfunc
> +                debug("running postfunc %s",postfunc)
>                  self.do_cleandirs(postfunc)
>                  wd = self.do_dirs(postfunc)
>                  if not postfunc.run(wd or cwd):
> -                    return False
> -            return True
> +                    if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                        return True
> +                    else:
> +                        return False

We really need to consider how to handle prefuncs/postfuncs.  Let's say
you have a task with two prefuncs and two postfuncs, and have set
TASK_EXPECTED_FAILURE.

Do we expect all 5 functions two fail?  Do we expect all prefuncs and
postfuncs to succeed?  Do we allow prefuncs to both fail and succeed?

> +            if(meta.get("TASK_EXPECTED_FAILURE") == self.name):
> +                return False
> +            else:
> +                return True
>  
>          finally:
>              # Cleanup stdin, stdout and stderr redirection

/Esben