26

I have multiple expensive functions that return results. I want to return a tuple of the results of all the checks if all the checks succeed. However, if one check fails I don't want to call the later checks, like the short-circuiting behavior of and. I could nest if statements, but that will get out of hand if there are a lot of checks. How can I get the short-circuit behavior of and while also storing the results for later use?

def check_a():
    # do something and return the result,
    # for simplicity, just make it "A"
    return "A"

def check_b():
    # do something and return the result,
    # for simplicity, just make it "B"
    return "B"

...

This doesn't short-circuit:

a = check_a()
b = check_b()
c = check_c()

if a and b and c:
    return a, b, c

This is messy if there are many checks:

if a:
   b = check_b()

   if b:
      c = check_c()

      if c:
          return a, b, c

Is there a shorter way to do this?

5
  • If DanielRoseman's answer isn't what you want, there's no shorter way of doing it. It's possible, but involves a lazy decorator with __call__ and __nonzero__ methods, something much more complicated than the simple if-else chain. Commented Sep 20, 2016 at 20:48
  • What should be returned if a previous check fails? Commented Sep 20, 2016 at 20:49
  • It returns None if a previous check fails Commented Sep 20, 2016 at 21:15
  • If one of your checks fails, you probably want to know which one it was. In the solutions you present, it is not clear how this information is exported (except from some of the variables a, b, c remaining undefined). Commented Sep 21, 2016 at 12:27
  • Not sure if Python supports this syntax but here is how PHP would manhandle it stackoverflow.com/q/39620629/2191572 Commented Sep 21, 2016 at 15:52

14 Answers 14

27

Just use a plain old for loop:

results = {}
for function in [check_a, check_b, ...]:
    results[function.__name__] = result = function()
    if not result:
        break

The results will be a mapping of the function name to their return values, and you can do what you want with the values after the loop breaks.

Use an else clause on the for loop if you want special handling for the case where all of the functions have returned truthy results.

Sign up to request clarification or add additional context in comments.

8 Comments

Is there a reason to prefer indexing results using the function name rather than the function itself?
Maybe so results can be accessed with a string literal, without necessarily having to obtain a reference on the function object? No, I can't really think of any convincing reason.
@WaleedKhan I guess hashing a function object takes significantly more time than hashing just its name? In any case: for this simple case given that the user already knows the order of the operations you could just use a list: results = [] and access the results by indexing.
@Bakuriu Why should hashing a function take much time? Most likely the hash is derived from the id() of the object which is in CPython the address of the object in memory. So it comes down to hashing an integer value.
@BlackJack And in Python hash(x) == x when x is an integer != -1
|
8

Write a function that takes an iterable of functions to run. Call each one and append the result to a list, or return None if the result is False. Either the function will stop calling further checks after one fails, or it will return the results of all the checks.

def all_or_none(checks, *args, **kwargs):
    out = []

    for check in checks:
        rv = check(*args, **kwargs)

        if not rv:
            return None

        out.append(rv)

    return out
rv = all_or_none((check_a, check_b, check_c))

# rv is a list if all checks passed, otherwise None
if rv is not None:
    return rv
def check_a(obj):
    ...

def check_b(obj):
    ...

# pass arguments to each check, useful for writing reusable checks
rv = all_or_none((check_a, check_b), obj=my_object)

Comments

6

In other languages that did have assignments as expressions you would be able to use

if (a = check_a()) and (b = check_b()) and (c = check_c()):

but Python is no such language. Still, we can circumvent the restriction and emulate that behaviour:

result = []
def put(value):
    result.append(value)
    return value

if put(check_a()) and put(check_b()) and put(check_c()):
    # if you need them as variables, you could do
    # (a, b, c) = result
    # but you just want
    return tuple(result)

This might loosen the connection between the variables and function calls a bit too much, so if you want to do lots of separate things with the variables, instead of using the result elements in the order they were put in the list, I would rather avoid this approach. Still, it might be quicker and shorter than some loop.

Comments

3

You could use either a list or an OrderedDict, using a for loop would serve the purpose of emulating short circuiting.

from collections import OrderedDict


def check_a():
    return "A"


def check_b():
    return "B"


def check_c():
    return "C"


def check_d():
    return False


def method1(*args):
    results = []
    for i, f in enumerate(args):
        value = f()
        results.append(value)
        if not value:
            return None

    return results


def method2(*args):
    results = OrderedDict()

    for f in args:
        results[f.__name__] = result = f()
        if not result:
            return None

    return results

# Case 1, it should return check_a, check_b, check_c
for m in [method1, method2]:
    print(m(check_a, check_b, check_c))

# Case 1, it should return None
for m in [method1, method2]:
    print(m(check_a, check_b, check_d, check_c))

2 Comments

In both functions, the break can just be return
@Barry Indeed, that last comparison using len wasn't clever at all. Thanks!
2

There are lots of ways to do this! Here's another.

You can use a generator expression to defer the execution of the functions. Then you can use itertools.takewhile to implement the short-circuiting logic by consuming items from the generator until one of them is false.

from itertools import takewhile
functions = (check_a, check_b, check_c)
generator = (f() for f in functions)
results = tuple(takewhile(bool, generator))
if len(results) == len(functions):
    return results

1 Comment

Image
bool should perform better than lambda x: x
2

Another way to tackle this is using a generator, since generators use lazy evaluation. First put all checks into a generator:

def checks():
    yield check_a()
    yield check_b()
    yield check_c()

Now you could force evaluation of everything by converting it to a list:

list(checks())

But the standard all function does proper short cut evaluation on the iterator returned from checks(), and returns whether all elements are truthy:

all(checks())

Last, if you want the results of succeeding checks up to the failure you can use itertools.takewhile to take the first run of truthy values only. Since the result of takewhile is lazy itself you'll need to convert it to a list to see the result in a REPL:

from itertools import takewhile
takewhile(lambda x: x, checks())
list(takewhile(lambda x: x, checks()))

Comments

1

main logic:

results = list(takewhile(lambda x: x, map(lambda x: x(), function_list)))
if len(results) == len(function_list):
  return results

you can learn a lot about collection transformations if you look at all methods of an api like http://www.scala-lang.org/api/2.11.7/#scala.collection.immutable.List and search/implement python equivalents

logic with setup and alternatives:

import sys
if sys.version_info.major == 2:
  from collections import imap
  map = imap

def test(bool):
  def inner():
    print(bool)
    return bool
  return inner

def function_for_return():
  function_list = [test(True),test(True),test(False),test(True)]

  from itertools import takewhile

  print("results:")

  results = list(takewhile(lambda x:x,map(lambda x:x(),function_list)))
  if len(results) == len(function_list):
    return results

  print(results)
  #personally i prefer another syntax:
  class Iterator(object):
    def __init__(self,iterable):
      self.iterator = iter(iterable)

    def __next__(self):
      return next(self.iterator)

    def __iter__(self):
      return self

    def map(self,f):
      return Iterator(map(f,self.iterator))

    def takewhile(self,f):
      return Iterator(takewhile(f,self.iterator))

  print("results2:")
  results2 = list(
    Iterator(function_list)
      .map(lambda x:x())
      .takewhile(lambda x:x)    
  )

  print(results2)

  print("with additional information")
  function_list2 = [(test(True),"a"),(test(True),"b"),(test(False),"c"),(test(True),"d")]
  results3 = list(
    Iterator(function_list2)
      .map(lambda x:(x[0](),x[1]))
      .takewhile(lambda x:x[0])    
  )
  print(results3)

function_for_return()

2 Comments

It would be nice if you could explain what this does – particularly the large block of code. You should also make it clearer that your first snippet will only work in Python 3 (unless using imap).
Image
In your itertool recipe, bool should be a better alternative to lambda x: x. You may also want to write (f() for f in function_list), which will short-circuit in both Python2 and Python3, instead of the map.
1

If you don't need to take an arbitrary number of expressions at runtime (possibly wrapped in lambdas), you can expand your code directly into this pattern:

def f ():
    try:
        return (<a> or jump(),
                <b> or jump(),
                <c> or jump())
    except NonLocalExit:
        return None

Where those definitions apply:

class NonLocalExit(Exception):
    pass

def jump():
    raise NonLocalExit()

Comments

1

Flexible short circuiting is really best done with Exceptions. For a very simple prototype you could even just assert each check result:

try:
    a = check_a()
    assert a
    b = check_b()
    assert b
    c = check_c()
    assert c
    return  a, b, c
except AssertionException as e:
    return None

You should probably raise a custom Exception instead. You could change your check_X functions to raise Exceptions themself, in an arbitrary nested way. Or you could wrap or decorate your check_X functions to raise errors on falsy return values.

In short, exception handling is very flexible and exactly what you are looking for, don't be afraid to use it. If you learned somewhere that exception handling is not to be used for your own flow control, this does not apply to python. Liberal use of exception handling is considered pythonic, as in EAFP.

3 Comments

Python is not an exception (ahem) to "don't use exceptions for control flow"
@AlexFoxGill Yes but another language another battle.
@alexforgill yes but this is exactly what exceptions are for
1

You mentioned 'short-circuiting' in your answer, which can be done with the 'or' statement. Top answer basically does the same thing, but in case someone wants to know more about this behaviour you could do this;

class Container(object):
    def __init__(self):
        self.values = []

    def check_and_cache(self, value, checking_function):
        value_true = checking_function(value)
        if value_true:
            self.values.append(value)
            return True

c = Container()
if not c.check_and_cache(a, check_a) or not c.check_and_cache(b, check_b) or not c.check_and_cache(c, check_c):
    print 'done'
return tuple(c.values)

The 'not .. or' setup of the if statements will result in a 'True' if the check fails, so the overall if statement passes without evaluating the remaining values.

Comments

0

Since I can not comment "wim":s answer as guest, I'll just add an extra answer. Since you want a tuple, you should collect the results in a list and then cast to tuple.

def short_eval(*checks):
    result = []
    for check in checks:
        checked = check()
        if not checked:
            break
        result.append(checked)
    return tuple(result)

# Example
wished = short_eval(check_a, check_b, check_c)

Comments

0

You can try use @lazy_function decorator from lazy_python package. Example of usage:

from lazy import lazy_function, strict

@lazy_function
def check(a, b):
    strict(print('Call: {} {}'.format(a, b)))
    if a + b > a * b:
        return '{}, {}'.format(a, b)

a = check(-1, -2)
b = check(1, 2)
c = check(-1, 2)

print('First condition')
if c and a and b: print('Ok: {}'.format((a, b)))

print('Second condition')
if c and b: print('Ok: {}'.format((c, b)))
# Output:
# First condition
# Call: -1 2
# Call: -1 -2
# Second condition
# Call: 1 2
# Ok: ('-1, 2', '1, 2')

Comments

0

This is similar to Bergi's answer but I think that answer misses the point of wanting separate functions (check_a, check_b, check_c):

list1 = []

def check_a():
    condition = True
    a = 1
    if (condition):
        list1.append(a)
        print ("checking a")
        return True
    else:
        return False

def check_b():
    condition = False
    b = 2
    if (condition):
        list1.append(b)
        print ("checking b")
        return True
    else:
        return False

def check_c():
    condition = True
    c = 3
    if (condition):
        list1.append(c)
        print ("checking c")
        return True
    else:
        return False


if check_a() and check_b() and check_c():
    # won't get here

tuple1 = tuple(list1)    
print (tuple1)    

# output is:
# checking a
# (1,)

Or, if you don't want to use the global list, pass a reference of a local list to each of the functions.

1 Comment

I didn't modify the called functions on purpose. I thought of something like def check_and_put_a(): nonlocal a = check_a(); return a however which could be called instead of put(check_a()) and would assign to a, but that's lots of repetition.
0

If the main objection is

This is messy if there are many checks:

if a:
   b = check_b()

   if b:
      c = check_c()

      if c:
          return a, b, c

A fairly nice pattern is to reverse the condition and return early

if not a:
    return  # None, or some value, or however you want to handle this
b = check_b()
if not b:
    return
c = check_c()
if not c:
    return

# ok, they were all truthy
return a, b, c

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.