Async branch doesn't return all results because of flaw in async_in_query_fetch()

[GoogleCodeExporter]

What steps will reproduce the problem?
Perform a proximity_fetch() which should return 2 results.


What is the expected output? What do you see instead?
Except to get 2 results.  Instead, I only get 1.

What version of the product are you using? On what operating system?
The async branch, GAE 1.3.3.

Please provide any additional information below.

The issue is in async_in_query_fetch() in util.py, specifically the _numkeep 
lambda:
_numkeep = lambda arr: int(math.ceil(len(arr) * 1.0 / total_results))

This returns 1 when passed in the list [[result1, result2], []], should return 
2.

My quick and dirty work around is to skip the whole map/reduce step if 
total_results <= 
max_results:

    if total_results <= max_results:
      return [ entity for entities in entities_by_value for entity in entities ]
    else:
      #this _numkeep is broken:  returns 1 for [['a','b'], []]
      _numkeep = lambda arr: int(math.ceil(len(arr) * 1.0 / total_results))
      entities_by_value = [val_entities[:_numkeep(val_entities)]
                           for val_entities in entities_by_value]
      return reduce(lambda x, y: x + y, entities_by_value)

Original issue reported on code.google.com by [email protected] on 24 Apr 2010 at 3:42

[GoogleCodeExporter]

I did this:

    # Return max_results entities, proportionally by geocell, since there
    # is no ordering.
    results = []
    iters = [iter(val_entities) for val_entities in entities_by_value]
    iter_index = 0
    while len(results) < max_results and len(iters) > 0:
        iter_index = iter_index%len(iters)
        current_iter = iters[iter_index]
        try:
            results.append(current_iter.next())
            iter_index = iter_index + 1
        except StopIteration:
            del iters[iter_index]
    return results

Original comment by [email protected] on 24 Aug 2010 at 1:40