[nonGallery] return backfill to order by date, batch size 1000 #8510
Conversation
drewgillies
requested review from
agr,
dannyjdev,
joelverhagen,
loic-sharma,
lyndaidaii,
ryuyu,
shishirx34,
skofman1,
xavierdecoster and
zhhyu
as code owners
| packages = packages | ||
| .Where(p => p.Created < lastCreateTime && p.Created > startTime) | ||
| .OrderBy(p => p.PackageRegistration.Id); | ||
| .Where(p => p.PackageStatusKey == PackageStatus.Available || p.PackageStatusKey == PackageStatus.Validating) |
There was a problem hiding this comment.
Validating packages won't be available in v3-flatcontainer. Could we stick to PackageStatus.Available?
There was a problem hiding this comment.
@joelverhagen Great insight. Thank you. Note that this is for all backfill jobs--this is the parent class.
@joelverhagen note that this is removed code that I reintroduced. It could be an implementation detail (i.e. a flag) set in the subclasses if we like, defaulting to false. |
drewgillies
force-pushed
the
dg-improvebackfill
branch
from
a96e185
to
6ef7d6c
Compare
| { | ||
| supportedFrameworks = _packageService.GetSupportedFrameworks(nuspecReader, files); | ||
| } | ||
| catch (Exception e) |
There was a problem hiding this comment.
We can catch a more specific exception here, right? I think it's a specific one that is caused by the bad portables.
There was a problem hiding this comment.
| catch (Exception e) | |
| catch |
if we end up using a generic catch-all like the other places
There was a problem hiding this comment.
It's a little generic--it's an ArgumentException which is why I opted for catching all. But I'm happy to make it explicit as exceptions not of this type will fall through and be logged, which could be useful.
drewgillies
force-pushed
the
dg-improvebackfill
branch
from
6ef7d6c
to
6bd2309
Compare
drewgillies
changed the title
Backfill job was ordered in such a way as to not work with cursoring. There was a filtering condition removed which also needed to be reinstated. I also upped the batching size to 1000 for the TFM backfill.