add/delete only changed overlap records

[jsteverman]

• moves ETAS database ops to Module ETAS

[aelkiss]

I think the split into OverlapTableUpdate and ClusterUpdate makes sense. I think we should try to come up with a better name for the module they're in (admittedly not my forte).

I'll separately comment with thoughts on a test to add to make sure we're doing what we think we are regarding minimizing churn.

[aelkiss]

Regarding testing that minimizing churn is working as expected:

The specs are a little bit hard to follow but I think are basically already doing this by checking that the delete count is smaller than the old overlap count & likewise that the add count is smaller than the new overlap count.

You could consider adding a more explicit test like:

describe "#deletes" do
  it "does not include things in both the old & new overlaps" do
    do_some_setup
    expect(deletes).not_to contain(thing_that_shouldnt_be_deleted)
  end
end

describe "#adds" do
  it "does not include things in both the old & new overlaps" do
    do_some_setup
    expect(adds).not_to contain(thing_that_shouldnt_need_to_be_re_added)
  end
end

Regarding testing that it's actually reducing churn in mysql, probably the way to do that would be to stub the overlap table and then check that it receives the expected number of insert & delete calls:

it "doesn't churn" do
  mock_overlap_table = stub(:overlap_table)
  allow(mock_overlap_table).to receive(:filter).and_return(mock_overlap_table)
  expect(mock_overlap_table).to receive(:delete).exactly(M).times
  expect(mock_overlap_table).to receive(:insert).exactly(N).times  

  do_some_setup
  described_class.new(mock_overlap_table, cluster).upsert
end

You could I suppose set an expectation that filter or insert receives the parameters you expect it to; that's probably overkill since we're already checking the actual contents of the table. Doing it this way just allows us to make sure that we aren't calling insert or delete more than we expected.

[aelkiss]

Changes look good to me. Should be good to go once you resolve the merge conflict & squash.