adding the cancel method for Hive, HadoopJava, Pig

[johnyu0520]

so that it kills the hadoop job that it launches

[logiclord]

Should we make this change for reportalpig and reportalhive as well ?

[johnyu0520]

Reportal's pig and hive seem to use a completely different code base.
We can do it, but I don't think should be in this pull request.

In addition, I think pig and hive are mostly cleaning up after themselves already, so this PR's most material impact would be on hadoopJava processes

[logiclord]

make sense. We can open a github bug to track it and handle reportal changes separately.

[johnyu0520]

incorporated feedback. Please check again.

Thanks!
John

[evlstyle]

nit - can be removed.

[johnyu0520]

um...not having this line actually causes a compile error, hence I had to add it in in the first place.

[logiclord]

LGTM

[davidzchen]

LGTM. Can you rebase squash the commits before merging? Thanks!

[davidzchen]

@johnyu0520 - One thing I might have failed to mention in my rebase instructions in azkaban/azkaban#517 is that in order to squash your commits, once git rebase -i (the -i flag stands for "interactive") opens your editor, you need to mark all of the commits except for the first one as fixup rather than pick. This way, git rebase will squash each commit marked as fixup with the previous commit, and as a result, you will end up with a single commit.

One other thing you can do is you can edit the commit message for your squashed commit. To do that, just use reword rather than pick for the first commit but still mark all following commits as fixup. Then, once the rebase is done, git will open your editor one more time to allow you to edit your commit message.

[johnyu0520]

@davidzchen
Thanks for the note.
I basically spent the whole of Friday doing rebasing and cloning and git diff apply :)

My observation is that rebasing does not work well for

1. a series of commits with accidental formatting changes in the middle
2. that has someone else's commits in the middle
   That was the issue with my azkaban's multiexecutor_canary pull request. I got out of it with a git diff apply (another one of your tips)

For this patch, I was fortunate to get away with git rebase -i (which I do use from time to time over the yeras). In the normal case where commits don't drag out for that long, this process is indeed pretty light weight

[davidzchen]

Thanks, @johnyu0520! I really appreciate you taking the time to for doing the rebase. Sorry for the hassle.

The reason why I am suggesting that we rebase and squash commits for pull requests is that this model is closer to the development model followed by Apache projects (i.e. one single patch per issue), and I think it would be great to eventually get Azkaban into Apache. :)

Rebasing would definitely result in a lot of merge conflicts if your branch has greatly diverged from master. I found that it becomes much easier to do development in a feature branch (as discussed in azkaban/azkaban#517) and rebase and squash commits as often as possible, such as whenever new changes land in master. That way, there would not suddenly be a ton of merge conflicts when rebasing rebase because the feature branch would not have diverged very far from master.