Review usage of Heroku

[nobodxbodon]

Reticket from #928 (comment)

IMO it's worthwhile to review the usage of Heroku, including what instances we use for each of our services.

This exercise may also give more insights about the infrastructure, which would be valuable in sharing the knowledge among the contributors.

@rohitpaulk

Heroku is the one item I definitely wouldn't move - $95/month is definitely worth it. We get database backups, monitoring, easy rollbacks + a reliable service overall. Trying to replicate that on another provider (like AWS) is likely going to cost us as much or more (including man hours).

Here's breakdowns based on the invoices:

2016-11

Application	Cost Type	Details	Amount
gip-rocks	dyno-hours	web	$7.00
gratipay	dyno-hours	scheduler	$2.45
gratipay	dyno-hours	web	$7.00
gratipay	dyno-hours	run	$0.01
gratipay	addon	scheduler	$0
gratipay	addon	postgresql:standard	$50.00
gratipay	addon	ssl:endpoint	$20.00
gratipay	addon	papertrail:fixa	$7.00
gratipay	addon	deployhooks:http	$0.00
gratipay-slackin	dyno-hours	web	$1.73
inside-gratipay-com	dyno-hours	web	$7.00
inside-gratipay-com	addon	deployhooks:http	$0.00

2016-12 only difference is:

Application	Cost Type	Details	Amount
- gratipay	dyno-hours	scheduler	$2.45
+ gratipay-slackin	dyno-hours	web	$7.00

[chadwhitacre]

I'm -1 on this. Heroku saves us a ton of devops time.

[nobodxbodon]

After creating the issue I realized there was some history about using Heroku. Thus I'm removing the part about looking for alternatives here, and focus on the reviewing usage part. IMO it'll still be valuable to share knowledge about the deployment and server configs. @whit537 if you agree, could you share some document about the setup (what kind of server instance is used for gratipay, etc)?

[chadwhitacre]

@nobodxbodon Here are the past two invoices from Heroku:

heroku.2016-11.pdf
heroku.2016-12.pdf

Does that give you a sufficient level of visibility into our usage in order to review?

[nobodxbodon]

Thanks for the material! Some questions after checking breakdowns:

• is papertrail:fixa the same function as the papertail item in review fixed cost #928 (comment)? Double paying?
• if gip-rocks is for https://github.com/gratipay/gip.rocks/, and if it doesn't bring traffic to gratipay, is it OK to run it as free-tier dyno?
• is gratipay-slackin serving our slack channel in some way?

[chadwhitacre]

is papertrail:fixa the same function as the papertail item in #928 (comment)? Double paying?

Yes. No.

if gip-rocks is for https://github.com/gratipay/gip.rocks/, and if it doesn't bring traffic to gratipay, is it OK to run it as free-tier dyno?

http://gip.rocks/ is behind Gratipay.com providing a service (processing images), not in front of Gratipay.com driving traffic. There is no free tier for Team accounts (cf. #871).

is gratipay-slackin serving our slack channel in some way?

Yes. It provides the auto-invite page at http://inside.gratipay.com/appendices/chat.

[nobodxbodon]

is papertrail:fixa the same function as the papertail item in #928 (comment)? Double paying?

Yes. No.

So shall we remove the Papertail item in budget table, as it's included in Heroku?

Also, adding $7 to Heroku for gratipay-slackin, resulting in $109 (from $95).

[chadwhitacre]

So shall we remove the Papertail item in budget table, as it's included in Heroku?

The canonical budget table is in the spreadsheet (which is permanent until gratipay/finances#12), not in the description on #928 (which we'll hopefully be able to close soon). I think it makes sense to keep Papertrail and Heroku itemized separately for visibility.

Also, adding $7 to Heroku for gratipay-slackin, resulting in $109 (from $95).

Thanks. Please update the spreadsheet as well.

[nobodxbodon]

OK I added in the speadsheet $7 to Heroku for gratipay-slackin, and left Papertail as is. Closing.

[chadwhitacre]

https://amazonlightsail.com/ ?

[rohitpaulk]

That looks like a VPS replacement, no different from digital ocean and linode

[Changaco]

Chad's comment got me digging into AWS. Apparently it's still less developer-friendly than Heroku, but https://aws.amazon.com/elasticbeanstalk/ and https://aws.amazon.com/rds/ seem to make it reasonably easy and cheap to run a Python web app with a PostgreSQL database.

[chadwhitacre]

Except that we'd also have access to Amazon services for Postgres, CDN, TLS, and logging (we're already using them for email, of course). I like the idea of infrastructure as code. I'd be interested to see a cost comparison with Heroku.

[chadwhitacre]

Heroku Dynos are $7, vs. $5/mo at Amazon (assuming they're roughly equal capacity).

Postgres is $50/mo at Heroku. Poking at AWS, it ... is hard to get a clear picture. Minimum pricing on a single-zone reserved instance is roughly $10/mo. But what about multiple availability zones? Backups? Bandwidth? Would we be closer to $10/mo or $25 or $50? I don't know.

For TLS/CDN ...

SNI Custom SSL is available at no additional cost beyond standard CloudFront data transfer and request fees.

https://aws.amazon.com/cloudfront/faqs/

That's probably going to be cheaper than the $9.95/mo we pay for CDN + the $20/mo we pay for SSL? Maybe a lot cheaper since we don't have much traffic right now?

Let's say the best case scenario is:

$10 = $5 x 2 EC2 instances
$15 for Postgres
$2 for logging
$5 for CloudFront+SNI
= $32/mo

It looks like in the best case, Amazon would clock in at about a third the cost of Heroku. Let's say it's more realistic to expect 50%.

Now, unlike with our other cost savings on #928, this one comes with a significant time burden. How much developer time would it take to migrate? What would it take to maintain AWS vs. Heroku in the future? Is the cost savings worth the time investment?

[chadwhitacre]

We might also try to factor AWS free tier into our calculation. I don't know where we stand with that or how it works.

[chadwhitacre]

@clone1018 in slack:

Yeah. I use AWS in the hipaa space and I love it
Its not cheap though, nor is it easy

[nobodxbodon]

Actually I read Heroku PostgreSQL vs. Amazon RDS for PostgreSQL before closing this, and I agreed with the summary below:

If I had to choose, I’d probably go with Heroku and Heroku PostgreSQL as a startup while I focused on actually getting my application developed and getting customers in the door. 
...
In the end, it really boils down to what costs you more: time or infrastructure.

IMO we currently have some breathing room after reviewing the fixed cost (but Travis negotiation might change that?), and with the npm project going on, time is more precious in the short term.

BTW I had the same urge to go for amazon, with an engineering mind, plus urge to remove unnecessary cost, as you know already :) But unless we absolutely need the extra saving, I'd say we focus on the business needs for the short term. Even if we do need the saving, hopefully we can find it somewhere else before changing infrastucture at this moment.

[nobodxbodon]

@whit537 maybe another option is to move the static sites (gip-rocks, inside, slackin?) to openshift? I don't have account that has access to their v2 platform, but maybe you do? And I'm sure these qualifies for their resource grants program. They should need little monitoring and managing, and migration will be simpler I suppose.

[Changaco]

Someone pointed out to me elsewhere that AWS also has some data transfer costs.

[clone1018]

Honestly, coming from both my day job and my night job, deploying usually sucks. You get so excited writing a new feature or designing some awesome thing, you go to deploy and run into issues! Even using concepts like continuous integrations somehow randomly causes issues.

Stick with Heroku until it's pain point (financially or manually) becomes more than figuring out a new infrastructure and developing new deployment processes.

[chadwhitacre]

Wow, even @nobodxbodon is okay with Heroku? Sounds like a consensus to me. :-)