Add Fog Support for Azure Stack

[ClaudiaBaur]

The cf cloud controller uses Fog for ARM as default for the blob store. In Fog, only endpoints for public Azure, China, US_Government and Germany are implemented, but not for Azure Stack yet.
See also storage_endpoint_suffix in

fog-azure-rm/lib/fog/azurerm/utilities/general.rb

Lines 109 to 126 in 72c7dd1

	# storage_endpoint_suffix is nil in ms_rest_azure 0.6.2
	# Reference the issue: https://github.com/Azure/azure-sdk-for-ruby/issues/603
	def storage_endpoint_suffix(environment = ENVIRONMENT_AZURE_CLOUD)
	case environment
	when ENVIRONMENT_AZURE_CHINA_CLOUD
	# MsRestAzure::AzureEnvironments::AzureChina.storage_endpoint_suffix
	'.core.chinacloudapi.cn'
	when ENVIRONMENT_AZURE_US_GOVERNMENT
	# MsRestAzure::AzureEnvironments::AzureUSGovernment.storage_endpoint_suffix
	'.core.usgovcloudapi.net'
	when ENVIRONMENT_AZURE_GERMAN_CLOUD
	# MsRestAzure::AzureEnvironments::AzureGermanCloud.storage_endpoint_suffix
	'.core.cloudapi.de'
	else
	# MsRestAzure::AzureEnvironments::Azure.storage_endpoint_suffix
	'.core.windows.net'
	end
	end

As a result, the storage_endpoint_suffix on Azure Stack is wrong, as it only uses the default for public Azure which is .core.windows.net.
Please add a storage endpoint for Azure Stack.

[bilal-naeem-confiz]

Hi @ClaudiaBaur
We are looking into this and will get back to you shortly.

[bingosummer]

I made some changes to support Azure Stack blob storage.
https://github.com/CloudFoundryOnAzure/fog-azure-rm

One issue is that the version of mime-types is too old. See #359.
If the dependency of mime-types can be removed, then the fog-azure-rm gem can work well with cloud foundry controller.

[dsboulder]

I'm very interested in this, is anyone willing to think about merging it?

[zrob]

Hey all,

A member of the team that maintains the cloud controller component for Cloud Foundry here. We would really like to be able to support folks on Azure Stack via Fog.

It would be great to hear what y'all think about the example changes above and if you plan to make any of these updates soon.

Thanks!

[bingosummer]

@dsboulder @zrob We had an offline discussion with fog-azure-rm team. From fog-azure-rm's point of view, updating the mime-types will cause conflicts in another application that are using fog-azure-rm e.g. OneOps. So mime-types won't be updated in near future.

The short term plan is: I forked fog-azure-rm to my personal org (CloudFoundryOnAzure).
https://github.com/CloudFoundryOnAzure/fog-azure-rm
I can add Azure Stack support into the repo and send a PR to cloud_controller_ng to change from the official gem to my private one.
After fog-azure-rm removes or upgrades mime-types in future, we can change it back.

How do you think of the above proposal?

[ClaudiaBaur]

I am also still waiting for this, would be interested to test...

[zrob]

I'd like to propose that the fog-azure team consider publishing an updated gem with updated mime-types. Any projects that have an issue moving forward can always pin their version to one that works for them.

It seems awkward to block moving this project forward on another project updating its dependencies. This seems like exactly what Gemfiles are designed to help with.

By not upgrading it also causes conflicts with other fog components that use up-to-date dependencies, for example #359.

[bingosummer]

Thanks @zrob . I totally agree with you.

@abbas-sheikh-confiz , after our last sync, could you please re-think about merging the codes? If you are OK, I can send a PR ASAP.

[bilal-naeem-confiz]

@zrob @bingosummer
I am thinking maybe we can create a separate branch in which we update the gem dependencies to what you guys require and release the gem as vA.B.C.1 (currently the version is 0.4.7 so your release would be something like v0.4.7.1).
Will this work for you guys?

[bingosummer]

Thanks @bilal-naeem-confiz .
@zrob , any comments on bilal's suggestion?

[zrob]

@bilal-naeem-confiz that sounds good and would be greatly appreciated :)

[bilal-naeem-confiz]

@bingosummer @zrob
I have created a separate branch for this named fog-arm-cf. Can you kindly close your existing PRs and generate PRs against this branch. I'll merge them and release a new fog-azure-rm gem for you guys to use.

[bingosummer]

Thanks @bilal-naeem-confiz . I will submit a PR to the new branch later.

[ghost]

@bingosummer, @bilal-naeem-confiz Is this merged on master CF branch now? We are installing CF on azure stack and would like to utilize Azure Stack Blobstorage

[zrob]

@lordcf this was merged into the CF cloud controller a few days ago, but has not made it into a release yet. It will be available in capi-release 1.67.0. You can watch for cf-deployment to begin consuming capi-release 1.67.0 or newer.

[bingosummer]

I've verified cf-deployment v4.0.0 with capi-release v1.67.0, and it works well with Azure Stack Blobstorage.
I make a PR to the bosh-setup template to support Azure Stack blobstorage.
After it's merged, you can verify it easily from your side. @ClaudiaBaur @lordcf

[ghost]

@bingosummer Yep, we could install CF 4.0 with capi-release v1.67.0 and it worked with Azure Stack Blobstorage as well for us.