BigQuery: Run job in one project that uses resources from another

[alixhami]

This is a feature request if it is not supported by the library. See last line for tldr. I am updating a ruby sample to match the python sample for extracting a table. The python sample extracts the bigquery-public-data shakespeare table to a GCS bucket. When attempting this in ruby, I ran into a permissions error because of the way the request was structured by the library. The ruby library's extract_job method is attached to a table instance, so it tries to use the table's project to run the job.

Environment details

• OS: macOS High Sierra
• Ruby version: 2.4.4
• Gem name and version: google-cloud-bigquery 1.8.0

Steps to reproduce

Run code with bucket_name replaced with an actual bucket:

require "google/cloud/bigquery"

bucket_name = "my-bucket"
bigquery = Google::Cloud::Bigquery.new project: "bigquery-public-data"
dataset = bigquery.dataset "samples"
table = dataset.table "shakespeare"
destination_uri = "gs://#{bucket_name}/shakespeare.csv"

extract_job = table.extract_job(destination_uri) do |updater|
  # Location must match that of the source table.
  updater.location = "US"
end
extract_job.wait_until_done!  # Waits for the job to complete

puts "Exported #{table.id} to #{destination_uri}"

I get this error on the line where the extract_job is defined:

Google::Cloud::PermissionDeniedError:
       accessDenied: Access Denied: Project bigquery-public-data: The user <MY-SERVICE-ACCOUNT> does not have bigquery.jobs.create permission in project bigquery-public-data.

Is there any way to send an extract job with one project that uses resources in another project?

[quartzmo]

@alixhami Thanks for this issue! I'm going to initially label it as a question, but I'll change it to a feature request if needed.

[quartzmo]

@blowmage and I looked into this, and it will require adding a project_id optional parameter to Project#dataset, similar to the project param in Pubsub::Project#topic.

[quartzmo]

@blowmage and I looked into this, and it will require adding a project_id optional parameter to Project#dataset, similar to the project param in Pubsub::Project#topic.

Unfortunately this approach turned out to be extremely disruptive and error-prone, so I have have opened #2692 to add project_id to the extract methods.

[quartzmo]

The new solution will be to add versions of #extract, #extract_job, #copy and #copy_job to Project, as discussed in PR #2692, which has been closed.