Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not determine schema/structure #1254

Closed
fedegl opened this issue Dec 17, 2021 · 7 comments
Closed

Could not determine schema/structure #1254

fedegl opened this issue Dec 17, 2021 · 7 comments

Comments

@fedegl
Copy link

fedegl commented Dec 17, 2021

I upgraded to Rails 7 and started getting these errors on the CI environment.

It seems to fail at loading the DB from dump and thus the tables don't exist when rails is initialized and crashed.

-----> Detecting rake tasks
       Could not determine schema/structure from `ActiveRecord::Base.schema_format`:
       [2021-12-17T20:01:11 (process) #668][ERROR] appsignal: Push API key not set after loading config
       [2021-12-17T20:01:11 (process) #668][ERROR] appsignal: Not starting, no valid config for this environment
             PaperTrail 12.1.0 is not compatible with ActiveRecord 7.0.0. We allow PT
             contributors to install incompatible versions of ActiveRecord, and this
             warning can be silenced with an environment variable, but this is a bad
             idea for normal use. Please install a compatible version of ActiveRecord
             instead (>= 5.2, < 7.0). Please see the discussion in paper_trail/compatibility.rb
             for details.
       /app/app/overrides/measured/units/volume_override.rb:3: warning: already initialized constant Measured::Volume
       /app/vendor/bundle/ruby/3.0.0/gems/measured-2.8.1/lib/measured/units/volume.rb:2: warning: previous definition of Volume was here
       /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/postgresql/database_statements.rb:19:in `exec': PG::UndefinedTable: ERROR:  relation "vendors" does not exist (ActiveRecord::StatementInvalid)
       LINE 9:  WHERE a.attrelid = '"vendors"'::regclass
                                   ^
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/postgresql/database_statements.rb:19:in `block (2 levels) in query'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activesupport-7.0.0/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activesupport-7.0.0/lib/active_support/dependencies/interlock.rb:41:in `permit_concurrent_loads'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/postgresql/database_statements.rb:18:in `block in query'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activesupport-7.0.0/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activesupport-7.0.0/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activesupport-7.0.0/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activesupport-7.0.0/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/abstract_adapter.rb:765:in `block in log'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activesupport-7.0.0/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/abstract_adapter.rb:756:in `log'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/postgresql/database_statements.rb:17:in `query'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/postgresql_adapter.rb:916:in `column_definitions'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/abstract/schema_statements.rb:116:in `columns'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/schema_cache.rb:117:in `block in columns'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/schema_cache.rb:116:in `fetch'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/schema_cache.rb:116:in `columns'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/schema_cache.rb:125:in `block in columns_hash'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/schema_cache.rb:124:in `fetch'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/connection_adapters/schema_cache.rb:124:in `columns_hash'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/model_schema.rb:568:in `load_schema!'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/attributes.rb:264:in `load_schema!'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/encryption/encryptable_record.rb:124:in `load_schema!'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/model_schema.rb:554:in `block in load_schema'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/model_schema.rb:551:in `synchronize'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/model_schema.rb:551:in `load_schema'
       	from /app/vendor/bundle/ruby/3.0.0/gems/activerecord-7.0.0/lib/active_record/model_schema.rb:412:in `columns'
       	from /app/app/models/concerns/ransack_unaccent.rb:5:in `included'
       	from /app/app/models/vendor.rb:5:in `include'
       	from /app/app/models/vendor.rb:5:in `<class:Vendor>'
       	from /app/app/models/vendor.rb:3:in `<main>'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:100:in `register'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/kernel.rb:27:in `require'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader/helpers.rb:95:in `const_get'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader/helpers.rb:95:in `cget'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader.rb:232:in `block (2 levels) in eager_load'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader/helpers.rb:26:in `block in ls'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader/helpers.rb:18:in `each_child'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader/helpers.rb:18:in `ls'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader.rb:227:in `block in eager_load'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader.rb:212:in `synchronize'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader.rb:212:in `eager_load'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader.rb:312:in `each'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/loader.rb:312:in `eager_load_all'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/application/finisher.rb:79:in `block in <module:Finisher>'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/initializable.rb:32:in `instance_exec'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/initializable.rb:32:in `run'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/initializable.rb:61:in `block in run_initializers'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:228:in `block in tsort_each'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:431:in `each_strongly_connected_component_from'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:349:in `block in each_strongly_connected_component'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:347:in `each'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:347:in `call'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:347:in `each_strongly_connected_component'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:226:in `tsort_each'
       	from /app/vendor/ruby-3.0.3/lib/ruby/3.0.0/tsort.rb:205:in `tsort_each'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/initializable.rb:60:in `run_initializers'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/application.rb:369:in `initialize!'
       	from /app/config/environment.rb:5:in `<main>'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:100:in `register'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
       	from /app/vendor/bundle/ruby/3.0.0/gems/zeitwerk-2.5.1/lib/zeitwerk/kernel.rb:35:in `require'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/application.rb:345:in `require_environment!'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/command/actions.rb:28:in `require_environment!'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/command/actions.rb:15:in `require_application_and_environment!'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/commands/runner/runner_command.rb:33:in `perform'
       	from /app/vendor/bundle/ruby/3.0.0/gems/thor-1.1.0/lib/thor/command.rb:27:in `run'
       	from /app/vendor/bundle/ruby/3.0.0/gems/thor-1.1.0/lib/thor/invocation.rb:127:in `invoke_command'
       	from /app/vendor/bundle/ruby/3.0.0/gems/thor-1.1.0/lib/thor.rb:392:in `dispatch'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/command/base.rb:87:in `perform'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/command.rb:48:in `invoke'
       	from /app/vendor/bundle/ruby/3.0.0/gems/railties-7.0.0/lib/rails/commands.rb:18:in `<main>'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:100:in `register'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
       	from /app/vendor/bundle/ruby/3.0.0/gems/bootsnap-1.9.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
       	from /app/bin/rails:4:in `<main>'```
@olliebennett
Copy link

olliebennett commented Jan 2, 2022

I'm seeing the same with Rails 7.0.0 on Ruby 2.7.4 (not updated to Ruby 3.x yet).

I've not modified the config.active_record.schema_format = :ruby line in my application.rb and don't see any other documentation on how to configure the schema differently in Rails 7.

Running bin/rails db:reset works fine locally (i.e. the database is created with schema correctly loaded) and my specs pass.

Interestingly, running ActiveRecord::Base.schema_format in my console returns :ruby but also raises a deprecation suggesting to use ActiveRecord.schema_format instead.

P.S. I tried removing the paper_trail (and paper_trail-association_tracking) gems (as I see you're also using PaperTrail) but it had no effect / seems unrelated.

@olliebennett
Copy link

olliebennett commented Jan 2, 2022

I see the Heroku Ruby Buildpack is printing the error here.

The code is checks for db:schema:load_if_ruby (which was already deprecated but now removed in Rails 7 - according to release notes) but then should fall back to running puts ActiveRecord::Base.schema_format and extracting the last line of that output (should be ruby in my case). Not sure why that's apparently blank (i.e. the log prints the newline character but then nothing else). Is there a newline printed after ruby perhaps? Could case result.lines.last.strip compact to avoid any blank extra lines?

Next step might be to do some hacking on the buildpack and see what's happening in more detail. I'll come back to that if this ticket doesn't otherwise move forwards before then :)

@schneems
Copy link
Contributor

schneems commented Jan 3, 2022

Hello. Sorry for the delay here. I was out. I took a look and I was unable to reproduce this behavior with a Rails 7 app https://github.com/sharpstone/rails-jsbundling. I get a successful migration:

              Bundle completed (45.67s)
              Cleaning up the bundler cache.
       -----> Installing node-v16.13.1-linux-x64
       -----> Installing yarn-v1.22.17
       -----> Clearing db:test:purge db:create db:structure:dump rake tasks
       -----> Detecting rake tasks
       -----> Preparing test database
              Running: rake db:schema:load
              db:schema:load completed (2.04s)
              Running: rake db:migrate
              db:migrate completed (2.01s)

I'm going to need some more information to debug this. Do you have any support tickets open with this behavior on Heroku? If not, do you have an example app that reproduces the behavior that you could post publically? https://codetriage.com/example_app

@olliebennett
Copy link

olliebennett commented Jan 3, 2022

Thanks @schneems - I've raised a Heroku ticket @ https://help.heroku.com/tickets/1064070 with an example failing build linked, and production access granted. Hope that helps.

@schneems
Copy link
Contributor

schneems commented Jan 3, 2022

Got it. I responded over on that channel. If anyone else has an issue number, feel free to link it on GitHub. I've also opened up a PR with my example app running on Heroku CI. I hope to write a failing test that captures one or more of the conditions you all are seeing and incorporate it into the test matrix. That will help me work on a fix and hopefully prevent future regressions.

@schneems
Copy link
Contributor

schneems commented Jan 3, 2022

I've also pushed what I believe may be a fix to my branch 2e98e35. You can try it out by replacing heroku/ruby in your application's test buildpacks with https://github.com/heroku/heroku-buildpack-ruby#schneems/rails7-fix-heroku-ci. if you try that out and it works, please let me know. Also know that this is only for testing. Once a fix is deployed you'll need to switch back to heroku/ruby.

Also, I still am in need of a working reproduction of a failure case to ensure no regressions in the future.

schneems added a commit that referenced this issue Jan 4, 2022
Added a test for Rails 7 to exercise behavior in #1254. Update the Rails 7 example app to use the released version instead of alpha. Also update the app to be Heroku CI compat app (adds app.json):

- sharpstone/rails-jsbundling@34bd25c
- sharpstone/rails-jsbundling@a98a4e4
- sharpstone/rails-jsbundling@b4ec664
- sharpstone/rails-jsbundling@1fb48b4

Add CI tests against Rails 7
schneems added a commit that referenced this issue Jan 4, 2022
* Test Rails 7 app against Heroku CI

Added a test for Rails 7 to exercise behavior in #1254. Update the Rails 7 example app to use the released version instead of alpha. Also update the app to be Heroku CI compat app (adds app.json):

- sharpstone/rails-jsbundling@34bd25c
- sharpstone/rails-jsbundling@a98a4e4
- sharpstone/rails-jsbundling@b4ec664
- sharpstone/rails-jsbundling@1fb48b4

Add CI tests against Rails 7

* Specialize Rails7 Database setup

In Rails 6.1 the tasks used for database setup in our test pack were deprecated. They were removed in Rails 7 https://edgeguides.rubyonrails.org/7_0_release_notes.html.

This commit moves to specialize the tasks for Rails7+ to use `db:schema:load` per deprecation instruction:

```
DEPRECATION WARNING: Using `bin/rails db:schema:load_if_ruby` is deprecated and will be removed in Rails 7.0. Configure the format using `config.active_record.schema_format = :ruby` to use `schema.rb` and run `bin/rails db:schema:load` instead. (called from <main> at /app/bin/rake:4)
```

* v236
@schneems
Copy link
Contributor

schneems commented Jan 4, 2022

I've deployed my fix, you can revert back to heroku/ruby if you tried it out. Thanks for reporting this and thanks for everyone who was able to test it out.

@schneems schneems closed this as completed Jan 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants