Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using schema_format = ruby fails with a single ORDER BY column #163

Closed
dbackeus opened this issue Sep 24, 2024 · 2 comments
Closed

Using schema_format = ruby fails with a single ORDER BY column #163

dbackeus opened this issue Sep 24, 2024 · 2 comments

Comments

@dbackeus
Copy link

dbackeus commented Sep 24, 2024

Given the following migration:

options = "MergeTree PARTITION BY toYYYYMM(timestamp) ORDER BY timestamp"
create_table :events, id: false, force: :cascade, options: do |t|
  t.datetime :timestamp, null: false
end

The generated schema.rb file looks like:

create_table "events", primary_key: "timestamp", id: :datetime, precision: nil, options: "MergeTree PARTITION BY toYYYYMM(timestamp) ORDER BY timestamp SETTINGS index_granularity = 8192", force: :cascade do |t|
end

When attempting to load this schema it fails with:

ActiveRecord::ActiveRecordError: Response code: 404: (ActiveRecord::ActiveRecordError)
Code: 47. DB::Exception: Missing columns: 'timestamp' while processing query: 'toYYYYMM(timestamp)', required columns: 'timestamp' 'timestamp'. (UNKNOWN_IDENTIFIER) (version 24.8.4.13 (official build))

It appears that the timestamp column needs to be explicitly defined inside of the block. It's not enough that it's declared as primary_key: "...".

The issue doesn't occur when ordering by multiple columns (ie. composite primary keys).
Also the issue doesn't occur when using schema_format = :sql.

@PNixx
Copy link
Owner

PNixx commented Sep 27, 2024

Fixed in v1.1.3

@PNixx PNixx closed this as completed Sep 27, 2024
@dbackeus
Copy link
Author

@PNixx thanks for looking into this, but unfortunately the new version did not fix this issue for me.

I'm no longer getting any primary_key: in the dump but I'm still not getting the timestamp columns added inside the create_table statements. Here's the diff I'm getting with the new version:

Screenshot 2024-09-28 at 12 23 23

Both the email_sg_events and web_clickstream_events tables are missing timestamp in the dump.

I tried both just running db:schema:dump on my existing database and dropping and re-migrating the database from scratch, both result in the same diff.

In the # SQL: CREATE TABLE ... comment the column does exist as expected, but not in the DSL output. Full email_sg_events section:

  # TABLE: email_sg_events
  # SQL: CREATE TABLE email_sg_events ( `uuid` Nullable(String), `email` Nullable(String), `event` Nullable(String), `sg_message_id` Nullable(String), `url` Nullable(String), `url_offset` Nullable(String), `ip` Nullable(String), `useragent` Nullable(String), `timestamp` DateTime, `reason` Nullable(String) ) ENGINE = MergeTree PARTITION BY toYYYYMM(timestamp) ORDER BY timestamp SETTINGS index_granularity = 8192
  create_table "email_sg_events", id: false, options: "MergeTree PARTITION BY toYYYYMM(timestamp) ORDER BY timestamp SETTINGS index_granularity = 8192", force: :cascade do |t|
    t.string "uuid"
    t.string "email"
    t.string "event"
    t.string "sg_message_id"
    t.string "url"
    t.string "url_offset"
    t.string "ip"
    t.string "useragent"
    t.string "reason"
  end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants