Add support for SQLite

[nickcharlton]

With increasing interest in SQLite across the Rails community as a whole, it's valuable for us to start exploring it too. This expands the existing checks for PostgreSQL to also include SQLite so that where we want to try out SQLite we're able to. It also adjusts the CI workflow to exclude PostgreSQL when generating a SQLite-based application.

https://rubyonrails.org/2024/11/7/rails-8-no-paas-required
https://fractaledmind.github.io/tags/sqlite/

Closes #1245

[nickcharlton]

I'm being a bit … careful (?) with my README changes here. I want to suggest it's possible, but we're just trying it out, and it's worth evaluating and not that you should necessarily switch away from Postgres.

That said, maybe we need an explicit example?

[stevepolitodesign]

Yeah, this is tricky, and I'm not sure the best approach to take. Suspenders is about limiting the choices we need to make, but I recognize we're in an exploratory phase.

I don't know if we have an explicit example (yet), but I imagine it would relate to avoiding a Platform as a service.

[nickcharlton]

I was on the fence about putting a change in at all, but as you can't comment on things you haven't changed in a PR something had to do 😅 .

It's almost like it either needs to be missing, or come with an explanation and I'm not sure what I'd want to do there.

[purinkle]

We are trying to keep Suspenders as close to Rails defaults as possible, but I still expect us to have an opinion. Otherwise, we should run rails new and be done with it. Are we trying out SQLite? Should team members start new Rails apps with SQLite? Suspenders should reflect our practices, not our aspirations.

[MatheusRich]

Are we trying out SQLite? Should team members start new Rails apps with SQLite?

I think there are a number of (small?) projects that would benefit from a simpler setup. You just want to try out an idea or build a side project. That shouldn't require you to deploy 2 different servers

---

That being said, should we just run

rails db:system:change --to=sqlite3

[louis-antonopoulos]

Should this also cover the SQLite case?

[louis-antonopoulos]

1. Should this test description reflect "PostgreSQL or SQLite"?

2. Should this or another test assert that with_database "sqlite" does not raise?

[louis-antonopoulos]

Should this block be tested that it is present for Postgres and absent for SQLite?

[stevepolitodesign]

We might want to borrow from the Rails CI template.

[nickcharlton]

Oh, interesting!

I also note that they're actually using Valkey, which diverges in another way from what we usually do:

https://github.com/rails/rails/blob/3b79de3df3d200acd0f291b1a08d9181ddee4605/railties/lib/rails/generators/rails/app/templates/github/ci.yml.tt#L66-L73

[stevepolitodesign]

Yeah, this is tricky, and I'm not sure the best approach to take. Suspenders is about limiting the choices we need to make, but I recognize we're in an exploratory phase.

I don't know if we have an explicit example (yet), but I imagine it would relate to avoiding a Platform as a service.

[stevepolitodesign]

That being said, should we just run

rails db:system:change --to=sqlite3

This is interesting, and I'm curious to see how this works on a new app generated by Suspenders. I imagine it would still require manually editing the CI file, though.

@nickcharlton is this worth exploring?

[nickcharlton]

Discussion of the why is best had on the issue I opened last year: #1245.

I was hoping to come up with a change which wouldn't exactly fully enable support, but rather open us up to using it for experiments (I have a particular use case in mind for testing where the database driver not being an operational dependency is a significant benefit). But there's not a path I could think of to doing that with our current implementation, so here we are.

We also don't need to merge this in any time soon, in combination with something like #1241 we could use this branch for generating new applications whilst we discover what works and what doesn't work.

---

I hadn't actually thought about changing the driver after we generate the application. In many ways, doing that isn't any more clunky than generating an application with a generator on a branch. It does work without making too much of a mess, a patch:

diff --git a/Dockerfile b/Dockerfile
index f0e3145..2251764 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -30,7 +30,7 @@ FROM base AS build
 
 # Install packages needed to build gems
 RUN apt-get update -qq && \
-    apt-get install --no-install-recommends -y build-essential git libpq-dev pkg-config && \
+    apt-get install --no-install-recommends -y build-essential git pkg-config && \
     rm -rf /var/lib/apt/lists /var/cache/apt/archives
 
 # Install application gems
diff --git a/Gemfile b/Gemfile
index ed0aadf..4b99686 100644
--- a/Gemfile
+++ b/Gemfile
@@ -7,7 +7,7 @@ gem "rails", "~> 8.0.1"
 gem "propshaft"
 
 # Use postgresql as the database for Active Record
-gem "pg", "~> 1.1"
+gem "sqlite3", ">= 2.1"
 
 # Use the Puma web server [https://github.com/puma/puma]
 gem "puma", ">= 5.0"
diff --git a/config/database.yml b/config/database.yml
index 24f9d67..2640cb5 100644
--- a/config/database.yml
+++ b/config/database.yml
@@ -1,97 +1,41 @@
-# PostgreSQL. Versions 9.3 and up are supported.
+# SQLite. Versions 3.8.0 and up are supported.
+#   gem install sqlite3
 #
-# Install the pg driver:
-#   gem install pg
-# On macOS with Homebrew:
-#   gem install pg -- --with-pg-config=/usr/local/bin/pg_config
-# On Windows:
-#   gem install pg
-#       Choose the win32 build.
-#       Install PostgreSQL and put its /bin directory on your path.
-#
-# Configure Using Gemfile
-# gem "pg"
+#   Ensure the SQLite 3 gem is defined in your Gemfile
+#   gem "sqlite3"
 #
 default: &default
-  adapter: postgresql
-  encoding: unicode
-  # For details on connection pooling, see Rails configuration guide
-  # https://guides.rubyonrails.org/configuring.html#database-pooling
+  adapter: sqlite3
   pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
+  timeout: 5000
 
 development:
   <<: *default
-  database: app_name_development
-
-  # The specified database role being used to connect to PostgreSQL.
-  # To create additional roles in PostgreSQL see `$ createuser --help`.
-  # When left blank, PostgreSQL will use the default role. This is
-  # the same name as the operating system user running Rails.
-  #username: app_name
-
-  # The password associated with the PostgreSQL role (username).
-  #password:
-
-  # Connect on a TCP socket. Omitted by default since the client uses a
-  # domain socket that doesn't need configuration. Windows does not have
-  # domain sockets, so uncomment these lines.
-  #host: localhost
-
-  # The TCP port the server listens on. Defaults to 5432.
-  # If your server runs on a different port number, change accordingly.
-  #port: 5432
-
-  # Schema search path. The server defaults to $user,public
-  #schema_search_path: myapp,sharedapp,public
-
-  # Minimum log levels, in increasing order:
-  #   debug5, debug4, debug3, debug2, debug1,
-  #   log, notice, warning, error, fatal, and panic
-  # Defaults to warning.
-  #min_messages: notice
+  database: storage/development.sqlite3
 
 # Warning: The database defined as "test" will be erased and
 # re-generated from your development database when you run "rake".
 # Do not set this db to the same as development or production.
 test:
   <<: *default
-  database: app_name_test
+  database: storage/test.sqlite3
 
-# As with config/credentials.yml, you never want to store sensitive information,
-# like your database password, in your source code. If your source code is
-# ever seen by anyone, they now have access to your database.
-#
-# Instead, provide the password or a full connection URL as an environment
-# variable when you boot the app. For example:
-#
-#   DATABASE_URL="postgres://myuser:mypass@localhost/somedatabase"
-#
-# If the connection URL is provided in the special DATABASE_URL environment
-# variable, Rails will automatically merge its configuration values on top of
-# the values provided in this file. Alternatively, you can specify a connection
-# URL environment variable explicitly:
-#
-#   production:
-#     url: <%= ENV["MY_APP_DATABASE_URL"] %>
-#
-# Read https://guides.rubyonrails.org/configuring.html#configuring-a-database
-# for a full overview on how database connection configuration can be specified.
-#
+
+# Store production database in the storage/ directory, which by default
+# is mounted as a persistent Docker volume in config/deploy.yml.
 production:
-  primary: &primary_production
+  primary:
     <<: *default
-    database: app_name_production
-    username: app_name
-    password: <%= ENV["APP_NAME_DATABASE_PASSWORD"] %>
+    database: storage/production.sqlite3
   cache:
-    <<: *primary_production
-    database: app_name_production_cache
+    <<: *default
+    database: storage/production_cache.sqlite3
     migrations_paths: db/cache_migrate
   queue:
-    <<: *primary_production
-    database: app_name_production_queue
+    <<: *default
+    database: storage/production_queue.sqlite3
     migrations_paths: db/queue_migrate
   cable:
-    <<: *primary_production
-    database: app_name_production_cable
+    <<: *default
+    database: storage/production_cable.sqlite3
     migrations_paths: db/cable_migrate