2024 data reorganization: ctx.db

[nmlorg]

New year (technically new decade!), new data reorganization. I'm going to end up undoing a whole lot of stuff.

The ctx object will get yet another data storage field (in addition to ctx.user, ctx.userinfo, and ctx.bot.multibot.conf):  ctx.db.

(At least ctx.userinfo and maybe ctx.bot.multibot.conf will eventually go away.)
 

There will be a new package:  metabot.db. It will be pluggable, like metabot.calendars.google:

• metabot.db.file or .local or .yaml (built in)
• metabot.db.firestore
• metabot.db.github (maybe?)
   
  

There will be three components:

• ctx.db.bot
• ctx.db.user
• ctx.db.chat

corresponding to a new, Cloud Firestore-friendly config hierarchy:

ctx	Firestore	user can
db.user.name	/users/{userid}/info/self.name	read
db.user.timezone	/users/{userid}/conf/self.timezone	read/write
db.bot.name	/bots/{botid}/info/self.name	read
db.bot.token	/bots/{botid}/conf/self.token	read/write
db.chat.name	/bots/{botid}/info/{chatid}.name	read
db.chat.timezone	/bots/{botid}/conf/{chatid}.timezone	read/write

 
Users will have direct API access to the Firestore db, using rules to give them read access to /users/{userid}/*/* (and /bots/{botid}/*/* for bots for which they are an admin) and write access to /users/{userid}/conf/* (and /bots/{botid}/conf/*).

When they interact with the bot to request settings, it will generate a custom auth token with the uid set to their Telegram userid, then generate a link to something like https://nmlorg.github.io/metabot/db.html#{authtoken}/{botid}.

(Or maybe db.html#{authtoken}/bots/{botid}, allowing for direct navigation to db.html#{authtoken}/users/{userid}, but generating a direct link to a specific group chat would be awkward (db.html#{authtoken}/bots/{botid}/info/{chatid} that also implies /bots/{botid}/conf/{chatid}?). It might be better to stick to something like /{botid}, /{botid}/{chatid}, and maybe just /self to directly access their own settings.)

[nmlorg]

I'm on the verge of convincing myself to not give users direct write access to anything under /bots. Instead, users would have write access to something like /users/{userid}/changes/changes, which they could append a tuple of document/field path and the new value. The bot would scan /users/*/changes and attempt to apply each change more or less exactly like it just received a Telegram message from the user to change the given value. (The structure of the changes file could even literally be an array of strings containing /set ... or /admin ..., but that might complicate some other loosely planned changes.)

The benefits would be both increased auditability (continuing to be able to log who did what) and stricter/easier value validation (in Python, near the code where the value is used, versus in Rule language, in an entirely separate package).

This could also allow me to eliminate the conf/info collections: All user data and settings on /users/{userid} directly, all bot stuff (even group data?) on /bots/{botusername} directly, etc.

[nmlorg]

Potential new hierarchy:

path	readable	what
/users/{userid}	everyone	public info (full name, icon)
/users/{userid}/private/self	{uid} = {userid}	settings (bots owned, timezone)
/users/{userid}/private/changes	{uid} = {userid}	pending changes
/bots/{botusername}	everyone	public info (full name, icon)
/bots/{botusername}/private/self	{botusername} in /users/{uid}/private/self.bots	settings (token, isrunning)
/groups/{groupid}	everyone	public info (title, icon, admins?)
/bots/{botusername}/private/{groupid}	{botusername} in /users/{uid}/private/self.bots or {botusername}/{groupid} in /users/{uid}/private/self.groups	settings (timezone, greeting, reminders)