ALEX P Week4 Databases #28
Conversation
| const csv = require("csv-parser"); | ||
| require("dotenv").config(); | ||
|
|
||
| async function loadCSVToMongoDB() { |
There was a problem hiding this comment.
Overall looks great! One thing I would recommend is to add some logic around making sure that you don't re-add the data if you run the script again. For example, you can check if there are any documents in the collection and if so, return early
|
|
||
| const result = await collection.aggregate([ | ||
| { $match: { Country: country } }, | ||
| { $group: { _id: "$Year", countPopulation: { $sum: { $add: ["$M", "$F"] } } } }, |
| const collection = db.collection("population"); | ||
|
|
||
| const result = await collection.aggregate([ | ||
| { $match: { Year: year, Age: ageGroup } }, |
There was a problem hiding this comment.
This will return all countries and their total populations, however the question only asks for populations per continent. You'll need the examine the data to see how the continent information is stored (hint: it's not intuitive). The output given in the assignment details may help you
| TotalPopulation: { $add: ["$M", "$F"] } | ||
| } | ||
| }, | ||
| { $group: { _id: "$Country", data: { $push: "$$ROOT" } } }, |
There was a problem hiding this comment.
I'm not sure what you get but when I run your code it logs [Object] for the results and I suspect that's because of the data: { $push: "$$ROOT" } .
This grouping will actually have no effect on the data because it's already matched on a specific year and age, so you can remove this clause completely
| const fs = require("fs"); | ||
| require("dotenv").config(); | ||
|
|
||
| async function importAccounts() { |
There was a problem hiding this comment.
Do you need this file? It looks like it's accomplishing the same purpose as setup.js
| throw new Error("Insufficient funds in the source account!"); | ||
| } | ||
|
|
||
| const fromChangeNumber = fromAccountDoc.account_changes.length + 1; |
There was a problem hiding this comment.
Using the number of items in the account_changes array as a change number may work, but will likely lead to the following problems:
- An account with thousands of changes will be very inefficient to read every time you need to calculate the next change number
- If for some reason a change was removed from the array (decreasing the length), you may end up with duplicate change numbers
- Multiple concurrent transfers may cause a race condition that results in duplicate change numbers because both transfers see the same number of changes
Can you think of a more robust way to calculate change numbers that handles all of these problems?
| remark | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
An essential part of mongodb transactions is that the transaction session must be included in all of the operations like this:
await accounts.findOneAndUpdate(
{ account_number: fromAccount },
{
$inc: { balance: -amount },
$push: {
account_changes: {
change_number: fromChangeNumber,
amount: -amount,
changed_date: new Date(),
remark
}
}
},
{ session } // must be present
);If you don't include the session then mongodb will not reliably be able to handle rollbacks and race conditions
|
Hi @dyaboykyl, |
| $group: { | ||
| _id: "$Country", | ||
| Year: { $first: "$Year" }, | ||
| Age: { $first: "$Age" }, | ||
| TotalPopulation: { $sum: "$TotalPopulation" }, | ||
| M: { $sum: "$M" }, | ||
| F: { $sum: "$F" }, | ||
| }, |
There was a problem hiding this comment.
This grouping will have no effect because there is only 1 document per year,age, and country filter. You can remove it altogether and you will still get the same results
| $match: { | ||
| Year: year, | ||
| Age: ageGroup, | ||
| Country: { $regex: /^[A-Z\s]+$/ }, |
There was a problem hiding this comment.
Nice solution! Technically it also includes "WORLD" which is not one of the continents in the desired output, but I think that's ok
| await accountsCollection.updateOne( | ||
| { account_number: fromAccount }, | ||
| { | ||
| $inc: { balance: -amount, change_counter: 1 }, |
There was a problem hiding this comment.
Using a separate field for the change counter is a great approach, nicely done. If you wanted to get really fancy and let mongodb do all the work, you could use an $set aggregation operator to reference the existing change counter in the new account_changes array element. That way you would not need to calculate the change number (lines 36 and 37)
Something like this:
await accounts.findOneAndUpdate(
{ account_number: account },
[
{
$set: {
balance: { $add: ["$balance", amount] },
next_change_number: { $add: ["$next_change_number", 1] },
account_changes: {
$concatArrays: [
"$account_changes", [{
change_number: "$next_change_number",
amount,
changed_date: new Date(),
remark,
}]
]
}
},
}
],
{ session }
);
Week 4 assignment is complete. Please check it.