Cant find path to datasets

[baukenieuwenhuis]

In the notebook ex 1, the yaml file connects to the path:

/workspaces/AE4353-Y24/data/2D_QUAD_HOVER.npz

however the path which I copied into the .json file is:

/home/baukenieuwenhuis/ae4353_dataset

the error I get is in the third cell when trying to prepare the data:

Should I change the path in the .json file? or is there an underlying problem.

[yilun-wu]

Do you have 2D_QUAD_HOVER.npz under /home/baukenieuwenhuis/ae4353_dataset?

[baukenieuwenhuis]

Yes that's the pwd path I copied following the readme, but it still says it cant find it

[Huizerd]

can you paste your devcontainer.json and 2d.yaml here?

[baukenieuwenhuis]

Yaml:

trajectory_dim: 2
dataset_file: /workspaces/AE4353-Y24/data/2D_QUAD_HOVER.npz

batchsize_train: 1024
batchsize_val: 4096
shuffle_train: True
shuffle_val: False
num_workers_train_loader: 0
num_workers_val_loader: 0

model_hidden_dims: 120
model_num_layers: 4

epochs: 2
validate_per_steps: 500
lr: 1.0 # to be tuned

json file:
{
"name": "ae4353",
"build": {
"dockerfile": "./Dockerfile",
"context": ".."
},
"mounts": [
// Bind mount data here if necessary e.g.
//source=/home/baukenieuwenhuis/ae4353_dataset,target=${containerWorkspaceFolder}/data,type=bind
],
"features": {
"ghcr.io/devcontainers/features/common-utils:2": {
"installZsh": "true",
"username": "vscode",
"userUid": "1000",
"userGid": "1000",
"upgradePackages": "true"
},
"ghcr.io/devcontainers/features/git:1": {
"version": "latest",
"ppa": "false"
}
},
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
"settings": {
// TODO: change to specific conda env python bin
"python.defaultInterpreterPath": "/opt/conda/envs/AE4353/bin/python",
"python.linting.enabled": true,
"jupyter.widgetScriptSources": [
"jsdelivr.com",
"unpkg.com"
],
"notebook.defaultFormatter": "ms-python.black-formatter",
"notebook.formatOnSave.enabled": true,
"python.terminal.activateEnvironment": true,
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter",
"editor.formatOnSave": true
}
},
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"ms-python.black-formatter",
"ms-toolsai.jupyter",
"ms-toolsai.tensorboard",
"github.copilot",
"github.copilot-chat"
]
}
},
"hostRequirements": {
"gpu": "optional"
},
// Uncomment the below "runArgs" section if you intend to use a GPU.
// "runArgs": [
// "--gpus",
// "all"
// ],
"postCreateCommand": "./.devcontainer/postCreateCommand.sh",
// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "uname -a",
// Set remoteUser to root to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "vscode",
"remoteEnv": {
"SERVICE_HOST": "host.docker.internal",
"SERVICE_PORT": "9876"
}
}

[Huizerd]

this line

//source=/home/baukenieuwenhuis/ae4353_dataset,target=${containerWorkspaceFolder}/data,type=bind

is still commented out (see the // at the start. it should look like

source=/home/baukenieuwenhuis/ae4353_dataset,target=${containerWorkspaceFolder}/data,type=bind

and then it should work!

[Huizerd]

you can also see this from the fact that it has a dark grey color in the vscode window, which indicates that it's a comment (like the one above saying // Bind mount data...

[baukenieuwenhuis]

//"source=,target=${containerWorkspaceFolder}/data,type=bind"

this is the original line, should I remove "" as well or should they stay? that's what i confused with commented line

[Huizerd]

it should look like this (with your folder name obviously)

so the quotes are necessary. they indicate that the whole thing is a string (like in python), which we want to send to the container. the // are like python's # for a comment

[baukenieuwenhuis]

Tried to rebuild the json file after fixing the line but now I get this error

[Huizerd]

someone else had these issues too. these packages are for security but not necessary for the course. so if you make your Dockerfile look like this:

commenting out lines 9-13 so it won't try to install these packages

[baukenieuwenhuis]

Still get the following message:

[Huizerd]

can you rebuild the container without cache?

[baukenieuwenhuis]

Gives the exact same error message

[Huizerd]

we can have a look today!