-
Notifications
You must be signed in to change notification settings - Fork 5
ComputingEnvironment
Home > TravelModel > UsersGuide > ComputingEnvironment
The hardware and software MTC uses to execute Travel Model One are described on this page. To date, MTC has not experimented enough with the model to define the minimum or ideal hardware configuration. As such, the description here is for a hardware set up that is sufficient. It is important to note that both the software and model structure are highly configurable and flexible; depending on the analysis needs, the required computing power could vary dramatically.
MTC uses a single server with the following characteristics:
- Operating system: Microsoft Windows Server 2012 R2 Standard, 64-bit;
- Processors: Two Intel Xeon E5-2680 v3 @ 2.50 GHz (24 cores, 48 logical processors);
- Memory: 256.0 GB RAM
We have also successfully run the Travel Model on Amazon EC2 instances (hundreds of runs!) using custom images from Citilabs with Cube pre-installed. Details:
- General purpose, m4.10xlarge as defined in June 2019
- 40 vCPUs
- Memory: 160GB RAM
- Turn on "Network Discovery". Verify it is successful by checking known network drives (e.g. M, L).
- Mount "mainmodelshare" to X: drive.
- Create a new network folder for model run.
For more details on configure the server machine, please refer to the documentation M:\Software\set_up_VM_for_TM1.5_runs.docx
.
The following software are required to execute the MTC travel model. Paths are set in SetPath.bat
The travel model currently uses version 6.4.5 of Bentley Cube software. Cube is used to build skims, manipulate networks, manipulate matrices, and perform assignments. Note: before installing Cube, make sure that Microsoft .NET Framework 3.5 is installed/enabled.
This is needed for MatrixManager.
The Cube Cluster software allows for the Cube scripts to be multi-threaded. In the current approach, the travel model uses 48 computing nodes. However, the Cube scripts can be manipulated to use any number of computing nodes across any number of machines, provided each machine has, at a minimum, a Cube Voyager node license. Cube Cluster is not strictly necessary, as the Cube scripts can be modified to use only a single computing node. Such an approach would dramatically increase run times.
The COMMPATH environment variable for use with Cube Cluster is set in SetPath.bat.
MTC's travel model operates on the open-source Coordinated Travel - Regional Activity-based Modeling Platform (or CT-RAMP) developed by Parsons Brinckerhoff. The software is written in the Java programming language. Because the CT-RAMP software compiles code "on-the-fly", the 64-bit Java Development Kit (version 1.8) must be installed on each computer running the CT-RAMP software. The Java Development Kit includes the Java Runtime Environment. The 64-bit version of the software allows CT-RAMP to take advantage of larger memory addresses.
Certain text file manipulations are handled in the travel model using the free GAWK software. GAWK must be installed on the computer executing the Cube scripts.
The CT-RAMP software allows discrete choice models to be specified via so-called UtilityExpressionCalculators. These files are Excel-based.
Python(64-bit) is used to execute a variety of scripts throughout the model run. See the section, Setup your python environment in Conda, in the SetupConfiguration page of the UsersGuide.
R (and Tableau) is used for calculating various metrics and generating output summaries.
- Install R
- Install RStudio
- Install RTools
- Install the following packages:
Package Version
--------------- ----------------
tidyverse 1.3.2
foreign 0.8-83
scales 1.2.1
reshape2 1.4.4
tidyr 1.2.1
broom 1.0.1
cli 3.4.1
crayon 1.5.2
dbplyr 2.2.1
dplyr 1.0.10
dtplyr 1.2.2
forcats 0.5.2
ggplot2 3.3.6
googledrive 2.0.0
googlesheets4 1.0.1
haven 2.5.1
hms 1.1.2
httr 1.4.4
jsonlite 1.8.3
lubridate 1.8.0
magrittr 2.0.3
modelr 0.1.9
pillar 1.8.1
purrr 0.3.5
readr 2.1.3
readxl 1.4.1
reprex 2.0.2
rlang 1.0.6
rstudioapi 0.14
rvest 1.0.3
stringr 1.4.1
tibble 3.1.8
xml2 1.3.3
backports 1.4.1
ellipsis 0.3.2
generics 0.1.3
glue 1.6.2
assertthat 0.2.1
blob 1.2.3
DBI 1.1.3
lifecycle 1.0.3
R6 2.5.1
tidyselect 1.2.0
vctrs 0.5.0
withr 2.5.0
data.table 1.14.4
gtable 0.3.1
isoband 0.2.6
MASS 7.3-58.1
mgcv 1.8-41
gargle 1.2.1
uuid 1.1-0
cellranger 1.1.0
curl 4.3.3
ids 1.0.1
rematch2 2.1.2
pkgconfig 2.0.3
mime 0.12
openssl 2.0.4
fansi 1.0.3
utf8 1.2.2
clipr 0.8.0
vroom 1.6.0
callr 3.7.3
fs 1.5.2
knitr 1.40
rmarkdown 2.17
selectr 0.4-2
stringi 1.7.8
processx 3.8.0
rematch 1.0.1
rappdirs 0.3.3
evaluate 0.17
highr 0.9
yaml 2.3.6
xfun 0.34
nlme 3.1-160
Matrix 1.5-1
askpass 1.1
bslib 0.4.1
htmltools 0.5.3
jquerylib 0.1.4
tinytex 0.42
farver 2.1.1
labeling 0.4.2
munsell 0.5.0
RColorBrewer 1.1-3
viridisLite 0.4.1
bit64 4.0.5
tzdb 0.3.0
sys 3.4.1
bit 4.0.4
sass 0.4.2
cachem 1.0.6
memoise 2.0.1
digest 0.6.30
base64enc 0.1-3
fastmap 1.1.0
lattice 0.20-45
colorspace 2.0-3
ps 1.7.2
plyr 1.8.7
Rcpp 1.0.9
Clone NetworkWrangler repo.
- For additional machine setups, please refer to the documentation
M:\Software\set_up_VM_for_TM1.5_runs.docx
. - MTC modeling staff typically use the one of the SetupModel scripts (e.g., SetUpModel_PBA50.bat) to copy required model inputs/code into place onto the modeling machines. This is optional, but we find it helps us to reduce errors and to verify that the model run was setup correctly so we prefer it over manual copying/setup.
Previously the MTC/ABAG Analytical Modeling Wiki (http://analytics.mtc.ca.gov/foswiki/Main/WebHome)
Please email [email protected] if you find anything missing here.