start of work to add basic setup

.gitignore

@@ -6,6 +6,7 @@
 
 # Terraform
 **/.terraform
+**/.terraform*
 **/terraform.tfplan
 **/terraform.tfstate
 **/terraform.tfstate.backup

README.md

@@ -40,15 +40,58 @@ Stay tuned!
 Once you have images, choose a directory under [examples](examples) to deploy from:
 
 ```bash
-$ cd examples/basic
-$ make
+$ cd examples/autoscale
+```
+
+Since we want to get hosts on the instance using the aws client, export your credentials to the environment
+for the instances:
+
+```bash
+export TF_VAR_aws_secret=$AWS_SECRET_ACCESS_KEY 
+export TF_VAR_aws_key=$AWS_ACCESS_KEY_ID 
+export TF_VAR_aws_session=$AWS_SESSION_TOKEN 
 ```
 
-### Develoipment
+And then init and build:
+
+```bash
+$ make init
+$ make fmt
+$ make validate
+$ make build
+```
 
-Note that I found it helpful to test commands first before actually running it
-automated! To connect to a new instance:
+And they all can be run with `make`:
+
+```bash
+$ make
+```
+
+You can then shell into any node, and check the status of Flux. I usually grab the instance
+name via "Connect" in the portal, but you could likely use the AWS client for this too.
 
 ```bash
 $ ssh -o 'IdentitiesOnly yes' -i "mykey.pem" [email protected]
 ```
+
+Check the cluster status and try running a job:
+
+```bash
+$ flux resource list
+     STATE NNODES   NCORES NODELIST
+      free      2        2 i-012fe4a110e14da1b.ec2.internal,i-0354d878a3fd6b017.ec2.internal
+ allocated      0        0 
+      down      0        0 
+```
+```bash
+[rocky@i-012fe4a110e14da1b ~]$ flux run -N 2 hostname
+i-012fe4a110e14da1b.ec2.internal
+i-0354d878a3fd6b017.ec2.internal
+```
+
+You can look at the startup script logs like this if you need to debug.
+```bash
+$ cat /var/log/cloud-init-output.log
+```
+
+That's it. Enjoy!

examples/autoscale/Makefile

@@ -0,0 +1,22 @@
+.PHONY: all
+all: init fmt validate build
+
+.PHONY: init
+init: 
+	terraform init	
+
+.PHONY: fmt
+fmt: 
+	terraform fmt
+
+.PHONY: validate
+validate: 
+	terraform validate
+
+.PHONY: build
+build: 
+	terraform apply
+
+.PHONY: destroy
+destroy: 
+	terraform destroy

examples/autoscale/README.md

@@ -0,0 +1,35 @@
+# Autoscale
+
+See the main [../../README.md](README.md) for instructions. Note that if you VM
+has Flux but not systemd, here is the last set of commands you can run instead of
+restarting the service:
+
+```bash
+mkdir -p /tmp/run/flux /var/lib/flux
+# Broker options shared between flux start and flux broker
+brokerOpts="-Scron.directory=/usr/local/etc/flux/system/cron.d  \
+     -Srundir=/run/flux   
+     -Sstatedir=/var/lib/flux   
+     -Slocal-uri=local:///run/flux/local   
+     -Slog-stderr-level=6   
+     -Slog-stderr-mode=local   
+     -Sbroker.rc2_none   
+     -Sbroker.quorum=0   
+     -Sbroker.quorum-timeout=none   
+     -Sbroker.exit-norestart=42   
+     -Scontent.restore=auto 
+     -Slog-stderr-mode=local 
+     -Stbon.zmqdebug=1"
+
+broker=$(echo $NODELIST | cut -d"," -f1)
+
+# TODO need to look at and debug permissions of things. For now, this seems to work.
+# I only put this here if we eventually want separate logic
+if [[ $(hostname) == "$broker" ]]; then
+   echo "Hello I am the broker, $(hostname)"
+   /usr/local/bin/flux broker --config-path=/usr/local/etc/flux/system/conf.d $brokerOptions
+else
+   echo "Hello I am a worker, $(hostname)"
+   /usr/local/bin/flux start -o --config /usr/local/etc/flux/system/conf.d $brokerOptions
+fi
+```