[Docs] Reapply Databricks agent docs changes from #4008 #1558

Job	Run time
unpack-envvars	1s
Docker Build Images (datacatalog) / build_docker	2m 8s
Docker Build Images (flyteadmin) / build_docker	2m 43s
Docker Build Images (flytecopilot) / build_docker	2m 8s
Docker Build Images (flytepropeller) / build_docker	3m 20s
Build Images / Build & Push Image (datacatalog) / Push to Github Registry	2m 30s
Build Images / Build & Push Image (flyteadmin) / Push to Github Registry	3m 13s
Build Images / Build & Push Image (flytecopilot) / Push to Github Registry	2m 37s
Build Images / Build & Push Image (flytepropeller) / Push to Github Registry	4m 12s
Build Images / Build & Push Image (flytescheduler) / Push to Github Registry	3m 21s
Check Go Generate (datacatalog) / Go Generate	1m 27s
Check Go Generate (flytecopilot) / Go Generate	1m 16s
Check Go Generate (flytepropeller) / Go Generate	2m 32s
Check Go Generate (flyteadmin) / Go Generate	2m 27s
Unit Tests (datacatalog) / Run Unit Test	2m 49s
Unit Tests (flyteadmin) / Run Unit Test	4m 24s
Unit Tests (flytecopilot) / Run Unit Test	2m 33s
Unit Tests (flyteplugins) / Run Unit Test	3m 57s
Unit Tests (flytepropeller) / Run Unit Test	4m 16s
Unit Tests (flytestdlib) / Run Unit Test	3m 50s
Lint (datacatalog) / Run Lint	2m 24s
Lint (flyteadmin) / Run Lint	3m 16s
Lint (flyteidl) / Run Lint	1m 50s
Lint (flyteplugins) / Run Lint	2m 54s
Lint (flytepropeller) / Run Lint	3m 16s
Lint (flytestdlib) / Run Lint	2m 42s
Integration Test (flyteadmin) / Integration tests	5m 53s
	1h 17m 59s

Provide feedback