Added support of readme generation #35

goruha · 2017-11-05T19:54:30Z

What

Support of README generation
Support pluggable datasources (check how we deal with make & terraform projects)
Generate readme with one command

Why

We have standard parts in readme files.
Different projects need different way to collect data for templates
Unify way to generate readme independent of project type

How to use

Create Makefile in repo root dir ( Example https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/blob/feature-support-readme-generation/Makefile )
Run $ make readme:init to fetch basic .README.md
Customize .README.md template as you need
Run $ make readme:build to generate README.md based on .README.md

Avaliable placeholders

{{ (datasource "git").name }} - Repository name
{{ (datasource "git").url }} - Repository URL
{{ (datasource "section").help }} - Cloudposse contact us text
{{ (datasource "section").contributing }} - Cloudposse code of conduct rules
{{ (datasource "license").apache2 }} - Cloudposse short license text
{{ (datasource "section").about }} - Cloudposse about text
{{ (datasource "section").warning }} - Warning DO NOT EDIT README.md
{{- (datasource "contributor").erik }} - Erik's contributor card
{{- (datasource "contributor").igor }} - Igor's contributor card
{{- (datasource "contributor").andrew }} - Andrew contributor card
{{- (datasource "contributor").konstantin }} - Konstantin's contributor card
{{- (datasource "contributor").sergey }} - Sergey's contributor card
{{- (datasource "contributor").valeriy }} - Valeriy's contributor card
{{- (datasource "contributor").vladimir }} - Vladimir's contributor card
{{- (datasource "contributor")._links }} - Links to github profiles and images to contributors (Required if at least one contributor specified in README)
{{ (datasource "terraform").input }} - Terraform module input variables table
{{ (datasource "terraform").output }} - Terraform module outputs table
{{ (datasource "make").help }} - Make targets list with descriptions (based on make help)

Under the hood

Gomplate as cli tool for gotempale
Terraform docs as tool to parse terraform module and get input\output

Example

cloudposse/terraform-aws-elastic-beanstalk-environment#19
#35

osterman · 2017-11-09T04:54:45Z

modules/docs/Makefile.docs

+.PHONY : docs\:build
+## Create README.md by building it from .README.md file
+docs\:build:


I think this should be called build-readme since it's very specific to readme. Or docs: should be renamed to readme:

osterman · 2017-11-09T04:57:15Z

What do you think about a templates/readme/.... structure. So then the README.md would go in there along with all data sources.

osterman · 2017-11-09T05:03:05Z

modules/docs/Makefile.docs

+.PHONY : docs\:make
+## Cleanup placeholders in README.md
+docs\:make:


I don't like the make target name as it's overloading the meaning of make

osterman · 2017-11-09T05:05:48Z

Seems like the build-harness should make use of the this convention for it's own README.md

osterman · 2017-11-10T06:19:54Z

templates/docs/sections.yml

@@ -0,0 +1,43 @@
+---
+  warning: |-
+    <!--------------------------------DO NOT EDIT README.md-------------------------------->


osterman · 2017-11-12T23:48:12Z

README.md

@@ -63,14 +65,20 @@ Available targets:
  help                                This help screen
  jenkins:run-job-with-tag            Run a Jenkins Job with $(TAG)
  make:lint                           Lint all makefiles
+  readme:build                        Create README.md by building it from .README.md file
+  readme:deps                         Install dependencies
+  readme:init                         Create basic minimalistic .README.md template file


Initialize a minimalistic README.md from template file

osterman · 2017-11-12T23:48:28Z

README.md

@@ -63,14 +65,20 @@ Available targets:
  help                                This help screen
  jenkins:run-job-with-tag            Run a Jenkins Job with $(TAG)
  make:lint                           Lint all makefiles
+  readme:build                        Create README.md by building it from .README.md file


Generate README.md from template file

osterman · 2017-11-13T00:01:56Z

templates/readme/terraform.yml

+  output: |-
+      ## Output
+
+      <!--------------------------------REQUIRE POSTPROCESSING-------------------------------->


What sort of postprocessing?

osterman · 2017-11-13T00:03:50Z

modules/terraform/Makefile.docs

+endef
+
+define terraform_data-docs-prepare-data
+	$(shell $(TERRAFORM_DOCS) json . > /tmp/terraform_data.json)


s/terraform_data.json/terraform-docs.json/

osterman · 2017-11-13T00:04:56Z

modules/readme/Makefile.docs.repo

+--datasource repo_data=file:///tmp/repo_data.yml
+endef
+
+define repo-docs-prepare-data


I don't know what this does

osterman · 2017-11-13T00:06:10Z

modules/readme/Makefile.docs.repo

+endef
+
+define repo_data-docs-provide
+--datasource repo_data=file:///tmp/repo_data.yml


I don't like the reliance on /tmp without a tmp directory. If we're usnig this in multiple projects, there will be collisions.

osterman · 2017-12-01T18:42:54Z

modules/docs/Makefile

+.PHONY : docs\:build
+## Create $OUT file by building it from $IN template file
+docs\:build:
+	@BUILD_HARNESS_PATH=$(BUILD_HARNESS_PATH) \


This is already exported, so should need to be passed

When I remove pass $BUILD_HARNESS_PATH
I got this error

2017/12/04 15:34:19 Can't read /home/goruha/projects/cloudposse/build-harness/: &os.PathError{Op:"read", Path:"/home/goruha/projects/cloudposse/build-harness/", Err:0x15} README.md doc generated

osterman · 2017-12-01T18:43:07Z

modules/docs/Makefile

+## Create $OUT file by building it from $IN template file
+docs\:build:
+	@BUILD_HARNESS_PATH=$(BUILD_HARNESS_PATH) \
+		GOMPLATE=$(GOMPLATE) \


Why not just export it above?

Do we have example for that?

osterman · 2017-12-01T21:18:18Z

modules/git/docs/module.sh

+}
+
+function git_data-docs-cleanup-data {
+  rm -rf $GIT_DATA_FILE


-f should be good enough and safer

osterman · 2017-12-01T21:18:49Z

modules/readme/Makefile.docs

+## Create README.md by building it from .README.md file
+readme\:build:
+	@IN=$(README_TEMPLATE_FILE) \
+		OUT=$(README_FILE) \


normalize whitespace

osterman · 2017-12-01T21:19:09Z

modules/terraform/docs/module.sh

+
+
+function terraform-docs-prepare-data {
+  if [ ! -s /usr/local/bin/$TERRAFORM_DOCS ]; then


normalize whitespace

osterman · 2017-12-05T03:56:41Z

Makefile

@@ -1,6 +1,6 @@
-BUILD_HARNESS_PATH ?= .
+BUILD_HARNESS_PATH ?= $(shell 'pwd')


osterman · 2017-12-05T04:04:48Z

modules/docs/Makefile

+		REPO=$(GOMPLATE_REPO) \
+		FILE=$(GOMPLATE)_$(OS)-amd64 \
+		VERSION=$(GOMPLATE_VERSION) \
+		OUTPUT=/usr/local/bin/$(GOMPLATE) > /dev/null


This should install to a path relative to the build-harness
a) avoids permission issues
b) avoids conflicts between different build-harness checkouts

Added support of readme generation

c937138

goruha requested a review from osterman November 5, 2017 19:55

goruha assigned osterman Nov 5, 2017

Simplify docs gen

f1e1bc2

osterman reviewed Nov 9, 2017

View reviewed changes

osterman reviewed Nov 10, 2017

View reviewed changes

goruha added 5 commits November 13, 2017 03:06

Made pluggable architecture and one command generation

772c210

Remove trunl

852cbc8

Fix README template

a03ede3

Fix README

91d2bcb

Fix README

d2216f7

osterman reviewed Nov 12, 2017

View reviewed changes

osterman reviewed Nov 13, 2017

View reviewed changes

Redo doc generate to bash script

a14309f

goruha requested review from aknysh and const-bon November 30, 2017 18:26

goruha assigned aknysh and const-bon Nov 30, 2017

Make all dynamic datasources be in tmp dir

0b312b6

osterman reviewed Dec 1, 2017

View reviewed changes

goruha added 3 commits December 4, 2017 16:17

Address comments

7dba942

Made wordwrap for terraform tables

9dc0d14

Remove wordwrap

334f941

osterman reviewed Dec 5, 2017

View reviewed changes

Address comments

eb502c4

osterman approved these changes Dec 8, 2017

View reviewed changes

goruha merged commit dbd0e31 into master Dec 8, 2017

goruha deleted the feature-readme-generation branch December 8, 2017 18:15

charlieparkes added a commit to charlieparkes/build-harness that referenced this pull request May 1, 2020

Sync upstream (cloudposse#35)

661e5e5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added support of readme generation #35

Added support of readme generation #35

goruha commented Nov 5, 2017 •

edited

Loading

osterman Nov 9, 2017

goruha Nov 11, 2017

osterman commented Nov 9, 2017

osterman Nov 9, 2017

goruha Nov 11, 2017

osterman commented Nov 9, 2017

osterman Nov 10, 2017 •

edited

Loading

goruha Nov 11, 2017

osterman Nov 12, 2017

osterman Nov 12, 2017

osterman Nov 13, 2017

osterman Nov 13, 2017

osterman Nov 13, 2017

osterman Nov 13, 2017

osterman Dec 1, 2017

goruha Dec 4, 2017

osterman Dec 1, 2017

goruha Dec 4, 2017

osterman Dec 1, 2017

goruha Dec 4, 2017

osterman Dec 1, 2017

goruha Dec 4, 2017

osterman Dec 1, 2017

goruha Dec 4, 2017

osterman Dec 5, 2017

goruha Dec 8, 2017

osterman Dec 5, 2017



		function terraform-docs-prepare-data {
		if [ ! -s /usr/local/bin/$TERRAFORM_DOCS ]; then

		@@ -1,6 +1,6 @@
		BUILD_HARNESS_PATH ?= .
		BUILD_HARNESS_PATH ?= $(shell 'pwd')

Added support of readme generation #35

Added support of readme generation #35

Conversation

goruha commented Nov 5, 2017 • edited Loading

What

Why

How to use

Avaliable placeholders

Under the hood

Example

Choose a reason for hiding this comment

Choose a reason for hiding this comment

osterman commented Nov 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

osterman commented Nov 9, 2017

osterman Nov 10, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

goruha commented Nov 5, 2017 •

edited

Loading

osterman Nov 10, 2017 •

edited

Loading