From fe190b416153c01b9cf2bd94ba7c89a63b54ba96 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Tue, 29 Jun 2021 20:58:53 -0700 Subject: [PATCH] Bringin back README.md #patch (#192) * Bringin back README.md Signed-off-by: Ketan Umare * Moved contents of developing.rst to README (#193) Signed-off-by: Prafulla Mahindrakar * Replaced doc_requirements.in with doc-requirements.in Signed-off-by: Prafulla Mahindrakar Co-authored-by: pmahindrakar-oss --- flyteidl/README.md | 76 +++++++++++++++++++++++++++++++++++++++++ flyteidl/developing.rst | 69 ------------------------------------- flyteidl/index.rst | 3 +- 3 files changed, 77 insertions(+), 71 deletions(-) create mode 100644 flyteidl/README.md delete mode 100644 flyteidl/developing.rst diff --git a/flyteidl/README.md b/flyteidl/README.md new file mode 100644 index 0000000000..eca7656623 --- /dev/null +++ b/flyteidl/README.md @@ -0,0 +1,76 @@ +# Flyte IDL + +This is one of the core repositories of Flyte and contains the Specification of the Flyte Lanugage using protobuf messages, the Backend API specification in gRPC and Swagger REST. The repo contains generated clients and protocol message structures in multiple languages. Along-with the generated code, the repository also contains the Golang clients for Flyte's backend API's (the services grouped under Flyteadmin). + +* [flyte.org](https://flyte.org) +* [Flyte Docs](http://docs.flyte.org) +* [FlyteIDL API reference documentation](https://docs.flyte.org/projects/flyteidl/en/stable/index.html) + +## Contributing to FlyteIDL + +## Tooling for FlyteIDL + +1. Run ``make download_tooling`` to install generator dependencies + +```bash + make download_tooling +``` + +2. Make sure docker is installed locally. +3. Once installed, run ``make generate`` to generate all the code and mock client for FlyteAdmin Service aswell as the docs for it. + +```bash + make generate +``` + +4. To add new dependencies for documentation generation, modify ``doc-requirements.in`` and then + +```bash + make doc-requirements.txt +``` + +## Docs structure + +The index.rst files for protos are kept in parallel folder structure under the docs folder. +All the proto definitions are within protos/flyteidl and there corresponding docs are kept in protos/docs + +``` +docs +├── admin +│   ├── admin.rst +│   └── index.rst +├── core +│   ├── core.rst +│   └── index.rst +├── datacatalog +│   ├── datacatalog.rst +│   └── index.rst +├── event +│   ├── event.rst +│   └── index.rst +├── plugins +│   ├── index.rst +│   └── plugins.rst +├── service +│   ├── index.rst +│   └── service.rst +``` + +Each module in protos has same named module under the docs also. +eg : protos/flyteidl/core has same named doc structure placing it index and other documentation files in protos/docs/core + + +## Docs Generation + +* If introducing a new module then follow the structure for core files in `generate_protos.sh` file which helps in generating the core documentation from its proto files. +``` + core_proto_files=`ls protos/flyteidl/core/*.proto |xargs` + # Remove any currently generated file + ls -d protos/docs/core/* | grep -v index.rst | xargs rm + protoc --doc_out=protos/docs/core --doc_opt=restructuredtext,core.rst -I=protos `echo $core_proto_files` +``` + +* ``make generate`` would have already generated the modified rst files. + +* ``make html`` Generate the sphinx documentation from the docs folder to use the modified rst for docs. + diff --git a/flyteidl/developing.rst b/flyteidl/developing.rst deleted file mode 100644 index da417e18b2..0000000000 --- a/flyteidl/developing.rst +++ /dev/null @@ -1,69 +0,0 @@ -Contributing to Flyte IDL -========================= - -This is one of the core repositories of Flyte and contains the Specification of -the Flyte Lanugage. The Specification is maintained using Googles fantastic -Protcol buffers library. The code and docs are auto-generated. - -* [flyte.org](https://flyte.org) -* [Flyte Docs](http://flyte.readthedocs.org/) -* [FlyteIDL Docs](http://flyteidl.readthedocs.org) - -Generate Code from protobuf ----------------------------- -#. Make sure docker is installed locally. -#. Once installed run, ``make generate`` to generate all the code and mock - client for FlyteAdmin Service - - .. prompt:: bash - - make generate - -#. You might have to run, ``make download_tooling`` to install generator - dependencies - - .. prompt:: bash - - make download_tooling - -Generate Docs from protobuf ----------------------------- - -#. To add new dependencies for a doc, modify ``doc_requirements.in`` and then - - .. prompt:: bash - - make doc-requirements.txt - -Docs structure -************** - -The index.rst files for protos are kept in parallel folder structure under the docs folder. -All the proto definitions are within protos/flyteidl and there corresponding docs are kept in protos/docs - -Each module in protos has same named module under the docs also. -eg : protos/flyteidl/core has same named doc structure placing it index and other documentation files in protos/docs/core - - -Docs Generation -*************** - -Relies on protoc plugin pseudomuto/protoc-gen-doc for generating the docs. -Currently pseudomuto/protoc-gen-doc is being used for generating the RST files which are placed in protos/docs and corresponding folder based on the module. -Until this code is checkedin https://github.com/pseudomuto/protoc-gen-doc/pull/440, use the branch to build the protoc-gen-doc executable -Follow the steps to build and use the protoc-gen-doc - -#. This will install all the tooling including protoc-gen-doc plugin for protoc used for documentation - - .. prompt:: bash - - make download_tooling - -#. The protoc-gen-doc will now be available for protoc.Following is an example from `generate_protos.sh` file which helps in generating the core documentation from its proto files - - .. prompt:: bash - - core_proto_files=`ls protos/flyteidl/core/*.proto |xargs` - # Remove any currently generated file - ls -d protos/docs/core/* | grep -v index.rst | xargs rm - protoc --doc_out=protos/docs/core --doc_opt=restructuredtext,core.rst -I=protos `echo $core_proto_files` diff --git a/flyteidl/index.rst b/flyteidl/index.rst index f4c6b2c99f..acfdb77834 100644 --- a/flyteidl/index.rst +++ b/flyteidl/index.rst @@ -28,6 +28,5 @@ This specification is used to generate client stubs for `FlyteKit + README protos/index - developing