Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CI's docs filter generation #35087

Merged
merged 2 commits into from
Oct 20, 2023
Merged

Conversation

Taragolis
Copy link
Contributor

Fix list generation for build-docs CI step.

Run breeze build-docs apache-airflow amazon apache-flink apache-spark celery cncf-kubernetes google
                                                                                
 Usage:                                                                         
 breeze build-docs                                                              
 [OPTIONS] [all-providers | providers-index | apache-airflow | docker-stack |   
 helm-chart | airbyte | alibaba | amazon | apache.beam | apache.cassandra |     
 apache.drill | apache.druid | apache.flink | apache.hdfs | apache.hive |       
 apache.impala | apache.kafka | apache.kylin | apache.livy | apache.pig |       
 apache.pinot | apache.spark | apache.sqoop | apprise | arangodb | asana |      
 atlassian.jira | celery | cloudant | cncf.kubernetes | common.sql |            
 daskexecutor | databricks | datadog | dbt.cloud | dingding | discord | docker  
 | elasticsearch | exasol | facebook | ftp | github | google | grpc | hashicorp 
 | http | imap | influxdb | jdbc | jenkins | microsoft.azure | microsoft.mssql  
 | microsoft.psrp | microsoft.winrm | mongo | mysql | neo4j | odbc | openfaas | 
 openlineage | opensearch | opsgenie | oracle | pagerduty | papermill | plexus  
 | postgres | presto | redis | salesforce | samba | segment | sendgrid | sftp | 
 singularity | slack | smtp | snowflake | sqlite | ssh | tableau | tabular |    
 telegram | trino | vertica | yandex | zendesk]...                              
                                                                                
                                                                                
 Try running the '--help' flag for more information.                            
                                                                                
╭─ Error ──────────────────────────────────────────────────────────────────────╮
│ Invalid value for '[all-providers | providers-index | apache-airflow |       │
│ docker-stack | helm-chart | airbyte | alibaba | amazon | apache.beam |       │
│ apache.cassandra | apache.drill | apache.druid | apache.flink | apache.hdfs  │
│ | apache.hive | apache.impala | apache.kafka | apache.kylin | apache.livy |  │
│ apache.pig | apache.pinot | apache.spark | apache.sqoop | apprise | arangodb │
│ | asana | atlassian.jira | celery | cloudant | cncf.kubernetes | common.sql  │
│ | daskexecutor | databricks | datadog | dbt.cloud | dingding | discord |     │
│ docker | elasticsearch | exasol | facebook | ftp | github | google | grpc |  │
│ hashicorp | http | imap | influxdb | jdbc | jenkins | microsoft.azure |      │
│ microsoft.mssql | microsoft.psrp | microsoft.winrm | mongo | mysql | neo4j | │
│ odbc | openfaas | openlineage | opensearch | opsgenie | oracle | pagerduty | │
│ papermill | plexus | postgres | presto | redis | salesforce | samba |        │
│ segment | sendgrid | sftp | singularity | slack | smtp | snowflake | sqlite  │
│ | ssh | tableau | tabular | telegram | trino | vertica | yandex |            │
│ zendesk]...': 'apache-flink' is not one of 'all-providers',                  │
│ 'providers-index', 'apache-airflow', 'docker-stack', 'helm-chart',           │
│ 'airbyte', 'alibaba', 'amazon', 'apache.beam', 'apache.cassandra',           │
│ 'apache.drill', 'apache.druid', 'apache.flink', 'apache.hdfs',               │
│ 'apache.hive', 'apache.impala', 'apache.kafka', 'apache.kylin',              │
│ 'apache.livy', 'apache.pig', 'apache.pinot', 'apache.spark', 'apache.sqoop', │
│ 'apprise', 'arangodb', 'asana', 'atlassian.jira', 'celery', 'cloudant',      │
│ 'cncf.kubernetes', 'common.sql', 'daskexecutor', 'databricks', 'datadog',    │
│ 'dbt.cloud', 'dingding', 'discord', 'docker', 'elasticsearch', 'exasol',     │
│ 'facebook', 'ftp', 'github', 'google', 'grpc', 'hashicorp', 'http', 'imap',  │
│ 'influxdb', 'jdbc', 'jenkins', 'microsoft.azure', 'microsoft.mssql',         │
│ 'microsoft.psrp', 'microsoft.winrm', 'mongo', 'mysql', 'neo4j', 'odbc',      │
│ 'openfaas', 'openlineage', 'opensearch', 'opsgenie', 'oracle', 'pagerduty',  │
│ 'papermill', 'plexus', 'postgres', 'presto', 'redis', 'salesforce', 'samba', │
│ 'segment', 'sendgrid', 'sftp', 'singularity', 'slack', 'smtp', 'snowflake',  │
│ 'sqlite', 'ssh', 'tableau', 'tabular', 'telegram', 'trino', 'vertica',       │
│ 'yandex', 'zendesk'.                                                         │
╰──────────────────────────────────────────────────────────────────────────────╯

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@Taragolis Taragolis added the type:misc/internal Changelog: Misc changes that should appear in change log label Oct 20, 2023
@potiuk
Copy link
Member

potiuk commented Oct 20, 2023

Right 🤦

@Taragolis
Copy link
Contributor Author

This case just not happen in main or during full tests that is why we miss that

@potiuk
Copy link
Member

potiuk commented Oct 20, 2023

This case just not happen in main or during full tests that is why we miss that

Yeah. Who tests the tests.

@potiuk
Copy link
Member

potiuk commented Oct 20, 2023

Enough tests succeeded to merge it.

@potiuk potiuk merged commit fdc443d into apache:main Oct 20, 2023
potiuk added a commit to potiuk/airflow that referenced this pull request Oct 21, 2023
The apache#35069 and apache#35087 that switched from `--package-filter` to
shorthand package names in CI did not handle all the cases - special
package names such as "apache-airflow" or "helm-chart" were treated
also as providers and the --package-filter constructed was wrong.

This caused a few PRs that got "selective" documentation build (only
when they changed provider + some airflow docs) would fail.
potiuk added a commit that referenced this pull request Oct 21, 2023
The #35069 and #35087 that switched from `--package-filter` to
shorthand package names in CI did not handle all the cases - special
package names such as "apache-airflow" or "helm-chart" were treated
also as providers and the --package-filter constructed was wrong.

This caused a few PRs that got "selective" documentation build (only
when they changed provider + some airflow docs) would fail.
@Taragolis Taragolis deleted the fix-docs-list-as-string branch October 23, 2023 09:52
potiuk pushed a commit that referenced this pull request Oct 29, 2023
* Fix CI's docs filter generation

* Fix test cases

(cherry picked from commit fdc443d)
potiuk added a commit that referenced this pull request Oct 29, 2023
The #35069 and #35087 that switched from `--package-filter` to
shorthand package names in CI did not handle all the cases - special
package names such as "apache-airflow" or "helm-chart" were treated
also as providers and the --package-filter constructed was wrong.

This caused a few PRs that got "selective" documentation build (only
when they changed provider + some airflow docs) would fail.

(cherry picked from commit a562174)
@potiuk potiuk added this to the Airflow 2.7.3 milestone Oct 29, 2023
@potiuk potiuk added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Oct 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:dev-tools changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) type:misc/internal Changelog: Misc changes that should appear in change log
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants