Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable recompilation when no avro schemas are in project #172

Merged
merged 7 commits into from
Dec 15, 2023

Conversation

kellen
Copy link
Contributor

@kellen kellen commented Nov 16, 2023

Currently a project without any avro schemas, but with a avroSpecificRecords configuration will not recompile those schemas.

If avroSpecificRecords contains records, this would always recompile those schemas.

@RustedBones
Copy link
Collaborator

I don't see why we need this:

  • if any avro file is changed, we recompile the avroSpecificRecords
  • if we change the avroSpecificRecords, we need to call reload, that invalidates all cache.

I added a test in #173 to illustrate this. The sbt output after the test step is

[info] [info] 9 examples, 0 failure, 0 error
[info] [info] Passed: Total 9, Failed 0, Errors 0, Passed 9
[info] [success] Total time: 8 s, completed Dec 4, 2023, 11:55:37 AM
[info] [info] Avro compiler 1.11.3 using stringType=String
[info] [info] Compiling Avro schemas class org.apache.avro.specific.TestRecordWithLogicalTypes
[info] [info] Compiling Avro IDL /tmp/sbt_5928dcbf/settings/src/main/avro_source/avdl.avdl
[info] [info] Compiling Avro schemas /tmp/sbt_5928dcbf/settings/src/main/avro_source/avsc.avsc
[info] [info] Compiling Avro protocol /tmp/sbt_5928dcbf/settings/src/main/avro_source/avpr.avpr
[info] [success] Total time: 0 s, completed Dec 4, 2023, 11:55:37 AM
[info] [success] Total time: 0 s, completed Dec 4, 2023, 11:55:37 AM
[info] [info] Avro compiler 1.11.3 using stringType=String
[info] [info] Compiling Avro schemas class org.apache.avro.specific.TestRecordWithLogicalTypes
[info] [info] Compiling Avro IDL /tmp/sbt_5928dcbf/settings/src/main/avro_source/avdl.avdl
[info] [info] Compiling Avro schemas /tmp/sbt_5928dcbf/settings/src/main/avro_source/avsc.avsc
[info] [info] Compiling Avro protocol /tmp/sbt_5928dcbf/settings/src/main/avro_source/avpr.avpr
[info] [info] compiling 6 Java sources to /tmp/sbt_5928dcbf/settings/target/scala-2.13/classes ...
[info] [info] done compiling
[info] [success] Total time: 1 s, completed Dec 4, 2023, 11:55:38 AM

We see the record being re-compiled.

@kellen kellen mentioned this pull request Dec 14, 2023
records, outDir, out.log, strType, fieldVis, enbDecimal, useNs,
optionalGetters, createSetters, builder
)
(outDir ** JavaFileFilter).get.toSet
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, we do not want to recompile those every time since this will impact all files including the model.
I think we can integrate that in the caching mechanism so it is compiled only once

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, my patience and sbt knowledge was insufficient when writing this.

@RustedBones RustedBones merged commit 4e135d4 into sbt:main Dec 15, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants