-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Elasticsearch feature ✨💥 (Lombiq Technologies: OSOE-83) #11052
Merged
Merged
Changes from 250 commits
Commits
Show all changes
314 commits
Select commit
Hold shift + click to select a range
da8d3e1
Fix Unit Tests
Skrypt b3be5eb
Set Elastic document id's with our ContentItemIds
Skrypt 4003136
Use BulkAsync instead of Bulk
Skrypt 4a81792
Add connection types
Skrypt 844ec82
Add connection options to appsettings.json file
Skrypt cd5a5f1
Enable Exception handling
Skrypt 032da9d
Simplify startup configuration code
Skrypt 355a3f2
Update appsettings example
Skrypt 892ed51
Fix delete issue when a document is not stored at least once.
Skrypt acdfe31
Fix ElasticSearch.DeleteDocuementsAsync
Skrypt 43d4b71
Add "force delete" an index configuration
Skrypt eaa3bce
Change how we do special indexing
Skrypt 17454fe
Add options to RawQuery
Skrypt 2b05bed
WIP _stored index configuration for ES and Lucene
Skrypt eac4dce
Display mappings for reference
Skrypt 65d45fb
WIP _source and/or Fields data fetching/displaying
Skrypt 495750a
Renaming
Skrypt 205f832
simplification
Skrypt 92b640f
wip
Skrypt a08668d
Fix create link
Skrypt 59e23a1
wip
Skrypt 2e21941
wip
Skrypt 631315b
format
Skrypt fe3af2b
revert
Skrypt f5f1759
cleanup
Skrypt 29484de
wip
Skrypt f855089
revert and cleanup
Skrypt cfd002a
revert Lucene
Skrypt ab6820f
Breaking changes documentation
Skrypt 037c86f
Add CloudId example in appsettings.json
Skrypt 07eb637
Renaming Elastic to Elasticsearch
Skrypt 6b3cf70
wip
Skrypt 8df205d
Fix templateoptions for ElasticSearch module
Skrypt af79b84
Cleanup
Skrypt 86d599a
fix NRE
Skrypt b261b0c
cleanup
Skrypt 9f38031
move interfaces
Skrypt 5581e66
documentation
Skrypt a032741
doc
Skrypt 82ce814
cleanup
Skrypt 96a823f
doc
Skrypt 1b96fb6
doc
Skrypt 5d4e96b
Move back to Nest 7.13.2 for OpenSearch compatibility
Skrypt 839bcaa
using latest Nest version
Skrypt 5714b2e
cleanup
Skrypt ea99d65
cleanup
Skrypt 64cc051
Cleanup
Skrypt e532db2
settings
Skrypt 7f221a3
cleanup
Skrypt c8833c0
better fix
Skrypt 2a103e0
Add keyword constant
Skrypt f92094f
Merge branch 'main' into skrypt/elasticsearch
Skrypt 281af68
doc
Skrypt a82a010
hints
Skrypt 07d9e1f
Fix Liquid named Queries
Skrypt b537bdd
Enhancement
Skrypt 7d21b70
Simplifying ElasticSearch Queries
Skrypt 8733334
cleanup
Skrypt 2c94eb0
don't store keyword
Skrypt 3e0818a
wip
Skrypt d1eab52
Cleanup + comments
Skrypt e597a5f
Comments
Skrypt 10e51e2
Comments
Skrypt b9648b6
Comments
Skrypt 620cb3a
Comments
Skrypt bce20bf
Comments
Skrypt ef753bb
Comments
Skrypt 801f9eb
cleanup
Skrypt 2933b7b
Add Analyzers settings on CreateIndex
Skrypt bc5e8fa
cleanup
Skrypt c88714a
Cleanup
Skrypt 19ebf7a
Fix deployment and naming
Skrypt 6f46f26
test
Skrypt 1442a58
rename
Skrypt c66a5db
rename
Skrypt 7f82bd8
rename
Skrypt 613cf31
cleanup/reorganization
Skrypt 9ab4509
cleanup
Skrypt e8d01d4
folder name
Skrypt 05e69c6
startup
Skrypt 5e375cb
singleton
Skrypt e4a92fe
Comments
Skrypt e20dd0c
Comments
Skrypt 4bcd32d
cleanup
Skrypt ed7b36b
cleanup
Skrypt 1a996db
comments
Skrypt f398d09
Add indexing backgroundtask
Skrypt affe417
cleanup
Skrypt 785b1fd
Remove ElasticQueryContext
Skrypt afbb856
Add a way to retrieve the current tenant name in the RecipeExecutor
Skrypt 00c473e
doc
Skrypt a460248
fix separator
Skrypt 1eb86c3
Secure indices per tenant
Skrypt c32e11a
Last Task Id
Skrypt fbbe496
comments
Skrypt cd8210d
Delete all documents in an index in one request
Skrypt 17d09a7
Insert back DeleteDocumentsAsync
Skrypt f664215
Add ElasticIndexInitializerService
Skrypt 2f11297
comment
Skrypt e5a626e
fix startup log messages
Skrypt b258dbd
We test the Elasticsearch connection on startup
Skrypt 0c0afae
Fix scoped service issue
Skrypt d547a95
Startup adjustments
Skrypt fd97b12
Proper index name based on ES specs
Skrypt 0746d71
fix
Skrypt ec72bd7
simplification
Skrypt aa33270
comment
Skrypt 8fe52f6
cleanup
Skrypt d3776e0
Update src/docs/reference/modules/Elasticsearch/README.md
Skrypt 1c556d8
Update src/docs/reference/modules/Elasticsearch/README.md
Skrypt ade06f3
Update src/OrchardCore.Modules/OrchardCore.Search.Elasticsearch/Views…
Skrypt 719da79
Update src/OrchardCore.Modules/OrchardCore.Search/Views/Search-Form.c…
Skrypt 1fcb13a
Update src/docs/reference/modules/Elasticsearch/README.md
Skrypt a83bdd4
Update src/OrchardCore.Modules/OrchardCore.Search.Elasticsearch/Start…
Skrypt 6df325f
Update src/OrchardCore/OrchardCore.Search.Elasticsearch.Core/Models/E…
Skrypt e0b42c8
Update src/OrchardCore.Cms.Web/appsettings.json
Skrypt 0ecf735
Fix permissions names
Skrypt f9f8133
Merge branch 'skrypt/elasticsearch' of https://github.com/OrchardCMS/…
Skrypt efa80af
update docker-compose.yml
Skrypt 5953558
comments
Skrypt 6b61dd7
remove unecessary comments
Skrypt 9a2e3e3
Update src/docs/reference/modules/Elasticsearch/README.md
Skrypt cfc0216
Update src/docs/reference/modules/Elasticsearch/README.md
Skrypt 878e3a8
comments
Skrypt e41b669
optimism is good
Skrypt a5a1a83
1.5
Skrypt 3375da9
1.5 breaking changes
Skrypt 4bc7120
doc
Skrypt dd1f23f
doc
Skrypt 4dab8c4
doc
Skrypt 1f7fd2d
doc
Skrypt 5b18a92
doc WSL2
Skrypt d6cac9d
doc
Skrypt 246a4ed
doc
Skrypt 0e461d5
doc
Skrypt 80dcba9
doc
Skrypt 986b684
doc
Skrypt bd24371
doc
Skrypt 36fcc24
doc
Skrypt ee5b588
doc
Skrypt 9db8a13
doc
Skrypt 87ea60f
Update src/docs/reference/modules/Elasticsearch/README.md
Skrypt f5a4025
Index name doc
Skrypt 41ede3a
Merge branch 'skrypt/elasticsearch' of https://github.com/OrchardCMS/…
Skrypt 522b212
enable OrchardCore.Search feature in recipes
Skrypt 283a12f
removing in the meantime
Skrypt ae09375
Better Lucene indexing for Text.
Skrypt 2174fa4
Fix unit test
Skrypt 575ba20
Lucene indexing settings
Skrypt 44fc512
DocumentIndexOptions
Skrypt 592caad
Comments
Skrypt 9f401dc
Fix enum flag values so that they don't overlap
Skrypt c42d1c3
same
Skrypt c3407a3
small change
Skrypt aa203e8
Lucene source data
Skrypt e1e262a
small change
Skrypt cae7ce2
Comments
Skrypt fd9ff11
bg task lock
Skrypt cfceb16
Fix SELECT N+1 issue
Skrypt 21236a0
perf adjustments
Skrypt 5f916dd
Add SQL migration
Skrypt ae507a1
5 min LockExpiration on background task
Skrypt 170e30d
I break unit tests
Skrypt 4e35047
deployment for search settings
Skrypt 5b13233
doc
Skrypt 5434755
doc
Skrypt 50030bd
doc
Skrypt b4d98da
Fix unit test
Skrypt a49bc0e
Move dynamic routing to startup
Skrypt a44d974
cleanup
Skrypt 5f8bf68
Use DynamicRouteValueTransformer instead
Skrypt edfbaf8
cleanup
Skrypt 913c899
Store AreaName in SearchSettings
Skrypt e2bdf47
Fix Elasticsearch contentpicker index selector
Skrypt 1c22741
Adding custom Elasticsearch mappings
Skrypt b7c598c
doc
Skrypt 13c7d4a
comments
Skrypt a5f1358
doc
Skrypt 2dc0d9e
Content.ContentItem.DisplayText.Analyzed not stored
Skrypt 8e5696e
cleanup
Skrypt 740ba0a
optimization
Skrypt 877405a
code optimization
Skrypt d3e2a24
store order
Skrypt cd475a9
fix unit test
Skrypt c1725aa
doc
Skrypt 31af06a
doc
Skrypt 7155580
doc
Skrypt 8c5346d
Fix taxonomy indexing
Skrypt 82a73d2
release doc
Skrypt 9eaecfe
comment
Skrypt b78f6d1
doc
Skrypt 6714582
More manual mappings
Skrypt f52ad4f
cleanup + doc
Skrypt 8a2ee3a
doc
Skrypt 34ea3c1
full-text mapping
Skrypt 9228997
cleanup
Skrypt 4f6171d
IContentIndexSettings
Skrypt 31964cc
cleanup
Skrypt 97f9bdb
fix NRE on setup
Skrypt b373fcf
Use current SearchProvider if only one is enabled
Skrypt 8beda06
Fix recipe Query for ES
Skrypt 13f7f4e
hint
Skrypt 67b75b3
Fix NRE with background task
Skrypt b507f91
optimization
Skrypt 84a8156
Merge branch 'main' into skrypt/elasticsearch
Skrypt fbe1bd3
Merge branch 'main' into skrypt/elasticsearch
Skrypt d73cfc2
Remove test file
Skrypt 3178d3d
Update src/OrchardCore.Modules/OrchardCore.Search/Deployment/SearchSe…
Skrypt 62b2e44
Update src/OrchardCore.Modules/OrchardCore.Search.Lucene/Migrations.cs
Skrypt 756c4c8
Update src/OrchardCore.Modules/OrchardCore.Search.Lucene/Migrations.cs
Skrypt 4ab1cd9
Update src/docs/reference/modules/Elasticsearch/README.md
Skrypt a2af577
doc
Skrypt 616cb1f
remove commented code
Skrypt 9c37687
Fix FullTextContentIndexHandler
Skrypt 1a93824
Add Lucene and Elasticsearch migration
Skrypt d344c83
doc
Skrypt b247834
Merge branch 'main' into skrypt/elasticsearch
Skrypt 6724e58
Fix deployment step
Skrypt d312466
Remove Elasticsearch migration
Skrypt b0f9532
cleanup
Skrypt 857ede2
Add ElasticSettingsStep
Skrypt dea0088
doc
Skrypt 53c61dc
doc
Skrypt 1d218d2
Add Rebuild and Reset indices recipe step
Skrypt b89eb92
Comments
Skrypt 289f799
Merge branch 'main' into skrypt/elasticsearch
Skrypt 534ca03
Merge migration step
Skrypt 7869065
Fix recipe steps
Skrypt fccd7aa
optimization
Skrypt a9afc59
doc
Skrypt 20e607f
Fix copy/paste mistake
Skrypt 6d15fad
Fix ResetIndexAsync
Skrypt 28d63ed
Merge branch 'main' into skrypt/elasticsearch
Skrypt 8712a5c
Fix GraphQL provider
Skrypt ee04028
Fix reset and rebuild recipe IncludeAll
Skrypt 77a3fbb
doc
Skrypt b5f0300
Revert changes on AdminController
Skrypt 764ed2d
Merge branch 'main' into skrypt/elasticsearch
Skrypt 815fde6
Merge branch 'main' into skrypt/elasticsearch
Skrypt 46e76cc
Fix build after merge
Skrypt bb849bc
Fix TablePrefix
Skrypt df26cc8
Fix non awaited delegate
Skrypt 6995ddf
Merge branch 'main' into skrypt/elasticsearch
Skrypt b902e8d
comment out appsettings
Skrypt ef7e8e9
removing unnecessary method param
Skrypt 063d3f0
add missing comma
Skrypt 094799c
Update src/OrchardCore.Modules/OrchardCore.Search.Elasticsearch/Views…
Skrypt d5ac9e6
Fix Elasticsearch page integration
Skrypt c45796a
Fix DeploymentSource and cleanup
Skrypt c87c73a
Rename to Elasticsearch Settings and
Skrypt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
src/OrchardCore.Modules/OrchardCore.ContentFields/Fields/TextField.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
89 changes: 89 additions & 0 deletions
89
src/OrchardCore.Modules/OrchardCore.ContentTypes/Recipes/LuceneRecipeEventHandler.cs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
using System; | ||
using System.Threading.Tasks; | ||
using Newtonsoft.Json.Linq; | ||
using OrchardCore.ContentManagement.Metadata.Records; | ||
using OrchardCore.Recipes.Events; | ||
using OrchardCore.Recipes.Models; | ||
|
||
namespace OrchardCore.ContentTypes | ||
{ | ||
/// <summary> | ||
/// This handler provides backward compatibility with ContentIndexSettings that have been migrated to LuceneContentIndexSettings. | ||
/// </summary> | ||
public class LuceneRecipeEventHandler : IRecipeEventHandler | ||
{ | ||
public RecipeExecutionContext Context { get; private set; } | ||
|
||
public Task RecipeStepExecutedAsync(RecipeExecutionContext context) => Task.CompletedTask; | ||
|
||
public Task ExecutionFailedAsync(string executionId, RecipeDescriptor descriptor) => Task.CompletedTask; | ||
|
||
public Task RecipeExecutedAsync(string executionId, RecipeDescriptor descriptor) => Task.CompletedTask; | ||
|
||
public Task RecipeExecutingAsync(string executionId, RecipeDescriptor descriptor) => Task.CompletedTask; | ||
|
||
public Task RecipeStepExecutingAsync(RecipeExecutionContext context) | ||
{ | ||
if (context.Name == "ReplaceContentDefinition" || context.Name == "ContentDefinition") | ||
{ | ||
var step = context.Step.ToObject<ContentDefinitionStepModel>(); | ||
|
||
foreach (var contentType in step.ContentTypes) | ||
{ | ||
foreach (var partDefinition in contentType.ContentTypePartDefinitionRecords) | ||
{ | ||
if (partDefinition.Settings != null) | ||
{ | ||
if (partDefinition.Settings.TryGetValue("ContentIndexSettings", out var existingPartSettings) && | ||
!partDefinition.Settings.ContainsKey("LuceneContentIndexSettings")) | ||
{ | ||
partDefinition.Settings.Add(new JProperty("LuceneContentIndexSettings", existingPartSettings)); | ||
} | ||
|
||
partDefinition.Settings.Remove("ContentIndexSettings"); | ||
} | ||
} | ||
} | ||
|
||
foreach (var partDefinition in step.ContentParts) | ||
{ | ||
if (partDefinition.Settings != null) | ||
{ | ||
if (partDefinition.Settings.TryGetValue("ContentIndexSettings", out var existingPartSettings) && | ||
!partDefinition.Settings.ContainsKey("LuceneContentIndexSettings")) | ||
{ | ||
partDefinition.Settings.Add(new JProperty("LuceneContentIndexSettings", existingPartSettings)); | ||
} | ||
|
||
partDefinition.Settings.Remove("ContentIndexSettings"); | ||
|
||
foreach (var fieldDefinition in partDefinition.ContentPartFieldDefinitionRecords) | ||
{ | ||
if (fieldDefinition.Settings != null) | ||
{ | ||
if (fieldDefinition.Settings.TryGetValue("ContentIndexSettings", out var existingFieldSettings) && | ||
!fieldDefinition.Settings.ContainsKey("LuceneContentIndexSettings")) | ||
{ | ||
fieldDefinition.Settings.Add(new JProperty("LuceneContentIndexSettings", existingFieldSettings)); | ||
} | ||
|
||
fieldDefinition.Settings.Remove("ContentIndexSettings"); | ||
} | ||
} | ||
} | ||
} | ||
|
||
context.Step = JObject.FromObject(step); | ||
} | ||
|
||
return Task.CompletedTask; | ||
} | ||
|
||
private class ContentDefinitionStepModel | ||
{ | ||
public string Name { get; set; } | ||
public ContentTypeDefinitionRecord[] ContentTypes { get; set; } = Array.Empty<ContentTypeDefinitionRecord>(); | ||
public ContentPartDefinitionRecord[] ContentParts { get; set; } = Array.Empty<ContentPartDefinitionRecord>(); | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comma is still missing here.