Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow describing an index from a given Type. #525

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9cbde32
Use System.Text.Json as the default serializer (#504)
sebastienros Dec 22, 2023
263f1d4
remake DescribeContext<>.For
hyzx86 Dec 28, 2023
8fa1f8e
fix UnitTest
hyzx86 Dec 28, 2023
cc9cc7e
Update README.md
hyzx86 Dec 29, 2023
c6c65cd
Update README.md
hyzx86 Dec 29, 2023
d4645fa
Avoid InvalidCastException
Dec 29, 2023
2fd4d5b
fix type cache clear
Dec 29, 2023
23f9c6b
Update README.md
hyzx86 Dec 29, 2023
25f4198
clear code style
Dec 29, 2023
e373ca0
Merge branch 'dynamicIndexType' of https://github.com/hyzx86/yessql i…
Dec 29, 2023
482ed90
Update PropertyIndex.cs
hyzx86 Dec 29, 2023
f850380
Simplify and Cleanup
MikeAlhayek Dec 29, 2023
86d4302
Moving the test. Test should fail MySQL for verification.
MikeAlhayek Dec 29, 2023
74936c1
Fix MySQL error
MikeAlhayek Dec 29, 2023
2c69da9
Last tewaks
MikeAlhayek Dec 29, 2023
144d363
If the same name type is already cached, it should be removed from th…
hyzx86 Dec 30, 2023
fbabfc5
add Dynamic Type UnitTest
hyzx86 Dec 31, 2023
1d5e8a6
validate index type instance
hyzx86 Dec 31, 2023
e83d236
remove error check
hyzx86 Dec 31, 2023
5371cd1
Emulating the usage scenario in OrchardCore, the store may be initial…
hyzx86 Dec 31, 2023
5df8f36
update unit test
hyzx86 Dec 31, 2023
df795ab
add IndexTypeCacheProvider
hyzx86 Jan 1, 2024
f74cc3e
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
355b69a
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
6122ff4
Update test/YesSql.Tests/Indexes/PropertyIndex.cs
hyzx86 Jan 2, 2024
135d110
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
5ee8f86
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
c500f20
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
b074891
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
10cdf25
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
7c18fb9
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
a52b5a4
Update test/YesSql.Tests/CoreTests.cs
hyzx86 Jan 2, 2024
c213254
Update src/YesSql.Abstractions/Indexes/DescribeContext.cs
hyzx86 Jan 2, 2024
01d217f
fix code format
hyzx86 Jan 2, 2024
e458a98
use vitrual methods instead Interface
hyzx86 Jan 2, 2024
554794d
use session.RegisterIndexes to instead store.RegisterIndexes<Property…
hyzx86 Jan 2, 2024
b945de0
Update test/YesSql.Tests/CoreTests.cs
hyzx86 Jan 2, 2024
be8d8f3
Update src/YesSql.Core/Commands/IndexCommand.cs
hyzx86 Jan 2, 2024
ebc0b32
Update src/YesSql.Abstractions/Indexes/IndexTypeCacheProvider.cs
hyzx86 Jan 2, 2024
bc1d2e7
Update src/YesSql.Abstractions/Indexes/IndexTypeCacheProvider.cs
hyzx86 Jan 2, 2024
72002c8
Update src/YesSql.Abstractions/Indexes/DescribeContext.cs
hyzx86 Jan 2, 2024
16acf5e
Update src/YesSql.Abstractions/IStore.cs
hyzx86 Jan 2, 2024
212a333
add UpdateTypeCache Sample
hyzx86 Jan 2, 2024
c40413d
Merge branch 'dynamicIndexType' of https://github.com/hyzx86/yessql i…
hyzx86 Jan 2, 2024
b0bf2ef
update default index cache
Jan 3, 2024
d5429ea
Restoring the test code
Jan 3, 2024
cd0928a
rename
Jan 3, 2024
784b81c
update unit test
Jan 3, 2024
88d1a56
Merge branch 'main' into release/5.0
MikeAlhayek Jan 26, 2024
ac2d719
merg from 5.x
hyzx86 Feb 26, 2024
8e31c26
Merge branch 'dynamicIndexType' of https://github.com/hyzx86/yessql i…
hyzx86 Feb 26, 2024
2b836e8
Merge remote-tracking branch 'origin/main' into dynamicIndexType
hyzx86 Apr 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 0 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,6 @@ A .NET document database interface for relational databases, because in SQL we (
[![NuGet](https://img.shields.io/nuget/v/YesSql.svg)](https://www.nuget.org/packages/YesSql)
[![MyGet](https://img.shields.io/myget/yessql/vpre/yessql.svg?label=MyGet)](https://www.myget.org/feed/yessql/package/nuget/yessql)

Dynamic index type support
-------------------

```c#
public class PropertyDynamicIndexProvider : IndexProvider<Property>
{
public override void Describe(DescribeContext<Property> context)
{
var propertyType = typeof(PropertyIndex);
context
.For(propertyType) //Specify a type directly
.Map(property => new PropertyIndex
{
Name = property.Name,
ForRent = property.ForRent,
IsOccupied = property.IsOccupied,
Location = property.Location
});
}
}
```

How does it work ?
-------------------

Expand Down
19 changes: 9 additions & 10 deletions src/YesSql.Abstractions/Indexes/DescribeContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ namespace YesSql.Indexes
{
public class DescribeContext<T> : IDescriptor
{
private readonly Dictionary<Type, List<IDescribeFor>> _describes = new Dictionary<Type, List<IDescribeFor>>();
private readonly Dictionary<Type, List<IDescribeFor>> _describes = [];

public IEnumerable<IndexDescriptor> Describe(params Type[] types)
{
Expand All @@ -32,30 +32,29 @@ public IMapFor<T, IIndex> For(Type indexType)

public IMapFor<T, TIndex> For<TIndex>() where TIndex : IIndex
{
return For<TIndex, object>();
return For<TIndex, object>(typeof(TIndex));
}

public IMapFor<T, TIndex> For<TIndex, TKey>() where TIndex : IIndex
{
return For<TIndex, object>(null);
return For<TIndex, object>(typeof(TIndex));
}
MikeAlhayek marked this conversation as resolved.
Show resolved Hide resolved

public IMapFor<T, TIndex> For<TIndex, TKey>(Type indexType) where TIndex : IIndex
{
ArgumentNullException.ThrowIfNull(indexType, nameof(indexType));

List<IDescribeFor> descriptors;

if (!_describes.TryGetValue(typeof(T), out descriptors))
{
descriptors = _describes[typeof(T)] = new List<IDescribeFor>();
descriptors = _describes[typeof(T)] = [];
}

var describeFor = new IndexDescriptor<T, TIndex, TKey>();

// if indexType is null , use default value :typeof(TIndex)
if (indexType != null)
var describeFor = new IndexDescriptor<T, TIndex, TKey>()
{
describeFor.IndexType = indexType;
}
IndexType = indexType,
};

descriptors.Add(describeFor);

Expand Down
24 changes: 9 additions & 15 deletions src/YesSql.Core/Commands/IndexCommand.cs
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,16 @@ namespace YesSql.Commands
public abstract class IndexCommand : IIndexCommand
{
protected const string ParameterSuffix = "_$$$";
private const string _separator = ", ";

protected readonly IStore _store;

private static readonly ConcurrentDictionary<PropertyInfo, PropertyInfoAccessor> PropertyAccessors = new();
private static readonly ConcurrentDictionary<Type, PropertyInfo[]> TypeProperties = new();
private static readonly ConcurrentDictionary<string, PropertyInfo[]> TypeProperties = new();
private static readonly ConcurrentDictionary<CompoundKey, string> InsertsList = new();
private static readonly ConcurrentDictionary<CompoundKey, string> UpdatesList = new();

protected static PropertyInfo[] KeysProperties = new[] { typeof(IIndex).GetProperty("Id") };
protected static PropertyInfo[] KeysProperties = [typeof(IIndex).GetProperty("Id")];

public abstract int ExecutionOrder { get; }

Expand Down Expand Up @@ -67,20 +68,13 @@ protected static void GetProperties(DbCommand command, object item, string suffi

protected static PropertyInfo[] TypePropertiesCache(Type type)
{
if (TypeProperties.TryGetValue(type, out var pis))
if (TypeProperties.TryGetValue(type.FullName, out var pis))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is mainly to avoid a scenario
Because dynamically typed indexes are allowed in the code, there is no guarantee that the type will not change during use,
For example, when I save the type, it has 5 fields, and then I add a new field, then I need to regenerate the type index.
Two types of inconsistent errors occur

2023-12-29 10:42:33.4084|SalesProtalDev|00-91aa5d5c1f96b614b2ebf3f2e6dfdbd5-5d2a9c17352de5d4-00||OrchardCore.ContentManagement.DefaultContentManager|ERROR|IContentHandler thrown from OrchardCore.Contents.AuditTrail.Handlers.AuditTrailContentHandler by InvalidCastException System.InvalidCastException: [A]EasyOC.DynamicIndex.AmisSchemaDIndex cannot be cast to [B]EasyOC.DynamicIndex.AmisSchemaDIndex. Type A originates from 'DynamicTypesAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array. Type B originates from 'DynamicTypesAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array.
at YesSql.Serialization.PropertyInfoAccessor.Invoker`2.Invoke(Object target)
at YesSql.Commands.IndexCommand.GetProperties(DbCommand command, Object item, String suffix, ISqlDialect dialect)
at YesSql.Commands.CreateIndexCommand.AddToBatch(ISqlDialect dialect, List`1 queries, DbCommand batchCommand, List`1 actions, Int32 index)
at YesSql.Session.BatchCommands()
at YesSql.Session.FlushAsync()
at YesSql.Session.FlushAsync()
at YesSql.Services.DefaultQuery.CountAsync()
at YesSql.Services.DefaultQuery.QueryIndex`1.YesSql.IQueryIndex<T>.CountAsync()
at OrchardCore.Contents.AuditTrail.Handlers.AuditTrailContentHandler.RecordAuditTrailEventAsync(String name, IContent content)
at OrchardCore.Modules.InvokeExtensions.InvokeAsync[TEvents,T1](IEnumerable`1 events, Func`3 dispatch, T1 arg1, ILogger logger) at YesSql.Serialization.PropertyInfoAccessor.Invoker`2.Invoke(Object target)
at YesSql.Commands.IndexCommand.GetProperties(DbCommand command, Object item, String suffix, ISqlDialect dialect)
at YesSql.Commands.CreateIndexCommand.AddToBatch(ISqlDialect dialect, List`1 queries, DbCommand batchCommand, List`1 actions, Int32 index)
at YesSql.Session.BatchCommands()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the index type may change while the program is running.

Just like this:

thrown from OrchardCore.Contents.AuditTrail.Handlers.AuditTrailContentHandler by InvalidCastException System.InvalidCastException: [A]EasyOC.DynamicIndex.AmisSchemaDIndex cannot be cast to [B]EasyOC.DynamicIndex.AmisSchemaDIndex. Type A originates from 'DynamicTypesAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array. Type B originates from 'DynamicTypesAssembly, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' in the context 'Default' in a byte array.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be no way to reproduce this scenario in a unit test, or to reproduce this scenario would require a complicated process

{
return pis;
}

// Clean up cache entries with the same name and different type
var exists = TypeProperties.Keys.FirstOrDefault(x => x.FullName == type.FullName);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To ensure that only meaningful types are cached, cached type with the same name will cleared

if (exists != null)
{
TypeProperties.Remove(exists, out _);
}

var properties = type.GetProperties().Where(IsWriteable).ToArray();
TypeProperties[type] = properties;
TypeProperties[type.FullName] = properties;
return properties;
}

Expand Down Expand Up @@ -109,7 +103,7 @@ protected string Inserts(Type type, ISqlDialect dialect)
sbColumnList.Append(dialect.QuoteForColumnName(property.Name));
if (i < allProperties.Length - 1)
{
sbColumnList.Append(", ");
sbColumnList.Append(_separator);
}
}

Expand All @@ -120,14 +114,14 @@ protected string Inserts(Type type, ISqlDialect dialect)
sbParameterList.Append("@").Append(property.Name).Append(ParameterSuffix);
if (i < allProperties.Length - 1)
{
sbParameterList.Append(", ");
sbParameterList.Append(_separator);
}
}

if (typeof(MapIndex).IsAssignableFrom(type))
{
// We can set the document id
sbColumnList.Append(", ").Append(dialect.QuoteForColumnName("DocumentId"));
sbColumnList.Append(_separator).Append(dialect.QuoteForColumnName("DocumentId"));
sbParameterList.Append(", @DocumentId").Append(ParameterSuffix);
}

Expand Down Expand Up @@ -171,7 +165,7 @@ protected string Updates(Type type, ISqlDialect dialect)
values.Append(dialect.QuoteForColumnName(property.Name) + " = @" + property.Name + ParameterSuffix);
if (i < allProperties.Length - 1)
{
values.Append(", ");
values.Append(_separator);
}
}

Expand Down
3 changes: 1 addition & 2 deletions test/YesSql.Tests/Indexes/PropertyIndex.cs
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,8 @@ public class PropertyDynamicIndexProvider : IndexProvider<Property>
{
public override void Describe(DescribeContext<Property> context)
{
var propertyType = typeof(PropertyIndex);
context
.For(propertyType)
.For(typeof(PropertyIndex))
.Map(property => new PropertyIndex
{
Name = property.Name,
Expand Down