Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow describing an index from a given Type. #525

Closed
wants to merge 52 commits into from
Closed
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9cbde32
Use System.Text.Json as the default serializer (#504)
sebastienros Dec 22, 2023
263f1d4
remake DescribeContext<>.For
hyzx86 Dec 28, 2023
8fa1f8e
fix UnitTest
hyzx86 Dec 28, 2023
cc9cc7e
Update README.md
hyzx86 Dec 29, 2023
c6c65cd
Update README.md
hyzx86 Dec 29, 2023
d4645fa
Avoid InvalidCastException
Dec 29, 2023
2fd4d5b
fix type cache clear
Dec 29, 2023
23f9c6b
Update README.md
hyzx86 Dec 29, 2023
25f4198
clear code style
Dec 29, 2023
e373ca0
Merge branch 'dynamicIndexType' of https://github.com/hyzx86/yessql i…
Dec 29, 2023
482ed90
Update PropertyIndex.cs
hyzx86 Dec 29, 2023
f850380
Simplify and Cleanup
MikeAlhayek Dec 29, 2023
86d4302
Moving the test. Test should fail MySQL for verification.
MikeAlhayek Dec 29, 2023
74936c1
Fix MySQL error
MikeAlhayek Dec 29, 2023
2c69da9
Last tewaks
MikeAlhayek Dec 29, 2023
144d363
If the same name type is already cached, it should be removed from th…
hyzx86 Dec 30, 2023
fbabfc5
add Dynamic Type UnitTest
hyzx86 Dec 31, 2023
1d5e8a6
validate index type instance
hyzx86 Dec 31, 2023
e83d236
remove error check
hyzx86 Dec 31, 2023
5371cd1
Emulating the usage scenario in OrchardCore, the store may be initial…
hyzx86 Dec 31, 2023
5df8f36
update unit test
hyzx86 Dec 31, 2023
df795ab
add IndexTypeCacheProvider
hyzx86 Jan 1, 2024
f74cc3e
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
355b69a
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
6122ff4
Update test/YesSql.Tests/Indexes/PropertyIndex.cs
hyzx86 Jan 2, 2024
135d110
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
5ee8f86
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
c500f20
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
b074891
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
10cdf25
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
7c18fb9
Update test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
hyzx86 Jan 2, 2024
a52b5a4
Update test/YesSql.Tests/CoreTests.cs
hyzx86 Jan 2, 2024
c213254
Update src/YesSql.Abstractions/Indexes/DescribeContext.cs
hyzx86 Jan 2, 2024
01d217f
fix code format
hyzx86 Jan 2, 2024
e458a98
use vitrual methods instead Interface
hyzx86 Jan 2, 2024
554794d
use session.RegisterIndexes to instead store.RegisterIndexes<Property…
hyzx86 Jan 2, 2024
b945de0
Update test/YesSql.Tests/CoreTests.cs
hyzx86 Jan 2, 2024
be8d8f3
Update src/YesSql.Core/Commands/IndexCommand.cs
hyzx86 Jan 2, 2024
ebc0b32
Update src/YesSql.Abstractions/Indexes/IndexTypeCacheProvider.cs
hyzx86 Jan 2, 2024
bc1d2e7
Update src/YesSql.Abstractions/Indexes/IndexTypeCacheProvider.cs
hyzx86 Jan 2, 2024
72002c8
Update src/YesSql.Abstractions/Indexes/DescribeContext.cs
hyzx86 Jan 2, 2024
16acf5e
Update src/YesSql.Abstractions/IStore.cs
hyzx86 Jan 2, 2024
212a333
add UpdateTypeCache Sample
hyzx86 Jan 2, 2024
c40413d
Merge branch 'dynamicIndexType' of https://github.com/hyzx86/yessql i…
hyzx86 Jan 2, 2024
b0bf2ef
update default index cache
Jan 3, 2024
d5429ea
Restoring the test code
Jan 3, 2024
cd0928a
rename
Jan 3, 2024
784b81c
update unit test
Jan 3, 2024
88d1a56
Merge branch 'main' into release/5.0
MikeAlhayek Jan 26, 2024
ac2d719
merg from 5.x
hyzx86 Feb 26, 2024
8e31c26
Merge branch 'dynamicIndexType' of https://github.com/hyzx86/yessql i…
hyzx86 Feb 26, 2024
2b836e8
Merge remote-tracking branch 'origin/main' into dynamicIndexType
hyzx86 Apr 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions src/YesSql.Abstractions/Indexes/DescribeContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ namespace YesSql.Indexes
{
public class DescribeContext<T> : IDescriptor
{
private readonly Dictionary<Type, List<IDescribeFor>> _describes = new Dictionary<Type, List<IDescribeFor>>();
private readonly Dictionary<Type, List<IDescribeFor>> _describes = [];

public IEnumerable<IndexDescriptor> Describe(params Type[] types)
{
Expand All @@ -25,21 +25,37 @@ public IEnumerable<IndexDescriptor> Describe(params Type[] types)
});
}

public IMapFor<T, IIndex> For(Type indexType)
{
return For<IIndex, object>(indexType);
}
MikeAlhayek marked this conversation as resolved.
Show resolved Hide resolved
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved

public IMapFor<T, TIndex> For<TIndex>() where TIndex : IIndex
{
return For<TIndex, object>();
return For<TIndex, object>(typeof(TIndex));
}

public IMapFor<T, TIndex> For<TIndex, TKey>() where TIndex : IIndex
{
return For<TIndex, object>(typeof(TIndex));
}
MikeAlhayek marked this conversation as resolved.
Show resolved Hide resolved
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved

public IMapFor<T, TIndex> For<TIndex, TKey>(Type indexType) where TIndex : IIndex
{
ArgumentNullException.ThrowIfNull(indexType, nameof(indexType));

List<IDescribeFor> descriptors;

if (!_describes.TryGetValue(typeof(T), out descriptors))
{
descriptors = _describes[typeof(T)] = new List<IDescribeFor>();
descriptors = _describes[typeof(T)] = [];
}

var describeFor = new IndexDescriptor<T, TIndex, TKey>();
var describeFor = new IndexDescriptor<T, TIndex, TKey>()
{
IndexType = indexType,
};

descriptors.Add(describeFor);

return describeFor;
Expand Down
3 changes: 2 additions & 1 deletion src/YesSql.Abstractions/Indexes/DescribeFor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,9 @@ public class IndexDescriptor<T, TIndex, TKey> : IDescribeFor, IMapFor<T, TIndex>
private Func<object, bool> _filter;

public PropertyInfo GroupProperty { get; set; }
public Type IndexType { get { return typeof(TIndex); } }

public Type IndexType { get; internal set; } = typeof(TIndex);

public Func<object, bool> Filter => _filter;

public IGroupFor<TIndex> Map(Func<T, IEnumerable<TIndex>> map)
Expand Down
14 changes: 7 additions & 7 deletions src/YesSql.Core/Commands/IndexCommand.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,16 @@ namespace YesSql.Commands
{
public abstract class IndexCommand : IIndexCommand
{
private const string _separator = ", ";
protected const string ParameterSuffix = "_$$$";

protected readonly IStore _store;
protected static PropertyInfo[] KeysProperties = [typeof(IIndex).GetProperty("Id")];

private static readonly ConcurrentDictionary<PropertyInfo, PropertyInfoAccessor> PropertyAccessors = new();
private static readonly ConcurrentDictionary<string, PropertyInfo[]> TypeProperties = new();
private static readonly ConcurrentDictionary<CompoundKey, string> InsertsList = new();
private static readonly ConcurrentDictionary<CompoundKey, string> UpdatesList = new();

protected static PropertyInfo[] KeysProperties = new[] { typeof(IIndex).GetProperty("Id") };
protected readonly IStore _store;

public abstract int ExecutionOrder { get; }

Expand Down Expand Up @@ -102,7 +102,7 @@ protected string Inserts(Type type, ISqlDialect dialect)
sbColumnList.Append(dialect.QuoteForColumnName(property.Name));
if (i < allProperties.Length - 1)
{
sbColumnList.Append(", ");
sbColumnList.Append(_separator);
}
}

Expand All @@ -113,14 +113,14 @@ protected string Inserts(Type type, ISqlDialect dialect)
sbParameterList.Append("@").Append(property.Name).Append(ParameterSuffix);
if (i < allProperties.Length - 1)
{
sbParameterList.Append(", ");
sbParameterList.Append(_separator);
}
}

if (typeof(MapIndex).IsAssignableFrom(type))
{
// We can set the document id
sbColumnList.Append(", ").Append(dialect.QuoteForColumnName("DocumentId"));
sbColumnList.Append(_separator).Append(dialect.QuoteForColumnName("DocumentId"));
sbParameterList.Append(", @DocumentId").Append(ParameterSuffix);
}

Expand Down Expand Up @@ -164,7 +164,7 @@ protected string Updates(Type type, ISqlDialect dialect)
values.Append(dialect.QuoteForColumnName(property.Name) + " = @" + property.Name + ParameterSuffix);
if (i < allProperties.Length - 1)
{
values.Append(", ");
values.Append(_separator);
}
}

Expand Down
86 changes: 86 additions & 0 deletions test/YesSql.Tests/CoreTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6296,6 +6296,92 @@ await session.SaveAsync(new Article
Assert.Equal(10, result);
}

[Fact]
public async Task PopulateIndexUsingGenericType()
{
var _configuration1 = CreateConfiguration();
var store1 = await StoreFactory.CreateAndInitializeAsync(_configuration1);
await using (var connection = store1.Configuration.ConnectionFactory.CreateConnection())
{
await connection.OpenAsync();

await using var transaction = await connection.BeginTransactionAsync(store1.Configuration.IsolationLevel);
var builder = new SchemaBuilder(store1.Configuration, transaction);

await builder
.DropMapIndexTableAsync<PropertyIndex>();

await builder
.CreateMapIndexTableAsync<PropertyIndex>(column => column
.Column<string>(nameof(PropertyIndex.Name), col => col.WithLength(254))
.Column<bool>(nameof(PropertyIndex.ForRent))
.Column<bool>(nameof(PropertyIndex.IsOccupied))
.Column<string>(nameof(PropertyIndex.Location), col => col.WithLength(500))
);

await builder
.AlterTableAsync(nameof(PropertyIndex), table => table
.CreateIndex("IDX_Property", "Name", "ForRent", "IsOccupied"));

await transaction.CommitAsync();
}

store1.RegisterIndexes<PropertyDynamicIndexProvider>();

var typeDef = new DynamicTypeDef()
{
NameSpace = "TestDynamicIndexNameSpace",
ClassName = "PropertyIndex",
Fields = new[] {
new DynamicField { Name="Name",FieldType=typeof(string)},
new DynamicField { Name="ForRent",FieldType=typeof(bool)},
new DynamicField { Name="IsOccupied",FieldType=typeof(bool)},
new DynamicField { Name = "Location", FieldType = typeof(string)}
}
};

var dynamicType = DynamicTypeGeneratorSample.GenType(typeDef);

PropertyDynamicIndexProvider.IndexTypeCache[dynamicType.FullName] = dynamicType;

await using var session = store1.CreateSession();
var property = new Property
{
Name = new string('*', 40),
IsOccupied = true,
ForRent = true,
Location = new string('*', 500)
};

await session.SaveAsync(property);
await session.SaveChangesAsync();
var testProperties = await session.Query<Property, PropertyIndex>(x => x.Id == 1).ListAsync();
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
Assert.NotEmpty(testProperties);

// Emulating the usage scenario in OrchardCore, the store may be initialized multiple times

var _configuration2 = CreateConfiguration();
var store2 = await StoreFactory.CreateAndInitializeAsync(_configuration2);
store2.RegisterIndexes<PropertyDynamicIndexProvider>();

// In production environment, we may change this type at any time
var changedType = DynamicTypeGeneratorSample.GenType(typeDef);

Assert.NotEqual(changedType, dynamicType);

PropertyDynamicIndexProvider.IndexTypeCache[dynamicType.FullName] = changedType;

await using var session2 = store2.CreateSession();
var testPropEntity = testProperties.FirstOrDefault();
await session2.SaveAsync(testPropEntity);
await session2.SaveChangesAsync();
var testProperties2 = await session2.Query<Property, PropertyIndex>().ListAsync();
Assert.NotEmpty(testProperties2);

}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done



hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
#region FilterTests

[Fact]
Expand Down
87 changes: 87 additions & 0 deletions test/YesSql.Tests/Indexes/DynamicTypeGeneratorSample.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
using System;
using System.Collections.Generic;
using System.Reflection;
using System.Reflection.Emit;
using YesSql.Indexes;

namespace YesSql.Tests.Indexes
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put ; at the end of the name space to save indentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
{
public class DynamicTypeGeneratorSample
{
public static Type GenType(DynamicTypeDef dynamicTypeDef)
{
var nameSpace = dynamicTypeDef.NameSpace;
var indexTypeFullName = dynamicTypeDef.NameSpace + "." + dynamicTypeDef.ClassName;
var assemblyName = "DynamicTypesAssembly";
// Create the dynamic assembly
AssemblyBuilder assemblyBuilder = AssemblyBuilder.DefineDynamicAssembly(
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
new AssemblyName(assemblyName), AssemblyBuilderAccess.Run);

// Create a dynamic module in the assembly
ModuleBuilder moduleBuilder = assemblyBuilder.DefineDynamicModule(nameSpace);
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
// Define the type
TypeBuilder typeBuilder = moduleBuilder.DefineType(indexTypeFullName, TypeAttributes.Public);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
TypeBuilder typeBuilder = moduleBuilder.DefineType(indexTypeFullName, TypeAttributes.Public);
var typeBuilder = moduleBuilder.DefineType(indexTypeFullName, TypeAttributes.Public);


typeBuilder.SetParent(typeof(MapIndex));

foreach (var item in dynamicTypeDef.Fields)
{
BuildProperties(typeBuilder, item);
}
// Create the type
Type dynamicType = typeBuilder.CreateType();
// Return the dynamic type
return dynamicType;
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anywhere you see multiple new lines, please reduce the. To one new line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done



Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

private static PropertyBuilder BuildProperties(TypeBuilder typeBuilder, DynamicField item)
{
var propertyName = item.Name;
var propType = item.FieldType;

// Define the property
PropertyBuilder propertyBuilder = typeBuilder.DefineProperty(propertyName, PropertyAttributes.None, propType, null);
FieldBuilder fieldBuilder = typeBuilder.DefineField("_" + propertyName, propType, FieldAttributes.Private);

// 使用 IL API 生成类型属性访问器
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
// Define the getter method
MethodBuilder getterBuilder = typeBuilder.DefineMethod("get_" + propertyName,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
MethodBuilder getterBuilder = typeBuilder.DefineMethod("get_" + propertyName,
var getterBuilder = typeBuilder.DefineMethod("get_" + propertyName,

MethodAttributes.Public | MethodAttributes.SpecialName | MethodAttributes.HideBySig, propType, Type.EmptyTypes);

ILGenerator getterIL = getterBuilder.GetILGenerator();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ILGenerator getterIL = getterBuilder.GetILGenerator();
var getterIL = getterBuilder.GetILGenerator();

getterIL.Emit(OpCodes.Ldarg_0);
getterIL.Emit(OpCodes.Ldfld, fieldBuilder);
getterIL.Emit(OpCodes.Ret);


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple new lines

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

// Define the setter method
MethodBuilder setterBuilder = typeBuilder.DefineMethod("set_" + propertyName,
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
MethodAttributes.Public | MethodAttributes.SpecialName | MethodAttributes.HideBySig, null, new[] { propType });
// Define the setter method
ILGenerator setterIL = setterBuilder.GetILGenerator();
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
setterIL.Emit(OpCodes.Ldarg_0);
setterIL.Emit(OpCodes.Ldarg_1);
setterIL.Emit(OpCodes.Stfld, fieldBuilder);
setterIL.Emit(OpCodes.Ret);

// Set the getter and setter methods for the property
propertyBuilder.SetGetMethod(getterBuilder);
propertyBuilder.SetSetMethod(setterBuilder);
return propertyBuilder;
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
}
}

public class DynamicTypeDef
{
public string NameSpace { get; set; }
public string ClassName { get; set; }
public IEnumerable<DynamicField> Fields { get; set; }
}
public class DynamicField
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add new line

hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
{
public string Name { get; set; }
public Type FieldType { get; set; }
}
}
45 changes: 45 additions & 0 deletions test/YesSql.Tests/Indexes/PropertyIndex.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
using System;
using System.Collections.Generic;
using System.Linq;
using YesSql.Indexes;
using YesSql.Tests.Models;

Expand All @@ -24,6 +27,48 @@ public override void Describe(DescribeContext<Property> context)
IsOccupied = property.IsOccupied,
Location = property.Location
});
}
}

public class PropertyDynamicIndexProvider : IndexProvider<Property>
hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
{
public static Dictionary<string, Type> IndexTypeCache = new Dictionary<string, Type>();
public override void Describe(DescribeContext<Property> context)
{
foreach (var dynamicType in IndexTypeCache.Values)
{
context
.For(dynamicType)
.Map(property =>
{
// It's just a test. We only have one type
var obj = Activator.CreateInstance(dynamicType);
foreach (var prop in dynamicType.GetProperties())
{
switch (prop.Name)
{
case "Name":
prop.SetValue(obj, property.Name);
break;
case "ForRent":
prop.SetValue(obj, property.ForRent);
break;
case "IsOccupied":
prop.SetValue(obj, property.IsOccupied);
break;
case "Location":
prop.SetValue(obj, property.Location);
break;
}
}

return (MapIndex)obj;
});
}


Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Multiple lines



hyzx86 marked this conversation as resolved.
Show resolved Hide resolved
}
}
}