Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PoParser PERF #15943

Merged
merged 9 commits into from
Aug 7, 2024
Merged

PoParser PERF #15943

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4904dca
PoParser PERF
hishamco May 2, 2024
8c22e2e
Fix the unit tests
hishamco May 2, 2024
c35f50a
Merge branch 'main' into hishamco/poparser-perf
hishamco May 3, 2024
fbd7707
Merge branch 'main' into hishamco/poparser-perf
hishamco May 17, 2024
b4bb275
Merge branch 'main' into hishamco/poparser-perf
hishamco Jun 15, 2024
c684e2c
Merge branch 'main' into hishamco/poparser-perf
hishamco Jun 16, 2024
bc043e2
Fix the build
hishamco Jun 16, 2024
2270907
Merge branch 'main' into hishamco/poparser-perf
hishamco Jul 18, 2024
612f311
Merge branch 'main' into hishamco/poparser-perf
hishamco Aug 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/OrchardCore/OrchardCore.Localization.Core/OrchardCore.Localization.Core.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@
<FrameworkReference Include="Microsoft.AspNetCore.App" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="ZString" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\OrchardCore.Localization.Abstractions\OrchardCore.Localization.Abstractions.csproj" />
</ItemGroup>
Expand Down
45 changes: 31 additions & 14 deletions src/OrchardCore/OrchardCore.Localization.Core/PortableObject/PoParser.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
using System;
using System.Collections.Frozen;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using Cysharp.Text;

namespace OrchardCore.Localization.PortableObject
{
Expand All @@ -11,12 +12,17 @@ namespace OrchardCore.Localization.PortableObject
/// </summary>
public class PoParser
{
private static readonly Dictionary<char, char> _escapeTranslations = new()
private static readonly FrozenDictionary<char, char> _escapeTranslations;

static PoParser()
{
{ 'n', '\n' },
{ 'r', '\r' },
{ 't', '\t' },
};
_escapeTranslations = new Dictionary<char, char>()
{
{ 'n', '\n' },
{ 'r', '\r' },
{ 't', '\t' }
}.ToFrozenDictionary();
}

/// <summary>
/// Parses a .po file.
Expand Down Expand Up @@ -55,31 +61,32 @@ public IEnumerable<CultureDictionaryRecord> Parse(TextReader reader)

private static string Unescape(string str)
{
StringBuilder sb = null;
var builder = default(Utf16ValueStringBuilder);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just check for \ in the input string. (IndexOf is fast, it's vectorized). If not return the string. Otherwise create the ZString, check the other usages in Orchard, it needs to be disposed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why we need to check? BTW the disposing in my mind after the tests pass

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is that you check that the builder is initialized in lots of places, to prevent initializing it if there is nothing to unescape.

My suggestion is to first check if there is anything to encode, by looking for \ in the whole string first. If there is not you return, if there is then you create the builder and don't check anymore.

Looking for \ is very fast as I explained.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example:

if (!str.Contains('\\'))
{
    return str;
}

using var builder = ZString.CreateStringBuilder();
... // no more checking, just use builder

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already did that two weeks ago but the few tests were broken, I found the issue and fix it

var escaped = false;
for (var i = 0; i < str.Length; i++)
{
var c = str[i];
if (escaped)
{
if (sb == null)
if (builder.Equals(default(Utf16ValueStringBuilder)))
{
sb = new StringBuilder(str.Length);
builder = ZString.CreateStringBuilder();

if (i > 1)
{
sb.Append(str[..(i - 1)]);
builder.Append(str[..(i - 1)]);
}
}

char unescaped;
if (_escapeTranslations.TryGetValue(c, out unescaped))
{
sb.Append(unescaped);
builder.Append(unescaped);
}
else
{
// General rule: \x ==> x
sb.Append(c);
TryAppend(builder, c);
}
escaped = false;
}
Expand All @@ -91,12 +98,22 @@ private static string Unescape(string str)
}
else
{
sb?.Append(c);
TryAppend(builder, c);
}
}
}

return sb?.ToString() ?? str;
return builder.Equals(default(Utf16ValueStringBuilder))
? str
: builder.ToString();

static void TryAppend(Utf16ValueStringBuilder builder, char c)
{
if (!builder.Equals(default(Utf16ValueStringBuilder)))
{
builder.Append(c);
}
}
}

private static string TrimQuote(string str)
Expand Down
Loading