fixing filtered update performance

[viktor-svub]

• update with regex filter, matching 83 packages of 210, resulted in runtime of about 30 minutes, instead of the expected cca 30 seconds of full update
• most of the time was spent in the filter function, constructing and compiling the regex for each nuget package dependency
• issue was completely fixed by extracting the regex object creation to be done only single time

[matthid]

Are you sure this no longer creates regex objects on every access?

[viktor-svub]

pretty much :)
as far I understand F# this should be constructor syntax, filling the Expression member with result of the expression after equals
and I'm seeing the described improvement on my machine, getting filtered update in approximately same time as full one
edit: including Debug build, so it's not just a result of compiler optimization, and it's indeed most likely called only on the filter instance creation

[0x53A]

Decompiled into C#:

		[CompilationMapping(SourceConstructFlags.ObjectType)]
		[Serializable]
		public class PackageMatch
		{
			public PackageMatch(string ex) : this()
			{
				this.ex = ex;
			}

			public Regex Expression
			{
				get
				{
					return new Regex("^" + this.ex + "$", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
				}
			}

			internal string ex;
		}

[0x53A]

I know there is a syntax to initialize it only once, I think it is something with val. Tbh, in that regard F# is really confusing.

[0x53A]

And would it maybe make sense to add ``RegexOptions.Compiled`? https://stackoverflow.com/questions/513412/how-does-regexoptions-compiled-work#7707369

[baronfel]

Here's a sharplab.io example: https://sharplab.io/#v2:DYLgZgzgPg9gDgUwHYAIDKBPCAXBBbAWAChjsNEUBhACgQA8QUcAnASyQHMBKFAXmJQo8+AEYJmKAG4BDYCgCidOMwQQIrGKl4p6KANQoARCOnNDAwYNLkEVAPoAlBAGMYeOAFdctBk2xtOHn4iQWE8MQlsAAtWCAA6RWVVdU0+HTp9IxMzIA===

You can see here that thanks to member val the property value is computed once on construction and then just reused, whereas for C_Recompute the property is recomputed each access.

[baronfel]

Note that we don't have access to System.Text.RegularExpressions in sharplab so that's why I didn't repro your scenario exactly.

[viktor-svub]

Oh my... :) Ok, I'll revise it to create the instance only once for sure.
At this moments it seems possible that all the slowdown was caused just by RegexOptions.Compiled while creating instance per comparison in either case.

[matthid]

Thanks for looking into this, always wondered why this is slower then full update

[viktor-svub]

OK, I have zero idea what happened in the Travis CI ...
Error in '/usr/bin/mono': realloc(): invalid next size: 0x00000000025e0a70

[matthid]

I'm ok with this. Personally I would probably prefer:

type PackageMatch(ex:String) =
    let regex = Regex("^" + ex + "$", RegexOptions.CultureInvariant ||| RegexOptions.IgnoreCase) 
    member _.Expression = regex

or even

type PackageMatch =
   { Expression : Regex }
module PackageMatch =
   let ofString ex =
     { Expression = Regex("^" + ex + "$", RegexOptions.CultureInvariant ||| RegexOptions.IgnoreCase) }