Longing to know

Continuing our examination of building LINQ expressions, let’s dive further into generic methods that can build expressions for any enumerable list of objects. In part one, I showed how to build a simple string expression, but we can make this much more robust.

In post one, the search was based on a simple “contains.” However, this method should support search by multiple values, equality comparison, case-insensitive matches, handle nulls, starts with, ends with, and maybe more.

To facilitate passing these options, I use a “SearchFilter” class.

public class SearchFilter
{
    public string SearchBy { get; set; }
    public string Criteria { get; set; }
    public FilterOperator Operator { get; set; }
    public string Filter { get; set; }
    public bool IncludeNull { get; set; }

    public SearchFilter()
    {
        SearchBy = "ByValue";
    }
}

public enum FilterOperator
{
    Equals,
    EqualTo,
    NotEquals,
    NotEqualTo,
    Contains,
    StartsWith,
    EndsWith,
    Before,
    PriorTo,
    LessThan,
    LessThanOrEqual,
    LessThanOrEqualTo,
    GreaterThan,
    GreaterThanOrEqual,
    GreaterThanOrEqualTo,
    DayOfWeek,
    WeekDay,
    FreeText,
    Default
}

To get the the LINQ Expression, the code is similar to that in part one, but it’s doing a lot more.

/// <summary>
/// Get a string expression and alternatively convert the property to lower and trimming it
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="result"></param>
/// <param name="toLower"></param>
/// <param name="trim"></param>
/// <returns></returns>
private static Expression<Func<T, bool>> GetStringExpression<T>(SearchFilter result, bool toLower = false,
     bool trim = false)
{
    ParameterExpression parameter = Expression.Parameter(typeof(T), "item");
    Expression property = GetPropertyExpression(parameter, result.Criteria);

If we want to search by multiple values, the code treats this as a “Contains” check only.

    if (result.SearchBy == BYMULTIPLEVALUES)
    {
        // Built a loosely typed array
        var splitFilter = result.Filter.Split(new[] { ',', '|' }, StringSplitOptions.RemoveEmptyEntries);
        var containsMethod = typeof(Enumerable).GetMethods().Where(x => x.Name == "Contains")
             .FirstOrDefault(x => x.GetParameters().Length == 2).MakeGenericMethod(property.Type);
        var callContains = Expression.Call(containsMethod, Expression.Constant(splitFilter), property);
        return Expression.Lambda<Func<T, bool>>(callContains, parameter);
    }
    else
    {

Next, you can see that we perform a switch on the filter type. This bit of code is interesting because it’s a mixture of Reflection to get the string methods, and introduces the “AndAlso,” “Constant,” and other expressions from the System.Expressions namespace.

        MethodInfo miOperator;
        var op = result.Operator.ToString().ToLower();

        if (string.IsNullOrWhiteSpace(result.Filter))
        {
            switch (op)
            {
                case "equal to":
                case "equalto":
                case "equals":
                    var nullOrEmptyCheck = Expression.Equal(property, Expression.Constant(string.Empty));
                    nullOrEmptyCheck = Expression.Or(nullOrEmptyCheck, Expression.Equal(property, Expression.Constant(null)));
                    return Expression.Lambda<Func<T, bool>>(nullOrEmptyCheck, parameter);
                default:
                    var notNullOrEmptyCheck = Expression.NotEqual(property, Expression.Constant(string.Empty));
                    notNullOrEmptyCheck = Expression.AndAlso(notNullOrEmptyCheck, Expression.NotEqual(property, Expression.Constant(null)));
                    return Expression.Lambda<Func<T, bool>>(notNullOrEmptyCheck, parameter);
            }
        }
        else
        {
            if (op == "notequalto" || op == "not equals" || op == "not equal to" || op == "notequals")
            {
                // Use these for NotEqual..
                bool nullableProp = IsNullableType(property.Type);
                Expression notEqual = nullableProp ?
                    Expression.Equal(Expression.Property(property, "HasValue"), Expression.Constant(true)) :
                    Expression.NotEqual(parameter, Expression.Constant(null));
                Type nonnullableType = typeof(string);
                notEqual = Expression.AndAlso(notEqual, Expression.NotEqual(nullableProp ?
                     Expression.Convert(property, nonnullableType) :
                     property, Expression.Constant(result.Filter)));
                return Expression.Lambda<Func<T, bool>>(notEqual, parameter);
            }
            else
            {
                switch (op)
                {
                    case "equal to":
                    case "equalto":
                    case "equals":
                        miOperator = typeof(string).GetMethod("Equals", new Type[] { typeof(string) });
                        break;
                    case "contains":
                        miOperator = typeof(string).GetMethod("Contains", new Type[] { typeof(string) });
                        break;
                    case "starts with":
                    case "startswith":
                        miOperator = typeof(string).GetMethod("StartsWith", new Type[] { typeof(string) });
                        break;
                    case "ends with":
                    case "endswith":
                        miOperator = typeof(string).GetMethod("EndsWith", new Type[] { typeof(string) });
                        break;
                    case "fulltext":
                        var stringMethod = typeof(string).GetMethod("IndexOf", new Type[] { typeof(string) });
                        var stringTarget = Expression.Constant(result.Filter);
                        var thisMethod = Expression.Call(property, stringMethod, stringTarget);
                        var equals = Expression.Equal(thisMethod, Expression.Constant(0));
                        return Expression.Lambda<Func<T, bool>>(equals, parameter);
                    default:
                        miOperator = typeof(string).GetMethod("Contains", new Type[] { typeof(string) });
                        break;
                }

After we have our base expression, we build out our final expression. There are a few bits of commented-out code, that illustrate that if you want your code tightly-coupled to EF DB functions, you can include expressions from the DbFunctions class.

                // Add trim and lowercase
                MethodInfo miTrim = typeof(string).GetMethod("Trim", Type.EmptyTypes);
                MethodInfo miLower = typeof(string).GetMethod("ToLower", Type.EmptyTypes);

                // Trim (x.Number.Trim)
                Expression trimMethod = Expression.Call(property, miTrim);
                //// LowerCase (x.Number.Trim.ToLower)
                Expression toLowerMethod = toLower ? Expression.Call(trimMethod, miLower) : trimMethod;
                // The target value ( == "301")
                Expression target = Expression.Constant(result.Filter.ToLower(), typeof(string));

                //// We need to deal with unicode conversion (we don't want it!!)
                //var isVarchar = property.Type.GetCustomAttributes(false).OfType<ColumnAttribute>().Where(q => q.TypeName != null && q.TypeName.Equals("varchar", StringComparison.InvariantCultureIgnoreCase)).Any();
                //var asNonUnicodeMethodInfo = typeof(DbFunctions).GetMethod("AsNonUnicode");                

                //// Now, just update our target
                //target = Expression.Call(asNonUnicodeMethodInfo, target);

                // Continue building the expression tree (x.Number.Trim().ToLower(), "StartsWith", "301")                
                Expression method = trim || toLower ? Expression.Call(toLowerMethod, miOperator, target) :
                    Expression.Call(property, miOperator, target);

                // Build the final expression ( x => x.Number.Trim.ToLower(), "StartsWith", "301" )
                return Expression.Lambda<Func<T, bool>>(method, parameter);
            }
        }
    }
}

One we get the the expression output, we can execute directly against any IQuerable. Pretty neat, imho.

Strings are unique in terms of building expressions. What if, for example, we wanted to build an expression for a numeric property? This is where things get intererstng because we move away from using Reflection and use, primarily, the Expressions library to build out all of our expressions.

private static Expression<Func<T, bool>> GetNumericExpression<T>(SearchFilter searchFilter)
{
    ParameterExpression parameter = Expression.Parameter(typeof(T), "item");
    Expression property = GetPropertyExpression(parameter, searchFilter.Criteria);
    Expression method = Expression.NotEqual(parameter, Expression.Constant(null));

    bool nullableProp = IsNullableType(property.Type) || (property.Type.IsEnum && IsNullableEnum(property.Type));
    Type nonnullableType = null;
    var isOrWithNull = false;

    if (nullableProp)
    {
        if (searchFilter.IncludeNull)
        {
            method = Expression.Equal(Expression.Property(property, "HasValue"), Expression.Constant(false));
            isOrWithNull = true;
        }
        else
        {
            method = Expression.Equal(Expression.Property(property, "HasValue"), Expression.Constant(true));
        }
    }

The above code shows the first difference between comparing non-strings. The Expression libraries expect everything to be typed properly. This also means that we would have a completely different, or even more generic method for generating expressions for other types.

Below, is one exception. If I’m searching by “MULTIPLEVALUES,” then I use reflection to get the Enumerable “Contains” method and call this method against the multiple values, once they are converted to the right type.

    if (searchFilter.SearchBy == BYMULTIPLEVALUES)
    {
        // Built a loosely typed array
        var splitFilter = searchFilter.Filter.Split(new[] { ',', '|' }, StringSplitOptions.RemoveEmptyEntries);
        var listType = typeof(List<>);
        var constructedListType = listType.MakeGenericType(property.Type);
        var instance = Activator.CreateInstance(constructedListType);
        var list = (IList)instance;
        foreach (var filter in splitFilter)
        {
            var parsedValue = ParseNumeric(property.Type, filter, nullableProp, out nonnullableType);
            list.Add(parsedValue);
        }

        var containsMethod = typeof(Enumerable)
             .GetMethods().Where(x => x.Name == "Contains")
             .FirstOrDefault(x => x.GetParameters().Length == 2).MakeGenericMethod(property.Type);
        var callContains = Expression.Call(containsMethod, Expression.Constant(list), nullableProp ? Expression.Convert(property, nonnullableType) : property);
        method = Expression.AndAlso(method, callContains);
    }
    else
    {

Below is a switch/case block similar to the one used for our string expression builder. The difference, though, is the pattern. You’ll also notice that the “string pattern” matching search types are handled by casting the target of the comparison to a string.

The basic pattern is this:

• Get the property expression for the property specified
• Define our comparison expression
• If the target property is nullable, convert it to the equivalent non-nullable type for comparison
• “And” or “Or” the property expression with the comparison expression
• Return the resulting expression
• In our calling code, combine all expressions with a predicate builder

        object parsedFilter = ParseNumeric(property.Type, searchFilter.Filter, nullableProp, out nonnullableType);
        switch (searchFilter.Operator.ToString().ToLower())
        {
            case "contains":
            case "starts with":
            case "startswith":
                var stringMethod = typeof(string).GetMethod(searchFilter.Operator == FilterOperator.Contains ? "Contains" : "StartsWith", new Type[] { typeof(string) });
                var convertToStringMethod = Expression.Call(
                    Expression.Convert(property, typeof(object)),
                    typeof(object).GetMethod("ToString"));
                var stringMethodTarget = Expression.Constant(searchFilter.Filter);
                var comparisonMethod = Expression.Call(convertToStringMethod, stringMethod, stringMethodTarget);
                method = isOrWithNull ? Expression.OrElse(method, comparisonMethod) : Expression.AndAlso(method, comparisonMethod);
                break;
            case "equal to":
            case "equalto":
            case "equals":
                //method = Expression.AndAlso(method, Expression.Equal(nullableProp || property.Type.IsEnum ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter)));
                var equalMethod = Expression.Equal(nullableProp ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter));
                method = isOrWithNull ? Expression.OrElse(method, equalMethod) : Expression.AndAlso(method, equalMethod);
                break;
            case "not equal to":
            case "not equals":
            case "notequalto":
            case "notequals":
                //method = Expression.AndAlso(method, Expression.NotEqual(nullableProp || property.Type.IsEnum ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter)));
                var notEqualMethod = Expression.NotEqual(nullableProp ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter));
                method = isOrWithNull ? Expression.OrElse(method, notEqualMethod) : Expression.AndAlso(method, notEqualMethod);
                break;
            case "less than":
            case "lessthan":
                var lessThanMethod = Expression.LessThan(nullableProp ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter));
                method = isOrWithNull ? Expression.OrElse(method, lessThanMethod) : Expression.AndAlso(method, lessThanMethod);
                break;
            case "less than or equal":
            case "less than or equal to":
            case "lessthanorequal":
            case "lessthanorequalto":
                var lessThanOrEqualMethod = Expression.LessThanOrEqual(nullableProp ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter));
                method = isOrWithNull ? Expression.OrElse(method, lessThanOrEqualMethod) : Expression.AndAlso(method, lessThanOrEqualMethod);
                break;
            case "greater than":
            case "greaterthan":
                var greaterThanMethod = Expression.GreaterThan(nullableProp ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter));
                method = isOrWithNull ? Expression.OrElse(method, greaterThanMethod) : Expression.AndAlso(method, greaterThanMethod);
                break;
            case "greater than or equal":
            case "greater than or equal to":
            case "greaterthanorequal":
            case "greaterthanorequalto":
                var greaterThanOrEqualMethod = Expression.GreaterThanOrEqual(nullableProp ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter));
                method = isOrWithNull ? Expression.OrElse(method, greaterThanOrEqualMethod) : Expression.AndAlso(method, greaterThanOrEqualMethod);
                break;
            default:
                var defaultMethod = Expression.GreaterThanOrEqual(nullableProp ? Expression.Convert(property, nonnullableType) : property, Expression.Constant(parsedFilter));
                method = isOrWithNull ? Expression.OrElse(method, defaultMethod) : Expression.AndAlso(method, defaultMethod);
                break;
        }
    }

    return Expression.Lambda<Func<T, bool>>(method, parameter);
}

static bool IsNullableType(Type t)
{
    return t.IsGenericType && t.GetGenericTypeDefinition() == typeof(Nullable<>);
}

Here’s the simple method that parses the string input to the appropriate type:

private static object ParseNumeric(Type propertyType, string filter, bool isNullableProp, out Type nonnullableType)
{
    object parsedFilter = null;
    nonnullableType = null;

    if (isNullableProp)
    {
        if (propertyType == typeof(int?) || propertyType.IsEnum || IsNullableEnum(propertyType))
        {
            nonnullableType = typeof(int);
            parsedFilter = int.Parse(filter);
        }
        else if (propertyType == typeof(short?))
        {
            nonnullableType = typeof(short);
            parsedFilter = short.Parse(filter);
        }
        else if (propertyType == typeof(long?))
        {
            nonnullableType = typeof(long);
            parsedFilter = long.Parse(filter);
        }
        else if (propertyType == typeof(decimal?))
        {
            nonnullableType = typeof(decimal);
            parsedFilter = decimal.Parse(filter);
        }
        else
        {
            nonnullableType = typeof(int);
            parsedFilter = int.Parse(filter);
        }
    }
    else
    {
        if (propertyType.IsEnum)
        {
            try
            {
                parsedFilter = Enum.ToObject(propertyType, int.Parse(filter));
            }
            catch
            {
                parsedFilter = Enum.ToObject(propertyType, 0);
            }
        }
        else
        {
            try
            {
                parsedFilter = Convert.ChangeType(filter, propertyType);
            }
            catch
            {
                parsedFilter = (propertyType == typeof(long) || nonnullableType == typeof(long)) ? long.MaxValue : int.MaxValue;
            }
        }
    }

    return parsedFilter;
}

Finally, here’s the simple helper method that converts the string property name/reference to an Expression. One nice thing about this method is that it uses an aggregation to handle “dot noted” properties. That is to say, it can return nested property expressions. This can be extremely useful.

static Expression GetPropertyExpression(ParameterExpression parameter, string propertyName)
{
    // Again, check for dot notation - the property expression will differ
    if (propertyName.Contains("."))
    {
        // Check for collections
        var properties = propertyName.Split('.');
        var firstProp = Expression.Property(parameter, properties[0]);
        var isCollection = typeof(List<>).IsAssignableFrom(parameter.Type);

        if (isCollection)
        {
            throw new NotImplementedException("Currently cannnot search across child collections.");
        }

        if (isCollection)
        {
            return properties.Aggregate(parameter, (Expression parent, string path) => Expression.Property(parent, path));
        }
        else
        {
            return properties.Aggregate(parameter, (Expression parent, string path) => Expression.Property(parent, path));
        }
    }
    else
    {
        return Expression.Property(parameter, propertyName);
    }
}