leftOuterJoin improvements

Author: Thorium

docs/RELEASE_NOTES.md

@@ -1,3 +1,9 @@
+### 1.5.25 - 03.07.2026
+* leftOuterJoin: groupBy after a left outer join is now supported (SQL semantics: COUNT counts the unmatched NULL row)
+* groupJoin: `groupJoin ... into g` + `for x in g do` now works (inner join semantics); aggregating the group (e.g. `g.Count()`) fails with a clear message instead of returning wrong results
+* leftOuterJoin detection is now gated on the GroupJoin operator and the LINQ DefaultIfEmpty method, so an unrelated `DefaultIfEmpty` call in a projection can no longer turn an inner join into a left outer join
+* Unmatched left-join rows no longer construct a throwaway entity during materialisation
+
 ### 1.5.24 - 12.06.2026
 * Initial support for standard F# `leftOuterJoin .. into g` / `g.DefaultIfEmpty()` left outer joins (single/multi-key and chained over several tables): unmatched columns are None/default and a whole selected entity is null #540 #697
 

docs/content/core/querying.fsx

@@ -210,7 +210,7 @@ exactlyOneOrDefault      |X |
 exists                   |X |                                                       |
 find                     |X |                                                       |
 groupBy                  |x | Simple support (2) |
-groupJoin                |  |                                                       |
+groupJoin                |x | Only when flattened: `groupJoin .. into g` + `for x in g` (inner join), see below |
 groupValBy	             |  |                                                       |
 head                     |X |                                                       |
 headOrDefault            |X |                                                       |
@@ -441,6 +441,31 @@ let res =
 Alternatively the `!!` operator does an inline left join, but (unlike the form above) the joined
 entity is populated with default values rather than being null when there is no match.
 
+The classic anti-join ("customers without orders") works by filtering the left-joined side with
+`IS NULL`, which under `UseOptionTypes` is an `IsNone` check:
+
+```fsharp
+let customersWithoutOrders =
+    query {
+        for cust in ctx.Main.Customers do
+        leftOuterJoin order in ctx.Main.Orders on (cust.CustomerId = order.CustomerId.Value) into result
+        for order in result.DefaultIfEmpty() do
+        where (order.CustomerId.IsNone)
+        select cust.CustomerId
+    }
+```
+
+Notes and limitations:
+
+* `groupJoin ... into g` followed by `for x in g do` is supported and behaves as an inner join.
+  Aggregating the group itself (e.g. `select (c, g.Count())`) is not supported - use `groupBy`
+  for aggregates.
+* A `groupBy` after a `leftOuterJoin` follows SQL semantics: `COUNT(*)` counts the unmatched
+  NULL row, so a customer with no orders gets count 1, not 0.
+* `DefaultIfEmpty(customDefaultValue)` is not supported; only the parameterless form is.
+* A filtered sub-query is not supported as a left-join destination (the filter would belong in
+  the JOIN's `ON` clause, which is not yet generated).
+
 ## Best practices working with queries
 
 ### When using Option types, check IsSome in where-clauses.

src/SQLProvider.Common/AssemblyInfo.fs

@@ -5,13 +5,13 @@ open System.Reflection
 [<assembly: AssemblyTitleAttribute("SQLProvider.Common")>]
 [<assembly: AssemblyProductAttribute("SQLProvider")>]
 [<assembly: AssemblyDescriptionAttribute("Type provider for SQL database access, common library")>]
-[<assembly: AssemblyVersionAttribute("1.5.24")>]
-[<assembly: AssemblyFileVersionAttribute("1.5.24")>]
+[<assembly: AssemblyVersionAttribute("1.5.25")>]
+[<assembly: AssemblyFileVersionAttribute("1.5.25")>]
 do ()
 
 module internal AssemblyVersionInformation =
     let [<Literal>] AssemblyTitle = "SQLProvider.Common"
     let [<Literal>] AssemblyProduct = "SQLProvider"
     let [<Literal>] AssemblyDescription = "Type provider for SQL database access, common library"
-    let [<Literal>] AssemblyVersion = "1.5.24"
-    let [<Literal>] AssemblyFileVersion = "1.5.24"
+    let [<Literal>] AssemblyVersion = "1.5.25"
+    let [<Literal>] AssemblyFileVersion = "1.5.25"

src/SQLProvider.Common/SqlRuntime.Common.fs

@@ -420,27 +420,31 @@ type SqlEntity(dc: ISqlDataContext, tableName, columns: ColumnLookup, activeColu
     /// Like GetSubTable, but returns null when the aliased table produced no matching row in a
     /// left outer join (i.e. all of its columns are null). Used by the `leftOuterJoin ... into g`
     /// form so the joined result-set entity is null instead of an entity full of default values.
-    /// Note: a real match always has a non-null primary key, so only genuine no-match rows
-    /// (every selected column null) are mapped to null.
+    /// Note: this heuristic is safe because the nullable path is only used for whole-entity
+    /// selects, where every column - including the join key, which is non-null on any match -
+    /// is selected. If partial-column materialisation ever reaches this path, a synthetic
+    /// match-flag column in the SELECT would be needed instead.
     member internal e.GetSubTableNullable(alias:string, tableName) =
-        let sub = e.GetSubTable(alias, tableName)
         // Decide no-match by looking ONLY at columns that are genuinely qualified with this alias.
         // GetSubTable also copies unqualified/foreign columns as a fallback (Utilities.checkPred),
         // which must not count here. checkPred strips the alias prefix, so a column belongs to this
         // alias exactly when the returned key differs from the original (a prefix was removed).
+        // Single pass without intermediate collections; this runs once per materialised row, and
+        // the sub-entity is only built when the row actually matched.
         let pred = Utilities.checkPred alias
-        let ownValues =
-            e.ColumnValues
-            |> Seq.choose (fun (k, v) ->
+        let mutable sawOwnColumn = false
+        let mutable sawValue = false
+        for (k, v) in e.ColumnValues do
+            if not sawValue then
                 match pred (k, v) with
-                | Some(stripped, value) when stripped <> k -> Some value
-                | _ -> None)
-            |> Seq.toList
-        let hasMatch =
-            match ownValues with
-            | [] -> true   // no alias-qualified columns to judge by: keep the entity
-            | _ -> ownValues |> List.exists (fun v -> not (isNull v))
-        if hasMatch then sub else Unchecked.defaultof<SqlEntity>
+                | Some(stripped, _) when stripped <> k ->
+                    sawOwnColumn <- true
+                    if not (isNull v) then sawValue <- true
+                | _ -> ()
+        // No alias-qualified columns to judge by (e.g. the provider returned unqualified column
+        // names): fall back to the populated entity rather than guessing no-match.
+        if sawValue || not sawOwnColumn then e.GetSubTable(alias, tableName)
+        else Unchecked.defaultof<SqlEntity>
 
     /// Maps database entity class to the type provided in generic attribute.
     /// You can define more detailed mapping via MappedColumnAttribute or propertyTypeMapping 
@@ -930,7 +934,27 @@ type GroupResultItems<'key, 'SqlEntity>(keyname:String*String*String*String*Stri
                 let ent = unbox<Microsoft.FSharp.Linq.RuntimeHelpers.AnonymousObject<SqlEntity,SqlEntity,SqlEntity,SqlEntity>> distinctItem
                 Seq.concat [| ent.Item1.ColumnValues; ent.Item2.ColumnValues; ent.Item3.ColumnValues; ent.Item4.ColumnValues; |]
                     |> Seq.distinct |> filterColumnValues
-            | _ -> failwith ("Unknown aggregate item: " + typeof<'SqlEntity>.Name)
+            | other ->
+                // Generic fallback for join-row shapes the fixed cases above don't cover:
+                // a groupBy after a leftOuterJoin / groupJoin carries the flattened group as an
+                // IEnumerable<SqlEntity> slot, and chained joins nest the tuples.
+                let rec collectEntities (o:obj) : SqlEntity seq =
+                    match o with
+                    | null -> Seq.empty
+                    | :? SqlEntity as e -> Seq.singleton e
+                    | :? System.Collections.Generic.IEnumerable<SqlEntity> as es -> es
+                    | o ->
+                        let t = o.GetType()
+                        if t.IsGenericType
+                           && (let g = t.GetGenericTypeDefinition()
+                               g.Name.StartsWith "AnonymousObject" || g.Name.StartsWith "Tuple`") then
+                            t.GetProperties()
+                            |> Seq.filter(fun p -> p.Name.StartsWith "Item")
+                            |> Seq.collect(fun p -> collectEntities (p.GetValue o))
+                        else Seq.empty
+                let ents = collectEntities other |> Seq.toArray
+                if ents.Length = 0 then failwith ("Unknown aggregate item: " + typeof<'SqlEntity>.Name)
+                else ents |> Seq.collect(fun e -> e.ColumnValues) |> Seq.distinct |> filterColumnValues
         let itm = 
             if Seq.isEmpty itms then
                 let cols = (keyname |> fun (x1,x2,x3,x4,x5,x6,x7) -> StringBuilder(x1).Append(" ").Append(x2).Append(" ").Append(x3).Append(" ").Append(x4).Append(" ").Append(x5).Append(" ").Append(x6).Append(" ").Append(x7).ToString()).Trim()