Jimmer Bug: `childFetcher != Null` Assertion Failure

This article addresses a bug encountered in Jimmer versions 0.9.99 through 0.9.116, specifically when querying ManyToOne properties using JOIN_ALWAYS in conjunction with allTableFields. This issue leads to an assertion failure where childFetcher is unexpectedly null. We will delve into the specifics of the bug, the conditions under which it occurs, and provide a detailed explanation with code examples.

Understanding the Bug

The core of the issue lies in how Jimmer handles ManyToOne relationships when fetched using JOIN_ALWAYS and the allTableFields method. In essence, the system incorrectly identifies ManyToOne attributes fetched via JOIN_ALWAYS with allTableFields as isJoinField. This misidentification results in a scenario where childFetcher is expected to be non-null, but it's actually null, leading to the assertion failure. To put it simply, Jimmer expects a child fetcher to be present for a joined field, but it's not there, causing the program to crash.

The Technical Details

When you use allTableFields, Jimmer attempts to fetch all fields of an entity, including related entities. With JOIN_ALWAYS, it's instructed to always join related tables to fetch the data in a single query. However, in this specific scenario involving ManyToOne relationships, the internal logic within Jimmer seems to falter. The system incorrectly assumes that a childFetcher should be available for these joined fields. A childFetcher is responsible for fetching child entities in a relationship. But because of the way ManyToOne relationships are handled in this context, the childFetcher remains null, leading to the Assertion childFetcher != null failure. This means the application unexpectedly encounters a null value where it anticipates an object, which triggers the assertion and halts execution.

Why This Matters

This bug can significantly impact applications using Jimmer, especially those relying heavily on fetching related data through ManyToOne relationships. The allTableFields method is a convenient way to retrieve all columns of a table and its immediate relations. When this functionality breaks down, developers are forced to resort to more verbose and potentially less efficient ways of fetching data. The failure of this core feature can introduce instability and require developers to implement workarounds, increasing development time and complexity.

Reproducing the Bug

The bug is readily reproducible and appears to be independent of mock setups, as it can be replicated on a real database. The following Kotlin code snippet demonstrates how to trigger the error:

    val sqlClient = newKSqlClient {
        setSqlFormatter(SqlFormatter.PRETTY)
        setExecutor(fakeExecutor)
        setConnectionManager(fakeConnectionManager)
        setDialect(PostgresDialect.INSTANCE)
        setForeignKeyEnabledByDefault(true)
        setDefaultReferenceFetchType(ReferenceFetchType.JOIN_ALWAYS)
    }

    @Test
    fun test___findAllTableFields() {  // this be error
        sqlClient.createQuery(TestEntity::class) {
            select(table.fetchBy {
                allTableFields()
            })
        }.execute()
    }

In this test case, we are creating a query using Jimmer's newKSqlClient. We set the defaultReferenceFetchType to JOIN_ALWAYS, which means that related entities should always be fetched via a join. The test___findAllTableFields function attempts to select all fields of TestEntity using allTableFields. This is where the bug manifests. When the query is executed, the assertion failure occurs because the childFetcher is null for the ManyToOne relationship.

Successful Test Cases (Workarounds)

Interestingly, the following test cases do not exhibit the bug, providing some insight into potential workarounds:

    @Test
    fun test___findAllTableFields_with_table() { // this passed
        sqlClient.createQuery(TestEntity::class) {
            select(table)
        }.execute()
    }

    @Test
    fun test___findAllTableFields_without_allTableFields() { // this passed
        sqlClient.createQuery(TestEntity::class) {
            select(table.fetchBy {
                allScalarFields()
                child()
            })
        }.execute()
    }

The test___findAllTableFields_with_table function simply selects the table itself, which bypasses the issue. The test___findAllTableFields_without_allTableFields function explicitly specifies the fields to fetch, including scalar fields and child entities, but avoids using allTableFields. This suggests that the bug is specifically triggered by the combination of allTableFields and JOIN_ALWAYS in the context of ManyToOne relationships.

Analyzing the Code and Potential Causes

To understand the root cause, let's break down what's happening in the failing test case. When allTableFields is used, Jimmer introspects the entity's properties and generates a fetch tree. This fetch tree dictates how the data will be fetched from the database. In the case of ManyToOne relationships, Jimmer needs to decide whether to fetch the related entity as part of the main query (using a join) or in a separate query. When JOIN_ALWAYS is specified, Jimmer should always opt for the join strategy.

The bug arises because the logic that determines whether a field is a join field (isJoinField) seems to be misidentifying ManyToOne attributes fetched via JOIN_ALWAYS with allTableFields. This misidentification leads to the expectation of a childFetcher, which is responsible for fetching the child entities. However, since the childFetcher is not correctly initialized in this scenario, it remains null, and the assertion fails.

Potential Causes

1. Incorrect Flagging of Join Fields: The most likely cause is an error in the logic that flags fields as join fields. It's possible that the condition for identifying a join field is too broad and incorrectly includes ManyToOne attributes fetched with JOIN_ALWAYS and allTableFields.
2. Initialization Issue with childFetcher: Another possibility is that the childFetcher is not being initialized correctly for ManyToOne relationships when using JOIN_ALWAYS and allTableFields. This could be due to a conditional check that is not properly accounting for this specific scenario.
3. Interaction Between allTableFields and JOIN_ALWAYS: The bug might stem from an unexpected interaction between allTableFields and JOIN_ALWAYS. The allTableFields method might be generating a fetch tree that is not fully compatible with the JOIN_ALWAYS fetch strategy for ManyToOne relationships.

Impact and Mitigation

The impact of this bug is primarily on developers using Jimmer's allTableFields functionality in conjunction with JOIN_ALWAYS for ManyToOne relationships. It can lead to unexpected application crashes and require developers to implement workarounds, such as explicitly specifying the fields to fetch instead of relying on allTableFields.

Mitigation Strategies

1. Avoid allTableFields with JOIN_ALWAYS for ManyToOne: The simplest workaround is to avoid using allTableFields in scenarios involving ManyToOne relationships when JOIN_ALWAYS is enabled. Instead, explicitly specify the fields to fetch, as demonstrated in the successful test case test___findAllTableFields_without_allTableFields.
2. Use Separate Queries: Another approach is to fetch the related entities in separate queries. This can be achieved by not using JOIN_ALWAYS and allowing Jimmer to fetch the related entities lazily or in a separate query when needed.
3. Upgrade Jimmer Version: If a fix for this bug is released in a newer version of Jimmer, upgrading to that version would be the most direct solution. Check the Jimmer release notes for information on bug fixes and updates.

Conclusion

The assertion failure caused by a null childFetcher when querying ManyToOne properties using JOIN_ALWAYS and allTableFields is a significant bug in Jimmer versions 0.9.99 through 0.9.116. It highlights the complexities of object-relational mapping and the importance of thorough testing, especially when dealing with relationships between entities. By understanding the conditions under which this bug occurs, developers can implement appropriate workarounds and mitigate its impact. We encourage the Jimmer development team to address this issue in future releases to ensure the stability and reliability of the framework.

For further information on Jimmer and its features, you can refer to the official Jimmer documentation.