Disable Selective Execution In --watch On Task Failure

When using the --watch flag in build tools like Mill, a crucial aspect to consider is how task failures are handled. Selective execution, a feature designed to optimize build times by only running tasks dependent on changed files, can lead to unexpected behavior if not managed correctly in the face of task failures. This article delves into the intricacies of selective execution within the --watch mode and why it's essential to disable it when a task fails.

Understanding Selective Execution in --watch

At its core, selective execution in --watch mode is a powerful mechanism for improving build performance. The --watch flag instructs the build tool to monitor the file system for changes and automatically re-run relevant tasks. Selective execution takes this a step further by analyzing task dependencies and only executing tasks that are affected by the changes. This can significantly reduce build times, especially in large projects with numerous tasks. For instance, if you modify a source file in one module, only the tasks related to that module and its dependents need to be re-run, rather than the entire build.

However, the efficiency of selective execution hinges on the assumption that tasks complete successfully. When a task fails, the build process may skip subsequent dependent tasks to terminate early, preventing further errors and saving time. This behavior is generally desirable, as it avoids cascading failures and provides quicker feedback. The problem arises when selective execution is still enabled after a failure. The tasks that were skipped due to the initial failure might never be executed in subsequent --watch cycles if the conditions that triggered the selective execution remain the same. This can lead to a situation where critical parts of the build are perpetually skipped, resulting in an inconsistent or incomplete build.

The Problem with Selective Execution After Failure

Imagine a scenario where you have three tasks: A, B, and C, where B depends on A, and C depends on B. If task A fails during a --watch execution, tasks B and C will be skipped. Now, let's say you fix the issue in the files related to task A and save the changes. With selective execution enabled, the build tool might only re-run task A because it detects changes only in A's source files. Tasks B and C, which were skipped in the previous run and might still be affected by the earlier failure, are not executed. This can leave the system in an inconsistent state, as the outputs of B and C might be outdated or incorrect.

This is a critical issue because it undermines the reliability of the --watch mode. Developers rely on --watch to ensure that their changes are correctly built and tested in an iterative manner. If skipped tasks are not re-run after a failure, developers may be working with a false sense of security, unaware that parts of their application are not being built correctly. The root cause of this problem lies in the fact that selective execution, by its very nature, optimizes for speed under the assumption of success. It doesn't inherently account for the need to re-run skipped tasks after a failure to ensure consistency.

Solutions for Handling Task Failures in --watch

To address this issue, there are two primary approaches that build tools can take:

1. Note Skipped Tasks and Re-run Them: One solution is to keep track of the tasks that were skipped due to a failure. In the subsequent --watch cycle, the build tool should ensure that these skipped tasks are executed, regardless of whether their input files have changed. This approach guarantees that all parts of the build are eventually executed, even after a failure. The complexity here lies in efficiently tracking the skipped tasks and ensuring they are re-run in the correct order, respecting dependencies. This method provides a robust solution by explicitly addressing the potential for skipped tasks to remain unevaluated.
2. Disable Selective Execution After Failure: A simpler and more conservative approach is to disable selective execution entirely after a task failure. In this scenario, the next --watch cycle would execute all tasks, regardless of whether their inputs have changed. This ensures that the entire build is re-evaluated, eliminating the risk of skipped tasks. While this approach might be less efficient in terms of build time, it provides a higher level of assurance that the system is in a consistent state. Disabling selective execution after a failure acts as a safety net, prioritizing correctness over speed in error scenarios.

Both solutions have their trade-offs. The first approach, noting and re-running skipped tasks, is more efficient but requires more complex implementation. The second approach, disabling selective execution after failure, is simpler to implement but might lead to longer build times. The choice between these approaches depends on the specific requirements and priorities of the build tool and the project it is used for.

Implementing the Solutions

Implementing either of these solutions requires modifications to the build tool's core logic. For the