It would be fantastic to have a permission to avoid users checking out the document when it has passed the Draft/WIP status, while allowing them to open/download the document. This would reduce administrative burdens for Document Managers taking care of accidental checkouts in the Datasource.
This functionality already exists. Dynamic (workflow state) permissions are central to ISO 19650 and other less structured workflows.
You can leverage this with or without the use of Workflow Rules Engine.
I may be misunderstanding you, but could revoking the "File Write" permission resolve this?
Users only require "File Read" to open the document read-only. "File Write" is what enables check out/check in.