lib/modules: better error for invalid option declarations

[ncfavier]

Make byName aware of whether it's processing options or config to give slightly more accurate error messages.

This presumably would have been useful in nix-community/home-manager#3793 (comment), cc @DamienCassou

with import ./lib;
(evalModules {
  modules = [ {
    options = {
      imports = [ {} ];
    };
  } ];
}).config

error: An option declaration for `imports' has type
       `list' rather than an attribute set.
       Did you mean to define this outside of `options'?

[roberth]

Good find!

[roberth]

This makes me doubt that byName would be a good abstraction.
Inlining byName will likely make the code easier to read and remove a bunch of function calls from the hot path.

Alternatively, you could use attr == "config" instead of introducing another lambda. That way at least performance is unaffected by this PR; not introducing a lambda and corresponding calls.