Trouble with recursive constraints

[jrick]

This may fall under the category of "working as intended, fix your repo layout" and we certainly want to fix it in the future, but at the moment two of our repos have a recursive dependency on each other, and both use the master branch as the constraint for the other. This is causing issues when running dep ensure after modifying the manifest or running ensure -update at any time. Just looking for advice on whether this is considered a bug, if a workaround is possible, or if we need to change the repo layout.

What version of dep are you using (dep version)?

Tested with 0.3.1 and current master (d10af5e).

What dep command did you run?

$ cd $GOPATH/src/github.com/decred/dcrd
$ dep ensure -v -update
Root project is "github.com/decred/dcrd"
 38 transitively valid internal packages
 26 external packages imported from 14 projects
(0)   ✓ select (root)
(1)	? attempt github.com/agl/ed25519 with 2 pkgs; 2 versions to try
(1)	    try github.com/agl/ed25519@278e1ec8e8a6e017cd07577924d6766039146ced
(1)	✓ select github.com/agl/ed25519@278e1ec8e8a6e017cd07577924d6766039146ced w/2 pkgs
(2)	? attempt github.com/btcsuite/go-socks with 1 pkgs; 2 versions to try
(2)	    try github.com/btcsuite/go-socks@master
(2)	✓ select github.com/btcsuite/go-socks@master w/1 pkgs
(3)	? attempt github.com/btcsuite/btclog with 1 pkgs; 4 versions to try
(3)	    try github.com/btcsuite/btclog@master
(3)	✓ select github.com/btcsuite/btclog@master w/1 pkgs
(4)	? attempt github.com/btcsuite/goleveldb with 7 pkgs; 2 versions to try
(4)	    try github.com/btcsuite/goleveldb@master
(4)	✓ select github.com/btcsuite/goleveldb@master w/12 pkgs
(5)	? attempt github.com/btcsuite/snappy-go with 1 pkgs; 3 versions to try
(5)	    try github.com/btcsuite/snappy-go@master
(5)	✓ select github.com/btcsuite/snappy-go@master w/1 pkgs
(6)	? attempt github.com/btcsuite/websocket with 1 pkgs; 3 versions to try
(6)	    try github.com/btcsuite/websocket@master
(6)	✓ select github.com/btcsuite/websocket@master w/1 pkgs
(7)	? attempt github.com/jrick/logrotate with 1 pkgs; 2 versions to try
(7)	    try github.com/jrick/logrotate@master
(7)	✓ select github.com/jrick/logrotate@master w/1 pkgs
(8)	? attempt github.com/decred/dcrutil with 3 pkgs; 18 versions to try
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/[email protected]
(9)	✗   github.com/decred/[email protected] not allowed by constraint master:
(9)	    master from (root)
(8)	    try github.com/decred/dcrutil@master
(8)	✓ select github.com/decred/dcrutil@master w/4 pkgs
(9)	? attempt github.com/davecgh/go-spew with 1 pkgs; 4 versions to try
(9)	    try github.com/davecgh/go-spew@adab96458c51a58dc1783b3335dcce5461522e75
(9)	✓ select github.com/davecgh/go-spew@adab96458c51a58dc1783b3335dcce5461522e75 w/1 pkgs
(10)  ? attempt github.com/decred/dcrrpcclient with 1 pkgs; 18 versions to try
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/[email protected]
(11)  ✗   github.com/decred/[email protected] not allowed by constraint master:
(11)      master from (root)
(10)      try github.com/decred/dcrrpcclient@master
(11)  ✗   github.com/decred/dcrrpcclient@master depends on github.com/decred/dcrd with master, but that's already selected at 
(10)      try github.com/decred/dcrrpcclient@upstream
(11)  ✗   github.com/decred/dcrrpcclient@upstream not allowed by constraint master:
(11)      master from (root)
(10)    ← no more versions of github.com/decred/dcrrpcclient to try; begin backtrack
(9)	← backtrack: no more versions of github.com/davecgh/go-spew to try
(8)	← backtrack: no more versions of github.com/decred/dcrutil to try
(7)	← backtrack: no more versions of github.com/jrick/logrotate to try
(6)	← backtrack: no more versions of github.com/btcsuite/websocket to try
(5)	← backtrack: no more versions of github.com/btcsuite/snappy-go to try
(4)	← backtrack: no more versions of github.com/btcsuite/goleveldb to try
(3)	← backtrack: no more versions of github.com/btcsuite/btclog to try
(2)	← backtrack: no more versions of github.com/btcsuite/go-socks to try
(1)	← backtrack: no more versions of github.com/agl/ed25519 to try
  ✗ solving failed

Solver wall times by segment:
     b-source-exists: 1.281540069s
              b-gmal: 601.990235ms
     b-list-versions: 400.056038ms
         b-list-pkgs: 230.511769ms
         select-root:  10.797913ms
            unselect:   2.386389ms
             satisfy:   1.760699ms
         select-atom:    849.768µs
            new-atom:    453.705µs
          b-pair-rev:    249.013µs
           backtrack:    165.024µs
      b-pair-version:     72.094µs
           b-matches:     59.435µs
  b-deduce-proj-root:     52.604µs
               other:     14.773µs

  TOTAL: 2.530959528s

ensure Solve(): No versions of github.com/decred/dcrrpcclient met constraints:
	v0.8.0: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.7.0: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.2.0: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.1.6: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.1.5: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.1.4: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.1.3: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.1.2: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.1.1: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.1.0: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.0.10: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.0.9: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.0.8: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.0.7: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.0.6: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	v0.0.5: Could not introduce github.com/decred/[email protected], as it is not allowed by constraint master from project github.com/decred/dcrd.
	master: Could not introduce github.com/decred/dcrrpcclient@master, as it has a dependency on github.com/decred/dcrd with constraint master, which does not allow the currently selected version of 
	upstream: Could not introduce github.com/decred/dcrrpcclient@upstream, as it is not allowed by constraint master from project github.com/decred/dcrd.

What did you expect to see?

Before the switch to dep, we used glide, and glide had no issues with the recursive repo layout and each project depending on master. I think this is due to dep being stricter about ensuring that all constraints can be satisfied.

I also tried using an override for the dcrrpcclient repo to force dep to not solve the version, but it still errored during solving.

[jrick]

We have since fixed this issue with our repo layout, but I made a couple of repos that experience a similar issue during dependency solving:

https://github.com/jrick/dep1254a-c
https://github.com/jrick/dep1254b

I am unable to dep init in the dep1254a-c project due to the same problem:

$ dep init -v
Getting direct dependencies...
Checked 3 directories for packages.
Found 1 direct dependencies.
Root project is "github.com/jrick/dep1254a-c"
 2 transitively valid internal packages
 1 external packages imported from 1 projects
(0)   ✓ select (root)
(1)	? attempt github.com/jrick/dep1254b with 1 pkgs; 1 versions to try
(1)	    try github.com/jrick/dep1254b@master
(2)	✗   github.com/jrick/dep1254b@master depends on github.com/jrick/dep1254a-c with master, but that's already selected at 
(1)	  ← no more versions of github.com/jrick/dep1254b to try; begin backtrack
  ✗ solving failed

Solver wall times by segment:
         b-list-pkgs: 381.730348ms
     b-source-exists: 355.366514ms
     b-list-versions: 308.477329ms
              b-gmal:  69.687749ms
            new-atom:    218.895µs
             satisfy:    139.708µs
         select-root:     51.434µs
           b-matches:     13.562µs
          b-pair-rev:      4.594µs
               other:      3.203µs
      b-pair-version:      1.744µs
  b-deduce-proj-root:        651ns

  TOTAL: 1.115695731s

No versions of github.com/jrick/dep1254b met constraints:
	master: Could not introduce github.com/jrick/dep1254b@master, as it has a dependency on github.com/jrick/dep1254a-c with constraint master, which does not allow the currently selected version of 

[sdboyer]

hi! seems like this might be the same underlying issue as #1131? ("recursive" threw me off when i initially read this - "circular/cyclic" is the way i think of it)

[jrick]

Yeah, cyclic is the correct term. Somehow that escaped me when I made this issue and didn't think to search for it. Closing since it's a known/duplicate issue